Missing Pieces: Transparency and Accountability in Public Body Automated (and Automated Assisted) Decision-making

Missing Pieces: People left in the dark as public bodies turn to AI and algorithms to inform decisions about their lives

Read the report here.

People are unable to scrutinise or challenge decisions that affect their rights because current mechanisms for finding information about the use of artificial intelligence (AI) and automated decision-making by public bodies are failing, finds new research by PLP.

Public bodies across the UK are increasingly using automated tools, including AI, to make or support important decisions about people’s lives such as social security, tax or adult social care services. Yet those affected often cannot find out whether a tool was used in their case, let alone how it works or whether it treated them fairly.

Missing Pieces: Transparency and Accountability in Public Body Automated (and Automated Assisted) Decision-making, funded by the Nuffield Foundation, finds a patchwork of transparency mechanisms that both individually and collectively fail to provide people with the information they need, in the time they require it. The report sets out how to close these gaps and turn the government’s commitment to transparency, accountability, and contestability into reality.

The report is published amid growing calls for legislative reform on AI, including from the Joint Committee on Human Rights.

Key findings include:

People are rarely told whether a tool has been used to make a decision. As a result, they are unable to effectively contest the use of tools or decisions made using them.

High-risk systems are missing from the government’s own register. Of 15 automated systems identified in the research that are believed to still be in use, only four appear on the government’s public register of algorithmic tools, the Algorithmic Transparency Recording Standard (ATRS), even though several fall within its mandatory scope.

Information about the risk of discrimination is especially thin. Of the 125 ATRS entries reviewed, researchers found only one linked to an Equality Impact Assessment (the document public bodies use to evaluate the risks of discrimination posed by a policy or decision).

Public bodies often withhold information under Freedom of Information exemptions, most commonly on law enforcement grounds. Other challenges include information withheld due to concerns about commercial confidentiality.

Notification requirements under data protection law are inadequate: current provisions under data protection law draw a binary distinction between decisions which are “solely” automated and those where there is a “human-in-the-loop”. Rights to be notified and informed do not apply where AI or automation is used to shape, inform or assist decisions made about them.

People working in public bodies support transparency but face barriers. Public body participants, without exception, were supportive of transparency but identified challenges to meeting requirements in practice. These include lack of technical capacity, concerns about public trust and difficulty obtaining information from third-party suppliers.

Information gaps create a “vicious circle” in judicial review litigation. Without enough information to assess the merits of a case, claimants find it too risky to issue a claim so never reach the stage when full disclosure is required.

What needs to happen?

The report concludes that the legal and regulatory framework for an increasingly automated state cannot be left to evolve through the courts alone. PLP recommends that:

Governance, engagement and design processes are informed by public law considerations from the outset.

The ATRS be placed on a statutory footing, to strengthen monitoring, compliance and enforcement.

A legal requirement is introduced that requires people to be notified when and how automated systems have been used to support individual decisions about them, with enforceable individual rights to information about those systems.