Lords debate government approach to automated decision-making
During the second reading debate of a Lords private members’ bill seeking to better regulate public sector algorithms, the government argued existing measures are sufficient to protect the public
The UK government has expressed “reservations” about legislative proposals from Lord Tim Clement-Jones to improve the scrutiny of algorithmic decision-making tools in the public sector, arguing the concerns raised by his bill are already covered by its own proposed data reforms and the existing Algorithmic Transparency Recording Standard (ATRS).
Introduced as a private members’ bill in September 2024, Clement-Jones said at the time the proposals were needed because the existing safeguards on algorithmic and automated decision-making (ADM) tools were not enough to ensure redress once the “computer says no”.
During the second reading debate of his bill in December 2024, Maggie Jones – the under-secretary of state at the Department for Science, Innovation and Technology (DSIT) – said the combination of government’s reforms to make the ATRS mandatory for central government departments, as well as its proposed Data Use and Access Bill (DUAB), are not only sufficient to address Clement-Jones’ concerns, but also better align with the government’s own “priorities of accelerating innovation, technology for good, and modern digital government”.
“These reforms strike the right balance between ensuring that organisations can make the best use of automated decision-making technology to support economic growth, productivity and service delivery, while maintaining high data protection standards and public trust,” she said.
However, Clement-Jones expressed his own reservations about the government data reforms, citing past incidents involving the use of ADM tools to describe the “dangers of unchecked algorithm systems” – including lack of transparency, loss of public trust in artificial intelligence (AI), and infiltration of bias and racism in unregulated decision-making systems.
He reminded Lords of the 2020 A-level and GCSE grading fiasco, where students unfairly missed out on university places; the use of ADM systems by councils, where 540,000 citizens were assigned “fraud” risk scores before refusing them of housing benefits on the basis of fraudulent data; and the Post Office Horizon scandal. “It is a dangerous confidence if the government really thinks that the ATRS, combined with the watered-down ADM provisions in the GDPR [General Data Protection Regulation], are going to be enough,” said Clement-Jones. “The bill is crucial to ensuring that the benefits of these technologies are realised while safeguarding democratic values and individual rights.”
Baroness Jones, however, argued that the government’s data reforms under the DUAB “specifically provide that human involvement must be meaningful. This is to prevent cursory human involvement being used to rubber-stamp decisions as having had meaningful involvement.”
Transparency concerns
One of Clement-Jones’ major concerns was about the transparency of ATRS. Since its launch in 2022, he said, only six records of algorithmic transparency in the public sector have been published.
Government defended itself by claiming that “a number of these records have been completed” since it was made mandatory for all government departments, and that those will soon be published.
As it stands, there are currently 23 records contained in the ATRS.
Clement-Jones’ bill also contains a section on Algorithmic Impact Assessments (AIA) that public authorities would have to conduct prior to the deployment of any decision-making tools. AIAs would make it mandatory for them to constantly assess and disclose the tool’s impact on safety and administrative decision-making when the algorithm is updated or the tools’ scope changes.
Conservative peer Viscount Camrose – Jones’ predecessor at DSIT – further argued that enforcing impact assessments even before purchase would be “unrealistic”, and, as a result, the bill would create an “administrative burden”. He also criticised Clement-Jones’ bill for being too prescriptive, rather than allowing public sector the flexibility to make choices in the face of their “wildly differing needs”.
Baroness Jones claimed the ATRS provided a holistic view on safeguards regarding the impact of the tool while avoiding negative outcomes, and that legislating for transparency was therefore not needed: “We do not believe that legislation for either mandatory transparency records or AIAs for public authorities is necessary at this time,” she said.
Read more about automated decision-making
- Swedish authorities urged to discontinue AI welfare system: Amnesty International is calling on Sweden’s social insurance agency to immediately discontinue its machine learning-based welfare system, following an investigation by Lighthouse Reports and Svenska Dagbladet that found it to be discriminatory.
- Denmark’s AI-powered welfare system fuels mass surveillance: Research reveals the automated tools used to flag individuals for benefit fraud violate individuals’ privacy and risk discriminating against marginalised groups.
- AI disempowers logistics workers while intensifying their work: Conversations on the algorithmic management of work largely revolve around unproven claims about productivity gains or job losses – less attention is paid to how AI and automation negatively affect low-paid workers.
References were also made to Canada, which has recently implemented a similar AIA framework to the one proposed. Martha Lane Fox requested caution in the implementation of Clement-Jones’ bill, saying it might add to bureaucracy. “Canada has also been trying to move to greater regulation of algorithmic transparency, and the implementation has been very heavy and difficult,” she said.
One key distinction of the private members’ bill is that it raises the importance of scrutinising systems during procurement stage – instead of after they are put to use – to ensure that systems are properly assessed for their efficacy and impacts before they are purchased.
Speaking in support of improved procurement measures, Lane-Fox said: “The skills on the digital procurement side of the civil service are under-egged, and the deals done with suppliers are far from ideal as we move to a world in which we want to encourage innovation but must also encourage safety. I very much hope that procurement will be positioned very closely at the heart of any future plans.”
The private bill is introduced by Clement-Jones at a time when government is looking to use AI to drive growth and boost the economy. Peter Kyle previously said in July 2024 that “we’re putting AI at the heart of the government’s agenda to boost growth and improve our public services”.
Meanwhile, prime minister Keir Starmer said during his AI Action Plan speech on 13 January that the technology is the “defining opportunity” of our generation. “Mark my words, Britain will be one of the great AI superpowers,” he said.
Originally published at ECT News
