Crown Commercial Service explains how to effectively procure artificial intelligence, with its top 5 guidelines for public sector AI procurement
CCS has helped to establish procurement guidelines to help UK public sector organisations maximise impact when adopting artificial intelligence (AI) technologies, while promoting innovation and operational efficiency.
The UK Government’s AI Opportunities Action Plan encourages acceleration of AI adoption across the UK, with the aim of boosting economic growth, providing jobs for the future and improving people’s everyday lives. The public sector is expected to play a crucial role in implementing this strategy.
The wide range of opportunities presented by AI have transformative potential for the UK public sector, but with these possibilities comes an increased need for clarity, responsibility and accountability. This starts with the procurement of AI as a public sector buyer. It is essential that foundations are put in place during the AI procurement process to ensure systems operate reliably and transparently.
5 key guidelines for public sector AI procurement
There are a number of key guidelines which should be applicable to all public sector buyers, not just those in government, who are considering adopting AI.
-
Know what different forms of AI there are, and their limits
Mainstream AI tools like ChatGPT are generative AI, creating new content from existing knowledge based on user prompts.
Other AI forms could fulfil different public sector purposes. For example, predictive AI uses current information to forecast outcomes, helping NHS trusts anticipate seasonal demand or model environmental impacts of development.
Another form is agentic AI, which operates autonomously with minimal user input, potentially automating document verification or creating virtual assistants for online applications or processes.
All AI tools have limitations, such as dependence on existing information. Evaluation should assess both capabilities and limitations, as understanding AI varieties helps organisations make appropriate procurement decisions.
Furthermore, although AI models are becoming increasingly successful at solving problems and saving organisations time and money, AI may not always be the best solution, particularly where integration with existing networks is needed. Buyers should have a clear understanding of the problem they need to solve before considering AI solutions, and remain open to alternatives.
-
Understand the benefits and risks of AI
Any procurement decision made by public sector organisations should have public interest and benefit at its heart. Any AI procurement should be guided by the desired public benefit, and supported by how an AI system can achieve it.
Buyers should also understand the risks AI can pose in a public sector environment. These could include a lack of the necessary digital skills to manage AI effectively, the impact of outdated legacy technology, or data privacy concerns, which could have significant consequences for people reliant on public services.
For example, difficulties integrating an AI system with outdated technology could create data gaps in NHS patient records, resulting in inconsistent recommendations and treatment delays, or cause problems processing and issuing vital benefit payments.
AI hallucinations, where systems generate misleading or inaccurate results by misunderstanding information or incorrectly correlating data, also represent a consistent risk that must be carefully managed, with human oversight built into the planning process.
Assessing the impact AI adoption is likely to have at the start of a procurement can help identify these risks early and allow them to be accounted for as part of the process.
-
Understand how to use AI lawfully, ethically and responsibly
AI offers unprecedented access to data and information, necessitating careful consideration of how users apply it.
This means, for example, buyers ensuring AI suppliers adhere robustly to data protection law, protecting intellectual property and copyright.
They should also be mindful of unconscious or societal biases and discrimination being fed into AI systems by historical data. AI buyers should understand how any system has been tested by its developer to minimise or eliminate these.
AI systems should be transparent and explainable – a requirement of the government’s AI procurement guidelines to maintain public trust.
-
Engage with the market early
Engagement with a diverse range of AI suppliers early in the process can help form more targeted and appropriate solutions right from the start, increasing the likelihood of a successful procurement and better project delivery.
Early engagement will also help ensure your AI requirements are kept proportionate to the size of the project, avoiding unnecessary resource intensity. Simpler requirements also make it easier for small and medium-sized enterprises (SMEs) and Voluntary, Community and Social Enterprise (VCSE) suppliers, which can potentially offer more niche or even bespoke solutions, to compete for contracts.
Buyers should also seek advice from commercial professionals within their organisations as part of this exercise to understand how to align AI to commercial requirements.
-
Have a plan to manage the whole AI lifecycle
When planning procurements involving AI, organisations must be sure they can continue to support their AI solutions across the longer term.
As AI is a long-term investment in many cases, it is important to have robust plans in place for ongoing support and maintenance of AI systems, as well as meaningful evaluation and ways to feed back any findings or new information into the cycle to inform future decisions about continuing to use AI.
Find out more
CCS has supported and co-authored the new AI Playbook for Government, which offers guidance on using AI safely, effectively and securely for civil servants and people working in government organisations.
For more information, you can also download our buyer’s guide to AI.
© Crown copyright
