Artificial intelligence (AI) has the potential to transform healthcare, from speeding up diagnoses to supporting personalised treatment plans. But before this promise can be fully realised, AI as medical devices (AIaMD) must be effectively regulated. Unlike traditional medical devices, AI raises unique challenges, for example, models often operate as “black boxes”, they can evolve significantly after approval, and they risk introducing bias or fostering overreliance by clinicians.
In this fast-moving area, the UK, led by the Medicines and Healthcare products Regulatory Agency (MHRA), is positioning itself as a leader in the regulation of AI in healthcare.
Regulation in the UK vs the EU
In the EU, AIaMD will be regulated through a combination of existing medical device regulations and the new EU AI Act. It has been suggested that this approach risks creating inconsistencies. By contrast, the UK is considering building specific provisions for AIaMD and software as a medical device (SaMD) directly into UK Medical Device Regulation, creating a more streamlined, self-contained framework.
The MHRA’s AI Airlock
To help shape its approach, the MHRA launched the AI Airlock in May 2024. This pre-market pilot programme allows companies to test AI medical devices in a controlled environment, helping to identify regulatory gaps and potential solutions.
The first cohort included four companies, each exploring a different aspect of AIaMD and a different regulatory challenge:
- Philips Medical Systems – Generative AI for radiology reports
- Newton’s Tree – Real-time monitoring of AI tools
- OncoFlow – Personalised cancer treatment planning
- AutoMedica – Large language models (LLMs) for clinician use
Reports from this first round are expected in autumn 2025, with a second cohort of nine technologies already in the selection phase.
Spotlight on the Participants
Philips Medical Systems – Use of Synthetic Data
Phillips Medical Systems are developing a generative AI system for use by radiologists. When radiologists review patient results and write their findings, the system creates a ‘Patient Impression’ section of the Radiology report, generated from the radiologist’s findings, in which the most important information for the referring physician is highlighted. Notably, they plan to test it using synthetic data, which is computer-generated data designed to mimic real patient records.
Synthetic data offers clear advantages, such as reducing privacy concerns and providing a broader range of training examples. However, it also carries risks such as potential model collapse caused by long-term reliance on synthetic data, and poorly designed data introducing bias. Philips’ involvement in the Airlock focuses on testing how synthetic data can be safely integrated into healthcare AI. With the potential benefits and risks of synthetic data, clear and effective regulation, as well as real time monitoring of model drift, will be essential to realising the potential of the use of synthetic data in AI models in the healthcare setting.
Newton’s Tree – Monitoring AI in Practice
Newton’s Tree is developing a monitoring service that tracks how AI tools perform once deployed. It analyses input data, model outputs, and usage patterns to detect drift or performance issues in real time. This allows for the continual assessment of an AI tool’s performance even after product roll-out. Importantly, the system runs on local hardware, reducing the need to share sensitive patient data externally.
Newton’s Tree’s participation in the airlock aims to “investigate how using a monitoring system can improve a product’s risk management by identifying performance and safety issues in real-time”. With the potential for the performance of AI tools to change significantly as they are used, for example with changing patient demographics, such monitoring could play a critical role in AIaMD regulation. Not only could this provide enhanced patient safety, but it could also increase confidence in AI tools, enabling a more streamlined roll-out.
OncoFlow – Faster, Personalised Cancer Care
OncoFlow aims to support clinicians in creating personalised cancer treatment plans. The technology could reduce delays in oncology pathways, allowing patients to begin treatment sooner and potentially improving survival rates. OncoFlow’s initial focus is on breast cancer, given both the high prevalence of cases and the considerable delays often experienced in patient pathways. In the longer term, however, the platform has been designed with adaptability in mind and can be extended to support a wide range of other cancer types.
Co-founder Aruni Ghose said the Airlock programme provided his team with the chance to validate the product in a simulated clinical setting and “pressure-test it against real regulatory standards” which has helped the company accelerate its progress “from idea to a validated MVP (Minimum Viable Product)”.
AutoMedica – Tackling the “Black Box” Problem of LLMs
AutoMedica is developing SmartGuideline, a large language model (LLM) trained specifically on National Institute of Health and Care Excellence (NICE) resources. LLMs are ‘black boxes’, meaning their internal processing and reasoning is opaque. Furthermore, they can ‘hallucinate’, presenting false information as if it were true.
By using retrieval-augmented generation (RAG) technology, SmartGuideline grounds its responses in an authoritative, verified knowledge base, aiming to improve accuracy and reduce errors. The Airlock will test whether this approach could make LLMs easier to regulate.
Regulation, IP, and the Future of AI in Healthcare
The MHRA’s Airlock demonstrates both the opportunities and challenges of regulating AI in healthcare and we eagerly await the full reports on the first cohort.
Beyond regulation, for innovators, intellectual property (IP) strategy is also vital. Developing AI medical technologies requires significant investment. Patents not only safeguard these innovations but can also establish clarity around ownership, licensing, and collaboration. For AIaMD, innovators should obtain robust patent portfolios, enabling them to maintain their market advantage, even as regulatory requirements change. As the UK carves out a leadership role in AI medical device regulation, innovators should look beyond compliance. Robust patent protection will be critical in turning regulatory approval into lasting market success.
.png)
-1.png)
.png)