1. Most clinical prediction models never reach independent validation
October 30 | Benchmarks & Evaluations
A systematic review in JAMIA tracked 1,072 machine learning clinical prediction models originally published from 2017 through 2019, examining subsequent citations through August 2025. Only 9.9% underwent external validation, and 5.2% reached clinical implementation. Among implemented models, 39% had no documented prior external validation. Publicly available code and better reporting were associated with validation, although neither consistently predicted implementation. The study describes the published literature, so undocumented local validation may have been missed.
My Take: The 39% figure deserves attention from hospital AI governance committees. Before approving deployment, ask for independent validation in a relevant population and documentation of what happened when the model was tested in its intended workflow.
2. NHS claims of AI-platform benefits face a hospital-level reality check
November 2 | Governance & Risk / Clinical Workflow
An investigation by The Guardian, drawing on more than 100 freedom-of-information requests obtained by Medact, found that 29 NHS hospital trusts listed as benefiting from Palantir’s Federated Data Platform said the system was not live or that they were not using its applications. NHS England said some trusts receive benefits from data submission and administrative functions. Palantir cited improved surgical capacity and discharge performance, while NHS England has acknowledged that its figures do not establish causation.
My Take: This is evidence that a deployment count is a poor substitute for measuring use and outcomes. Evaluation should distinguish deployment, active use, and measurable benefit.
3. New reporting examines Epic’s control over access to clinical data
October 28, newly identified | Governance & Risk / Clinical Workflow
In an original STAT investigation, reporters interviewed more than 50 people about Epic’s relationships with competing healthcare software vendors. Executives from 12 companies alleged difficulties obtaining data interfaces or technical specifications, among other barriers. Epic disputed the implication that it restricts access unfairly and said it provides more data connections than federal law requires. The reported concerns could affect third-party AI products seeking access to patient information and EHR workflows. STAT’s full article requires a subscription.
My Take: Integration is part of an AI product’s clinical usefulness. Hospitals need a method to compare alternatives. Read more on this topic the Ashoo Review article, The CIO’s Impossible Comparison, on evaluating native EHR AI against outside vendors.
4. FDA and CMS confront overlapping oversight of autonomous clinical AI
October 30 | Governance & Risk
STAT reported that federal officials are working through potential jurisdictional conflicts as states create pathways to test AI systems performing clinical tasks. States regulate medical practice and professional licensing, while the FDA oversees medical devices. Systems that make treatment decisions or prescribe with limited physician involvement may expose gaps between those authorities. Detailed reporting is behind STAT’s subscription paywall.
My Take: A state-sponsored pilot needs an explicit answer to who oversees the system, what evidence is required, and who can stop deployment when performance deteriorates. Those responsibilities should be established before patients enter a pilot.