A couple of notes before you get into this article.
This month I’m presenting the first in a series of talks on AI in Medicine with my friends at EB Medicine. I encourage you to join me for the first session: “Same Patient, Two Answers: How AI Gets Your Patient Wrong”. Registration is free, so join me and bring your questions!
I also had the pleasure of being a guest on the Stimulus podcast with my friend, Dr. Rob Orman, to discuss some traps in clinical AI. You can listen to it here or in your favorite podcast app.
As always, thanks for reading Ashoo Review: AI in Medicine. If you enjoy it, consider subscribing and telling a friend.
Sam
In today’s article, we will focus on two different approaches to implementing AI tools. This month, both tools were in the news and had opposing adoption stories. The first tool received FDA authorization and was placed in ambulances within a month. The second tool has been on the market for years, but a major health system still tested it on 3,856 CT scans before using it on a patient. That second story illustrates how adoption should work in your health system.
Queen of Hearts
The current AHA STEMI criteria can miss occluded arteries. Every emergency physician knows a patient whose first ECG showed no ST elevation, whose troponin came back positive, and whose cath hours later showed a closed vessel. Occlusion MI without ST elevation is the blind spot the OMI paradigm was created for in 2018, and one the Queen of Hearts AI ECG reader was built to solve.
Queen of Hearts, formerly the STEMI AI ECG Model in Powerful Medical’s PMcardio platform, reads a standard 12-lead ECG and flags findings suggestive of acute coronary occlusion, including STEMI equivalents. The FDA granted it De Novo authorization on September 3, 2026, creating a new classification for AI ECG models aimed at life-threatening diagnoses. Fewer than ten AI devices a year arrive by the De Novo pathway, which is intended for applicants with no similar approved devices to copy. The FDA approval arrived after Breakthrough Device Designation in March 2025 and includes an authorized Predetermined Change Control Plan, which lets the company modify the model after authorization without a new submission (more on this process here).
The evidence is a multicenter US registry published in JACC: Cardiovascular Interventions. Investigators took 1,032 patients who had emergent cath-lab activation at three primary PCI centers between January 2020 and May 2024 and ran the first ECGs through the model afterwards, blinded. Angiography and biomarkers confirmed 601 STEMIs, leaving 431 false-positive activations. The model caught 553 of the 601 on the first ECG. Standard triage caught 427. False-positive activations were 7.9% for the model versus 41.8% for standard triage. The paper was presented at Transcatheter Cardiovascular Therapeutics, the Cardiovascular Research Foundation's annual meeting, and won the journal’s Spencer King Award.
Those results are impressive. But to interpret them correctly, we have to keep three things in mind:
The study is retrospective. The model was graded on ECGs from cases with known outcomes, and it was never prospectively trialed.
The population was pre-selected. Every patient in the study already had cath lab activation and a known coronary occlusion, which is a very different population than the undifferentiated chest pain patients in your ED waiting room.
The lead author, Robert Herman, co-founded Powerful Medical and serves as its chief medical officer. Stephen Smith, whose ECG teaching many of us trained on, helped develop the model and co-authored the paper. Both have a vested interest, which does not make the numbers false, but it does mean the sellers graded the test, which can introduce bias.
Despite all that, the editor-in-chief of JACC, Mohamed Alkhouli, a Mayo Clinic cardiologist, praised the work with one caveat: The model was built to detect occluded arteries rather than STEMI. It needs prospective validation across a diverse patient population.
Twenty-seven days after authorization, Tanner Health announced it had gone live with the service in all of its ambulances, emergency departments, and cath labs in west Georgia and east Alabama. Queen of Hearts is also connected to its Epic and EMS systems. The announcement doesn’t name a validation period. The AI is live and informing clinical decisions on real patients from day one.
A day later, Powerful Medical announced a partnership with Corpuls to build Queen of Hearts into the defibrillators that ride in ambulances in more than 70 countries, starting in Europe. To be fair, the model has a European track record. It has carried a CE mark since 2022. Smith describes years of his work with H. Pendell Meyers, and the company reports more than 120,000 heart attacks detected in Europe during 2025. A prospective European study, AI ECG TIMI, has enrolled 703 patients at nine sites in Belgium, Italy, and Austria. Its results are not yet published.
The important thing to keep in mind is that a retrospective review, along with company-reported data on a European population, is not sufficient evidence for a U.S.-based implementation.
Aidoc at Northwell
AI Doc is a radiology assistant tool built to detect intracranial aneurysms and flag them for radiologists. It was cleared by the FDA in April of 2022 under a 510(k). The indication is to triage and notify, meaning it flags suspected cases and moves them up the worklist for the radiologist to read.
Despite the FDA clearance, in 2023 Northwell Health System began its own study of the product. It used it on 3,856 consecutive brain CTAs from its inpatient, emergency, and outpatient populations. It was used in shadow mode, meaning it could interpret the images, but radiologists still read every study blinded to the AI system’s output. Neuroradiologists adjudicated any disagreements between the radiologist and the AI software, and the entire study was published in the Journal of the American College of Radiology (September, 2026)
The AI found 55 aneurysms that the radiologists had missed, a 39% relative increase in detection. Sensitivity was 84.6% for the AI versus 71.8% for the radiologists. By setting, inpatients gained 18 true detections against 7 false alerts, a ratio of 2.57. Emergency cases fared similarly. Outpatients did not. Northwell published many operational measures (relative enhanced detection rate, gain-to-pain, number-needed-to-examine) that most AI papers omit.
The paper also included costs. Radiologists were right far more often, with a positive predictive value of 92.7% versus the AI’s 78.2%. Radiologists also caught 30 aneurysms the AI missed. Of 101 findings the AI flagged alone, 46 were false. In outpatients, false alerts outnumbered true finds. The AI’s catches were small, most under 3 mm. One year of follow-up on its 55 findings produced 40 clinic visits, 40 added imaging studies, 12 diagnostic angiograms, and 2 preventive operations. Whether those 55 patients are better off is unknown, and something the paper acknowledges.
The study is fascinating. Not because of its construction, but because it was performed by a health system that bought a product and then tested it before deployment. However, they were not the only ones to test it on their own patients.
A University of Washington group ran the same Aidoc tool on 2,534 CTAs from 2018 to 2021 and compared it with fellowship-trained neuroradiologists. AI sensitivity was 70.5% compared to neuroradiologists at 94.0%. Same vendor, same product, opposite result. That tells us that a model’s performance isn’t fixed at the time of FDA clearance. It changes depending on where the model runs and who overreads it. Northwell’s method (measure it here, on our patients, before it counts) is the only way a buyer learns which version of the tool they own.
Two definitions of ready
A 510(k) clearance says a device resembles something on the market. A De Novo grant says a device can be sold as a novel product. Neither says the device performs well in your ED, on your patients, inside your workflow. FDA clearance is a regulatory decision built on evidence a manufacturer chose to submit. Local validation is a clinical decision by the institution that deploys the tool.
Queen of Hearts was authorized on a retrospective registry and was in ambulances within the month, deployed by its first US customer as routine practice. Aidoc, cleared in 2022, still had to pass a prospective, blinded run of 3,856 scans at Northwell before anyone there relied on it.
That local validation raises another question: how long does the validation last? Queen of Hearts has an FDA-authorized Predetermined Change Control Plan that allows the model to change after authorization. The version a hospital validates today may not be the same version it is using a year from now. Hospitals need to know when a model changes enough to require another look at its performance, which makes validation an ongoing responsibility.
The standard
Before an AI tool influences a decision in your hospital, run it prospectively in your environment, with its output hidden from the clinicians. Then record the operational numbers by setting and deploy it only where the evidence favors patients. Northwell published a template any large system can copy, including the setting split.
Until buyers demand that validation as a condition of purchase, here are five questions that should be asked:
Was this validated prospectively, or graded on old cases with known outcomes?
Was the validation population my population, or patients pre-selected because the answer was known?
Who ran the validation, and what do they earn if I buy?
What does a false positive cost my workflow: activations, callbacks, scans, sleep?
What can change inside this model after purchase, and how will I know when it does?
Queen of Hearts may prove to be everything its registry suggests. The occlusion problem is serious, the cardiology behind the model is real, and confirmed prospective numbers would mean saved myocardium at scale. The Aidoc numbers may be as good as the tool gets, and its outpatient results may be the ones other hospitals pay attention to when deciding where to use it. Nobody knows yet, including the vendors. What separates the two stories is institutional behavior.
FDA clearance is permission to sell. It has never been proof that a tool works on your patients.