When Does an Algorithm Become AI?
One of the struggles I hear about time and time again from hospital IT leaders and clinicians is the frustration over what qualifies as artificial intelligence. It feels like every new piece of software has an “AI” feature. How are we supposed to keep track of all these tools if we can’t even agree on a definition?
One method is to narrow the scope of that definition by focusing on whether or not a product uses machine learning. That framing allows us to differentiate “AI” from our other clinical decision support tools.
Three examples:
An ECG machine interprets a tracing and labels it as “Atrial fibrillation with rapid ventricular response”.
A sepsis detection tool connected to the EHR identifies a patient as high risk.
Radiology assistant software labels a chest X-ray with “Pneumothorax identified”.
In all three cases, machines are helping physicians make medical decisions by drawing our attention to problems. Computers have been assisting physicians in this way for decades before the rise of “AI” as we know it. So what changed?
What Does It Mean for a Machine to “Learn”?
Historically, software was given an algorithm, or a set of rules, to reach an answer. Machine learning is fundamentally different. A machine is given many examples and learns a pattern from the data.
ECG:
The older ECG machine interprets by measuring intervals (PR, RR, QRS, QT, etc.), ST deviation, and voltage, and applying those values to a set of criteria. It knows atrial fibrillation based on strict criteria.
A machine learning program is given millions of ECGs labeled as atrial fibrillation and learns to recognize the pattern based on their similarities.
Early Sepsis Detector:
A program looks at vital signs and specific labs and applies an algorithm. Once specific values reach a set threshold, an alert is triggered.
A machine learning program is given millions of sepsis cases and builds a pattern for sepsis evolution. Then it’s connected to a live EHR and looks for the patterns it identified in the training set. When it sees something familiar, it triggers an alert.
Radiology Assistant:
A radiologist points a program at an abdominal aortic aneurysm, which then measures the diameter and triggers an alert if the measurement is over 5 cm.
A machine learning program is given millions of images of abdominal aortic aneurysms, pneumonias, and pneumothoraces, and learns patterns for their appearance on X-ray and CT. It then labels images with these findings before a radiologist reviews them.
Historical programs relied primarily on relationships specified by humans. Machine-learning models learn predictive relationships from examples. In both cases, humans determine the task. What changes is how the machine learns the relationship between the input and the answer.
Is It a Definitive Answer or a Best Guess?
Now you may be thinking: how does a machine learning algorithm “know” something is present if it isn’t given the strict criteria to define it? The answer is a best guess of sorts, based on millions of examples and statistics.
Back to our examples for some ways a machine learning model might reach a conclusion:
ECG: Based on the patterns identified in this ECG, atrial fibrillation is the most likely diagnosis.
Sepsis: The trend in this patient’s laboratory data, vital signs, past medical history, and home medications put them at high risk for progression to septic shock.
Radiology: Based on the patterns in this image, a pneumothorax is likely present.
Each diagnosis is made based on statistics and pattern recognition. It’s not random, but it might be helpful to think of the AI starting the answer with “My best guess is…”. That helps frame the probabilistic nature (likelihood method) of a machine learning model’s answers.
What Happens When We Stop Telling the Computer What to Look For?
If the machine learning program is recognizing patterns on its own, is it reaching the same conclusions as humans? That’s a question that raises the stakes for what machine learning can do, or do wrong.
ECG: The sinus-rhythm AF example
Researchers at Mayo demonstrated that a neural network could analyze an ECG showing sinus rhythm and identify patients associated with atrial fibrillation. That tells us the model isn’t just applying the diagnostic criteria for atrial fibrillation, because atrial fibrillation isn’t actually present. It may be “seeing” subtle electrical changes associated with atrial remodeling that humans either don’t appreciate or just don’t quantify. It has found something we didn’t teach it to associate with atrial fibrillation.
Radiology
Sometimes the information a model extracts from an image isn’t something physicians traditionally interpret from that image at all. Researchers have shown, for example, that deep-learning models can predict age, sex, smoking status, and cardiovascular risk factors from retinal fundus photographs. The model wasn’t given a physician-created visual checklist for identifying a smoker. It learned image patterns associated with the labels.
In both examples, the machine learning model found patterns not previously described.
How Do We Tell Discovery From Shortcut?
But what if the machine has just identified a shortcut, and not an actual novel pattern?
For example, what if the model has learned that patients with pneumothorax have chest tubes on X-ray, or that patients with early sepsis have that term appear in a clinical note somewhere, or even that ECGs associated with atrial fibrillation disproportionately came from older patients, a particular hospital, or a particular ECG machine? Every one of those associations might be statistically real. None necessarily means the model has discovered previously unrecognized atrial physiology.
How do we know the model learned a relationship that will remain useful for the clinical task we want it to perform? We validate the findings. Validation involves testing the model’s pattern recognition in a new dataset to see if the skill holds up.
We use:
Patients it has never seen, with different disease prevalence
Different environments (another hospital or city or country)
Different clinicians, devices, workflows
Datasets that may have shortcut labels missing
ECG: Does the hidden AF signal persist across hospitals, ECG machines, and patient populations?
Sepsis: Does the detector still work when documentation, antibiotic use, and patient mix change?
Consider the Epic Sepsis Model, for example. Its performance depends on the model version, institution, threshold, and implementation. When Wong and colleagues tested it externally across multiple centers, they found that accuracy varied significantly based on where you deployed it and how you tuned the alert threshold.
Radiology: Does performance hold across different scanners, acquisition protocols, and hospitals?
If chest tubes might be helping the model detect pneumothorax, test pneumothoraces without chest tubes. Published work has shown substantially poorer pneumothorax detection in images without chest drains, illustrating shortcut learning.
Validation asks whether the learned pattern survives when the circumstances change.
Does It Help Patients?
After validating that the model’s new skill is accurate, there is one more question we need to address: is it clinically useful? And more specifically, does it cause harm?
ECG
Does the model identify more atrial fibrillation than a standard approach? Does the detection of propensity for atrial fibrillation while the patient is in a sinus rhythm change treatment or improve outcomes? Does it reduce strokes?
Sepsis
Is the early warning alert helpful? Does it result in earlier treatment by clinicians? Does it prevent cases of septic shock or reduce ICU escalation? Or does the model trigger an increase in false alarms?
Radiology
Does the system prioritization shorten time to diagnosis? Does the interpretation assistance improve care? Is there an increase in delays for studies that are deemed low priority by the model?
Conclusion: How to Tell the Difference
So when a vendor tells you their software uses “AI,” what should you be asking?
First, does it use machine learning? If the software applies a set of pre-programmed rules, it’s not AI. It’s decision support, and that’s ok. Those tools have been valuable for decades. But they’re different from machine learning, which learns patterns from examples rather than applying rules you’ve written for it.
Second, has the model been validated in your setting? Machine learning trained on one hospital’s data may not work the same way in yours. A tool that performed well in a published study might behave differently when you deploy it with your patient population and your documentation practices.
Third, does it change how you treat patients or improve outcomes? An AI that identifies atrial fibrillation in a sinus rhythm ECG is interesting, but only if it leads to treatment decisions that prevent strokes. An early warning system for sepsis only matters if clinicians act on it and if that action improves survival. The technology itself is secondary to the outcome.
The same capability that lets machine learning find useful patterns we never knew existed, also lets it become very good at finding patterns we never wanted it to learn. The risk is real, and so is the possibility of genuine discovery. Our job is to ask which one we’ve got before we deploy it.