The CIO's Impossible Comparison
How hospitals are supposed to choose between Epic's AI and all others with almost nothing to go on
A CIO and a CMIO sit down with a slide deck, a budget line, and a critical decision: activate Epic’s native AI tools, bring in a third-party vendor, or build a custom AI tool through an API.
They need comparison data, but there isn’t any. The single largest, most consequential technology decision many U.S. hospitals will make this decade is being made with far less evidence than we would accept for a new drug or device. Let’s dive deeper.
As always, if you enjoy reading, subscribe and tell a friend.
Sam
It’s not a hypothetical problem.
More than 85% of Epic’s customer base is already using some form of Epic AI
EHR vendor dependency is cited as the top execution barrier by 74% of health system technology leaders
A majority of Epic’s customers report spending a quarter of their IT bandwidth simply managing multiple vendor integrations rather than building anything new.
So how are hospitals supposed to decide about Epic’s AI features? Three questions capture the actual decision tree a CIO faces:
How do you compare Epic’s tools against alternatives when no published performance data exists?
Are you left comparing on price alone?
Can you even connect an outside AI tool to Epic, and what does that actually cost in dollars and time?
Each answer turns out to be more complicated than it first appears.
The Comparison Vacuum
According to EPIC, more than 200 organizations use Penny for professional coding, with many seeing 20% of more coding-related denial reductions. Art’s Insights feature is used more than 16 million times a month. At The Christ Hospital, Art’s radiology finding extraction is credited with a 69% early lung cancer detection rate compared to a national average of 46%. At Rush University Medical Center, Emmie delivered a 58% reduction in billing-related customer service messages.
These are genuine wins, not vague marketing language. But notice what they have in common: every one of them is a curated success story that Epic chose to publish, from a customer Epic chose to name, describing an outcome Epic chose to measure. None of them come with a comparison arm. None of them tell you what a third-party tool, or no AI tool at all, would have achieved at the same hospital over the same period.
What doesn’t exist is any neutral, third-party study putting Epic’s tools head-to-head against Abridge, Ambience, Nabla, Suki, or Oracle Health’s Clinical AI Agent, on the same patients, measuring the same things. The closest proxies available each fall short in a specific, disqualifying way:
KLAS “Best in KLAS” rankings exist for ambient AI and clinical documentation, and vendors like Abridge and Ambience have won them in 2025 and 2026. But KLAS rankings are built from customer satisfaction surveys. They measure whether clinicians like a tool, not whether it performs more accurately or safely than the alternative.
Vendor comparison content, like the “Abridge vs. Ambience” writeups that circulate among health IT consultants, reads like independent analysis but is typically produced by resellers and integration firms with a commercial stake in one outcome or another.
To be fair to Epic, native integration solves a real, well-documented problem. Third-party AI tools often exist as a separate app or window that a clinician has to switch into, breaking their workflow. Epic’s tools, by contrast, draw on the patient’s complete longitudinal record natively, inside the same interface the clinician already lives in. That is legitimate integration depth, and Epic has a structural claim to it that outside vendors have to work much harder to match.
But “we’re already inside your workflow, so trust us” is a switching-cost argument, not a quality argument. It explains why Epic’s tools are more convenient to adopt. It says nothing about whether Art’s diagnostic suggestions, Emmie’s patient explanations, or Penny’s coding recommendations are more accurate than a competitor’s because nobody has measured that, and Epic has no more incentive to fund that comparison than a third-party vendor would have to fund one showing Epic winning.
The honest, unsatisfying answer for how hospitals resolve this: they don’t rely on external comparison data. They run their own limited pilots of both options against their own patient population and staff, because nobody else has done, or has an incentive to do, that comparison for them. That’s not a failure of any individual CIO’s diligence. It’s a structural gap in the market that every hospital is left to close on its own, one expensive pilot at a time.
Price Is Visible. Quality Isn’t.
No CIO is literally forced to decide on price alone. But price is the one variable in this decision with any real transparency, and that asymmetry quietly warps the whole comparison because when quality is unmeasurable and price is a line item, price tends to win by default, whether or not it should.
The other differentiators hospitals actually weigh are real, but none of them are clinical outcome measures:
Integration depth. Does the tool live inside Epic natively, or does it require clinicians to leave the EHR? This is the “house vs. guest in the house” distinction discussed above. It’s a genuine operational factor, not evidence of accuracy or safety.
Specialty coverage breadth. Some third-party tools market coverage across 200-plus specialties, from oncology to emergency medicine. That’s a real capability difference, but it says nothing about performance within any single specialty.
Vendor relationship tier with Epic. Some vendors, like Abridge and Nuance, hold a spot in Epic’s invite-only “Workshop” co-development tier. Others sit in the more generic, self-service “Connection Hub” listing. This tells you something about durability and long-term support risk. It tells you nothing about whether the underlying AI is any good.
Lock-in and commoditization risk. As Epic rolls out “good enough” native alternatives, standalone point solutions face real competitive pressure. Betting on a third-party tool means betting that it survives Epic’s gravitational pull.
The uncomfortable throughline across all four: they’re operational and strategic risk proxies, not evidence. A CIO weighing them isn’t answering “which AI is better”. They’re answering “which AI is safer to bet on organizationally.”
Yes, You Can Connect Outside AI to Epic
Epic is not a walled garden on this point. The technical path to connecting an outside AI tool, whether a commercial product or a custom build on an API, is real, standardized, and already proven in production.
The mechanisms, at the level a CIO actually needs:
SMART on FHIR is an open industry standard, not an Epic-proprietary one, that lets a third-party application launch inside Epic’s interface (Hyperspace, Hyperdrive, or MyChart) with OAuth2-based access to the patient’s clinical context.
CDS Hooks is a lighter-weight alternative for narrower use cases: an external tool fires in real time on a specific clinical event (opening a chart, signing an order) and returns a recommendation card, without the overhead of launching a full separate application.
Epic Showroom (formerly App Orchard) is the marketplace and certification pathway, with three tiers:
Connection Hub, a basic listing starting around $500 a year, where most third-party apps live.
Toolbox, offering curated visibility
Workshop, reserved for vendors Epic is actively co-developing with, not a tier you can apply for.
Abridge, Ambience, Nabla, and Suki all run as third-party ambient scribes launched inside Epic today via exactly this pathway. Connecting a custom AI build is well-trodden ground, not an experimental frontier.
What it actually costs, in dollars:
A basic FHIR read integration for a first site: $15,000–$40,000
Bidirectional integration (the tool can write back into the chart, not just read from it): $40,000–$80,000
Each additional hospital site: $5,000–$20,000 in configuration, testing, and governance overhead
Enterprise multi-site deployments (five-plus hospitals): $100,000–$500,000+
Annual maintenance: 15–20% of build cost, ongoing
Those figures answer the technical and financial questions. They don’t answer the harder question, which is where most integration projects actually stall.
“Integration Is 20% Development, 80% Negotiation”
Brendan Keeler, a widely cited EHR integration analyst, has a line that captures this better than any cost table: integration is 20% development, 80% negotiation.
It’s helpful sitting thinking about what that 80% actually is, because it’s several negotiations stacked, and none of them get easier just because a vendor has already done this at another hospital:
Per-site IT governance review. Even though Epic’s FHIR APIs are standardized across every customer, each hospital runs its own instance with its own security policies. A vendor doesn’t clear “Epic” once. They clear each hospital’s IT and security team separately, every time.
Scope and access-level negotiation. What data can the tool read? Can it write back into the chart? Is access tied to individual user logins or a backend service account? These aren’t just technical settings. They’re negotiated line by line with each hospital’s compliance and security staff, who have their own risk tolerance and institutional precedent to defend.
Contract terms with Epic itself, separate from any single hospital deal: Showroom tier, vendor services registration, fees.
Institutional change management: Getting clinical leadership, IT, compliance, and often legal to actually agree to put a new tool inside their clinicians’ workflow.
This negotiation stack isn’t identical across hospitals, even though the underlying FHIR technology may be. Each hospital is, in effect, a separate legal and organizational gate a vendor has to clear from scratch, no matter how many times they’ve cleared an equivalent gate elsewhere. That mismatch is exactly why Epic’s own native tools carry a durable structural advantage that has nothing to do with whether their AI is actually better. They skip the entire negotiation stack because they’re already inside the walls, already covered by the hospital’s existing Epic contract and security posture.
How Vendors Are Fighting Back
Third-party vendors aren’t standing still against this asymmetry. Several strategies have emerged, each attacking a different piece of the 80%:
Pre-certification as a trust shortcut. HITRUST certification (including a new AI-specific security certification launched in response to healthcare’s AI boom) and SOC 2 Type II audits let a vendor prove its security posture once and present that proof to every prospective hospital, rather than rebuilding trust from zero at each site. HITRUST is now often a baseline requirement for even getting considered in procurement.
Integration aggregators. Platforms like Redox normalize FHIR, HL7 v2, and proprietary EHR APIs into a single interface, letting a vendor avoid building direct integrations against three to five different EHR systems from scratch, at the cost of some data fidelity and an extra hop in the data flow.
Design-partner-first rollout. Land one engaged hospital willing to work through the full registration and security review cycle together, get the integration proven in production, then use that as a template for the second and third hospital, accepting that some per-site variability will still surface each time.
Climbing into Epic’s Workshop tier. Vendors like Abridge and Nuance have secured Epic’s invite-only co-development relationship, which functions as Epic effectively vouching for them. It’s a different negotiation dynamic from approaching each hospital cold.
Leaning on CDS Hooks for narrower use cases, trading a smaller interaction surface for meaningfully lower integration complexity.
Regulatory tailwinds. The 21st Century Cures Act’s information-blocking provisions and ONC’s HTI-1 rule, which raised the certification baseline to USCDI v3 as of January 2026, give vendors a legally backed claim to FHIR access, shifting some leverage away from ad hoc hospital IT gatekeeping.
None of this eliminates the negotiation. It compresses or front-loads it. Bidirectional write-back (actually pushing AI-generated content into the medical record, not just reading from it) remains the hardest and slowest part of any integration, regardless of how much pre-certification a vendor obtains.
What CIOs Actually Have to Work With
A few genuinely useful resources exist for the leader trying to navigate this, though none of them is the comprehensive primer this decision deserves.
The Health Sector Coordinating Council’s “Health Industry AI Cyber Governance Framework Implementation Guide” (May 2026) is the closest thing to an authoritative, non-commercial reference, covering governance committee structure scaled to hospital size, escalation authority across CMO, CMIO, and Privacy Officer roles, and a benefit-risk framework tied explicitly to FDA regulatory status.
Qventus’s “Beyond the Pilot” survey of more than 60 CIOs, Chief AI Officers, and CMIOs offers real peer benchmarking: 74% cite EHR vendor dependency as their top execution barrier; 72% say they’d prefer a single consolidated AI partner over a fragmented multi-vendor stack, but only 13% have actually achieved that consolidation. It’s vendor-published, so it should be read as useful data with a thumb on the scale, not neutral research.
Narrower academic frameworks exist too, like peer-reviewed pragmatic-trial protocols for evaluating ambient AI specifically, which bring real methodological rigor but only to one slice of the larger decision.
What none of these does is tie the whole picture together. A CIO today has to assemble governance policy from one source, vendor economics from a consulting firm’s blog, integration cost data from an engineering guide, and validation-status skepticism from wherever they can find it, with no neutral party doing that synthesis for them. That absence is, itself, worth naming plainly: the market has produced plenty of pieces but no assembled whole.
Takeaway
Individual hospitals cannot solve this problem on their own. It is a fundamental issue with the entire healthcare market, which tends to favor established players, like Epic, over newer or better alternatives, simply because they already hold the power.
A few things worth holding onto if you’re the one making this call:
Don’t mistake Epic’s adoption percentages for comparative quality evidence. They are usage statistics, curated by the company reporting them. They tell you Epic AI is widely used. They don’t tell you it’s the best option.
Budget for the 80%, not just the 20%. The dollar figures for FHIR integration are real, but the calendar time and staff bandwidth consumed by per-site negotiation are the actual cost drivers, and they’re the ones most commonly missing from a project’s original scope.
Ask any vendor directly what Showroom tier they hold. It won’t tell you if their AI is accurate, but it will tell you something real about how durable their access is likely to be.
Treat the absence of comparative data as a mandate to pilot, not a reason to default to the incumbent. Nobody else is going to run that comparison for you. That doesn’t mean it isn’t worth running.
No one selling you this technology is going to prove it works for you. That job still belongs to the hospital buying it.