Who Grades the Evidence? Five Products, Five Different Answers
OpenEvidence’s new EvidenceGrad vs UpToDate, DynaMed, Vera Health and Consensus
OpenEvidence launched EvidenceGrade. Let’s walk through what this new feature actually does and how it compares to UpToDate Expert AI, DynaMed, Consensus, and Vera Health. I’ll also touch on when to use each one, when there’s no real difference, and most importantly, who’s on the hook when the grade is wrong. For more on curated knowledge and AI, read this previous article: “When Clinical AI Says ‘I Don’t Know’ ”
As always, if you enjoy reading, subscribe and tell a friend.
Sam
On July 10, 2026, OpenEvidence launched EvidenceGrade, a feature that assigns a letter grade to the strength of evidence beneath every AI-generated clinical answer. It’s an improvement in the value of the responses OpenEvidence creates, and it lands in a market where other tools have been grading evidence in multiple different ways for quite some time.
What EvidenceGrade Does
EvidenceGrade builds on GRADE, the Grading of Recommendations, Assessment, Development, and Evaluation framework behind Cochrane reviews, WHO guidance, and most major clinical practice guidelines. GRADE has run evidence-based medicine for two decades. What’s new is speed, not the underlying idea of an overall grade. OpenEvidence computes a single grade for the whole answer live, at the moment a clinician asks a question, a step that has previously required human editors working ahead of time.
When a clinician asks OpenEvidence a question, the system first runs a classification step, checking whether the question amounts to a clear evidentiary claim. Simple definition lookups and summarization requests get filtered out here. Questions that pass get scored on study design, consistency of findings, precision, and directness- the same inputs GRADE has always used- and the system returns a letter grade for strength, with a separate “U” designation for evidence that can’t be graded.
Two details are worth noting. OpenEvidence’s classifier decides which questions get graded, not the clinician. And the grading itself comes from a single company applying GRADE algorithmically to its own retrieval results, a different process than GRADE’s original consensus-panel model.
Five Products, Five Methods
OE’s EvidenceGrade scores literature live, at the moment a clinician asks a question. Scoring extends to questions that no formal appraisal has reviewed yet, and the tool runs at point-of-care speed.
UpToDate’s Expert AI launched in late 2025 and sits on top of content that physician editors already reviewed and graded using GRADE, with Grade 1 or 2 for recommendation strength and Grade A, B, or C for evidence quality. The AI layer surfaces existing editorial work conversationally, with rationale and links back to the original topics. Scoring depends on what editors have written and reviewed, so very recent literature can lag what OpenEvidence pulls in real time.
DynaMed shares UpToDate’s editorial foundation: physician editors review and rate evidence with GRADE ahead of time. The method diverges at synthesis. DynaMed’s AI layer rates the individual sources it cites but doesn’t roll them up into a single grade for the answer as a whole, the way UpToDate’s per-recommendation grade or OpenEvidence’s per-answer letter does.
Vera Health also grades the strength of underlying evidence behind its answers and positions itself as a direct point-of-care competitor to OpenEvidence. That grading applies to the individual sources it cites, the same source-level approach DynaMed’s AI takes, rather than producing one overall grade for the answer.
Consensus takes a different approach. Its default ranking sorts papers by citation count, journal reputation, and recency, all signals of a paper’s influence and reach rather than the rigor of its methodology. Consensus also offers an optional evidence-hierarchy filter that lets a clinician sort by study design, from meta-analyses and systematic reviews down through RCTs and observational studies. Nothing is graded automatically here; the clinician chooses whether to apply the filter every time.
Pros and Cons
OpenEvidence offers speed and reach. A grade can appear under a question no editor has ever reviewed, in the time a clinician has between patients. That speed comes from a single company’s automated interpretation of GRADE, run through a classifier a clinician can’t inspect.
UpToDate carries the weight of editorial review behind every grade, built up over years with an established audit trail. Questions about very recent literature can outrun what the editorial team has reached, so timeliness suffers where OpenEvidence’s live scoring holds an advantage.
DynaMed shares UpToDate’s tradeoff of editorial rigor against editorial pace. Its AI layer grades each source it cites, but leaves the work of weighing those individual grades into one overall judgment to the clinician, since it doesn’t generate the single per-recommendation grade UpToDate does.
Vera Health competes on benchmark performance and broader geographic reach and on a grading methodology it hasn’t named as clearly as its competitors have. It also stops at source-level grades rather than synthesizing one for the answer, so a clinician still has to weigh conflicting source grades themselves. A clinician outside the US or UK who lost access to OpenEvidence may find Vera Health the more practical option.
Consensus puts control directly in the clinician’s hands. No classifier decides what gets graded, and every AI-generated summary traces back to an exact sentence in the source paper. That control depends on the clinician knowing to apply the hierarchy filter; skip it, and the ranking on screen reflects citation counts and journal reputation, which are signals of popularity rather than rigor. Consensus also serves general research across all fields, so it lacks the clinical workflow tuning built into the other four tools.
When to Actually Use Which
A fast, evidentiary bedside question, something like whether a drug reduces mortality in a given population, fits OpenEvidence well. Watch for whether a banner appears at all; no banner means the classifier judged the question outside EvidenceGrade’s scope, which is worth a second look if the question felt evidentiary to you.
A well-established clinical question or standard management of a common chronic condition plays to UpToDate or DynaMed’s strength. Human editorial review has already happened, and the subscription cost buys that vetting.
A clinician outside the US or UK, or one weighing the benchmark claims directly, has reason to look at Vera Health alongside or instead of OpenEvidence, keeping in mind that the comparative claims come from Vera Health itself.
Deep literature review on an emerging or contested question, or research headed into a paper or grant application, calls for Consensus, used deliberately with the hierarchy filter engaged and the full-text tool open to check claims directly.
Many everyday questions have no clear winner among these five. When UpToDate already holds a well-established Grade A recommendation for a common condition, OpenEvidence’s live grade on the same question will likely land in the same place, since both draw on the same underlying evidence base. The choice there comes down to workflow preference, editorial certainty against conversational speed. A clinician willing to spend an extra ninety seconds applying Consensus’s RCT filter can reach a similar result to what EvidenceGrade delivers automatically.
Risk and Bias, In Both Directions
The editorial model behind UpToDate and DynaMed carries its own risks. A recommendation can sit unchanged for months after new evidence complicates it, and every grading decision reflects the judgment of the specific editors who wrote it, not an infallible panel.
OpenEvidence’s automated model carries its own risk. A letter grade can read as more authoritative than a live scoring process actually supports, which invites automation bias. The classification step still makes a judgment call the clinician doesn’t get to weigh in on: whether a question counts as evidentiary at all. Grading quality also depends entirely on what the retrieval system pulled in the first place, a step a clinician can’t audit in real time.
DynaMed’s AI and Vera Health carry a different risk, tied to the same source-level grading described above. Neither tool synthesizes the individual source grades into one judgment, which means the aggregation OpenEvidence and UpToDate perform automatically becomes a manual step for a clinician moving quickly between patients, and manual synthesis done under time pressure is where inconsistency creeps in.
Consensus shifts risk toward the clinician directly. Someone who doesn’t know to apply the evidence-hierarchy filter, or who doesn’t recognize that the default ranking measures popularity rather than rigor, gets no protection from the tool itself.
Each model fails in its own way, and the failure a clinician is exposed to depends on which tool they picked and how well they understand what runs underneath it.
The Question Nobody Has Answered: Liability
Clinicians are not attorneys, and nothing here is legal advice. It’s worth asking plainly, since none of the marketing for any of these five products addresses it directly: Does a clinician carry more or less liability for relying on an AI-generated evidence grade compared to a human-editor-vetted one?
I could not find an answer to this question. Malpractice and negligence standards generally turn on whether a clinician exercised reasonable judgment consistent with the standard of care, and historically that inquiry centers on the clinician’s own decision rather than the reference tool behind it. All five products here position themselves as clinical decision support rather than autonomous diagnostic tools, which keeps the clinician as the final decision-maker of record, the same SaMD-versus-CDS framing this newsletter has examined before.
Whether courts will eventually treat a real-time AI grade differently from an incremental human editorial grade remains genuinely untested. I don’t have an answer, and anyone claiming otherwise is speculating.
What holds steady across all five tools is this: whatever grade appears on the screen, the decision that follows still belongs to the clinician who acts on it.