Mission-Critical Evals at Scale: Learnings from 100,000 Medical Decisions

How do you catch a consequential medical error when an LLM’s answer looks reasonable? Christopher Lovejoy, a medical doctor turned AI engineer, approaches this as a problem of building an evaluation system that can keep working as request volume grows.

Lovejoy reports that Anterior serves insurance providers covering 50 million American lives. The lessons here come from 18 months of development: getting an LLM MVP to work is increasingly easy, but growing request volume exposes edge cases that the MVP never encountered.

Anterior’s prior-authorization product supports decisions about whether a treatment request should be approved or reviewed by a clinician. It receives medical records and guidelines containing questions that must be answered. In the example, a guideline asks whether the patient has had a previous brain MRI suspicious for multiple sclerosis, as part of determining whether they should receive an MRI of the cervical spine.

The AI supplies apparently persuasive evidence: a brain MRI shows hyperintensity in the infratentorial, juxtacortical, and periventricular white matter, noted as consistent with multiple sclerosis. It concludes that this confirms prior findings suspicious for MS. But the patient already has an established MS diagnosis. In Lovejoy’s explanation of this particular guideline question, suspicious implies an unconfirmed diagnosis; established disease does not satisfy that distinction. The answer cites relevant evidence but interprets it incorrectly.

Lovejoy illustrates the scale problem with an error occurring once per 1,000 or 10,000 cases in a system processing more than 100,000 cases daily. Those are illustrative frequencies and volumes, not measured Anterior throughput. Rare mistakes become a recurring operational problem. He also points to lawsuits over inappropriate healthcare AI automation: identifying and handling these failures is a requirement, not an optional quality improvement.

Human review is the first layer. Anterior built an internal clinical team and a review dashboard called Scalpel. The interface places the medical record and guideline on the right, accessible without scrolling, and the question and required context on the left. Keeping the evidence alongside the decision lets reviewers assess many questions quickly.

For the MRI example, the reviewer labels the answer incorrect and saves a critique explaining why. That critique states what is wrong; a reference answer states what the correct response should be. Combining the critique with the original answer produces what Lovejoy calls a ground truth, which can then be reused in offline evaluations.

The resulting record separates the guideline question, its Boolean answer, and its reasoning. A compact Python representation of the example is:

reference_answer = {
    "question": "Has the patient had a prior brain MRI suspicious for MS?",
    "is_met": False,
    "reasoning": (
        "The MRI findings reflect the patient's established MS diagnosis, "
        "rather than findings suspicious for an unconfirmed diagnosis."
    ),
}

Here, is_met = False answers this guideline question; it is not a treatment-denial decision. Preserving the reasoning makes the clinically important distinction available to subsequent evaluation.

Suppose an MVP processes 1,000 medical decisions a day and the team reviews half of them. Lovejoy’s staffing example assumes each clinician can perform 100 reviews per day. The initial workload is manageable, but maintaining coverage as volume grows rapidly changes the staffing requirement.

All rows use the same assumed capacity of 100 reviews per clinician per day. At 10,000 decisions, maintaining 50% coverage would require a clinical review team larger than Anterior’s entire company at the time. Reducing coverage to 5% temporarily restores the original staffing requirement, but another increase in volume brings the problem back.

That leaves two distinct questions: Which cases should receive human review? And how did the system perform on the cases nobody reviewed? A smaller sample reduces labor, but does not by itself answer either question.

The reference answers collected through clinical review can populate offline evaluation datasets: reusable cases outside the product against which the team repeatedly runs its pipeline. Gold-standard subsets can cover different enterprises, medical specialties and conditions, difficult questions, complex cases, and ambiguous outcomes. Tracking scores over time gives engineers a way to compare pipeline changes and retain lessons from previously observed failures.

The limitation is when those lessons arrive. A new edge case may reach the customer before a reviewer identifies it and adds it to the dataset. Medical records have an enormous, heterogeneous input space, so production keeps generating unfamiliar cases. Offline evaluation remains useful for iteration, but cannot be the only mechanism for detecting live failures.

Reference-free evaluation, also called label-free evaluation, assesses an answer before its true outcome is known through human review. Removing the need to wait for a reference label makes evaluation possible while cases are being processed, when there is still time to respond.

An LLM-as-judge is a starting point for answering both the review-selection and unreviewed-performance questions:

1. Send the inputs through the LLM pipeline being evaluated.
2. Pass the resulting output to a judge with an explicit scoring system.
3. Have the judge assess the dimensions relevant to the task.

Those dimensions might be helpfulness, concision, brand tone, or confidence that a binary or multiclass answer is correct. The scoring system defines what the evaluator is being asked to assess.

Anterior’s pipeline produces one of two outcomes: approval or escalation for review. Its reference-free evaluation can use an LLM judge, logic-based confidence estimation, or a combination. The evaluator produces a confidence grade ranging from strong belief that the pipeline is correct to such low confidence that it actively predicts the answer is wrong. Applying a threshold then yields a predicted correct output.

These are two different outputs with different uses:

The second is still a prediction, not a human-established reference answer. Keeping the distinction explicit allows the system to estimate performance and prioritize review separately.

The predicted correct outputs provide an estimated view of performance across incoming cases, including those without human review. That gives the team a live signal it can respond to and communicate to customers. Where a case receives both automated evaluation and human review, their outputs can be compared to measure alignment and assess how much confidence to place in the evaluator.

The confidence grades serve a different purpose: allocating scarce reviewer attention. Anterior combines them with contextual factors such as procedure cost, risk of bias, and previous error rates. A dynamically ordered queue can then surface the most relevant cases with a high predicted probability of error, rather than treating every case as equally valuable to review.

The evaluator surfaces cases; human review determines accuracy. Those reviews then validate and improve the evaluator, which changes which cases it prioritizes next. Lovejoy describes this cycle as “validating the validator.” It connects review selection to evaluator improvement instead of treating the judge as a fixed, trusted component.

Over repeated iterations, Lovejoy argues, the pool of unfamiliar edge cases shrinks and the system’s ability to detect them improves. This accumulated capability is harder to reproduce than the visible product: it depends on processing substantial volumes of real data and repeatedly incorporating expert feedback.

Once the team has established confidence in the evaluator’s performance, it can become part of the production pipeline itself. Inputs pass through the original pipeline, and its outputs pass through reference-free evaluation before delivery. A sufficiently confident result can return to the customer; a concerning result triggers further action.

Lovejoy describes three alternative routes:

• Another LLM pipeline: Send the case through a different processing path, potentially using more expensive models.
• Internal clinical review: Ask an on-call clinician to review the case before returning the result.
• Customer review: Surface the case in the customer’s dashboard for their own team to assess.

Evaluation now changes what happens to an answer before the customer relies on it. The system can spend additional computation or expert attention on the cases that need it.

The operational objective is to grow case volume without continually expanding the clinical review team. Lovejoy reports that an unnamed competitor hired more than 800 nurses for reviews, while Anterior reviews tens of thousands of cases with fewer than 10 clinical experts. He does not specify a time window for that case volume, so this is a staffing account rather than a matched productivity comparison.

After several iterations, Lovejoy reports AI–human review alignment comparable to agreement between human reviewers. The slide reports 94% alignment between AI and human reviews, versus 95% between human reviewers. The talk does not detail the sample or agreement calculation.

Returning to the MRI example, the practical improvement is the ability to identify the incorrect answer and correct it quickly enough to meet customer timing SLAs. Catching an error is useful only if the system can also respond within the time available to deliver the decision.

Lovejoy reports nearly 96% F1 for prior authorization in a recent study. He presents the result as industry-leading, but the study methods and comparison conditions are not provided here. That historical F1 result should not be conflated with a different accuracy metric.

The customer outcome is more personal. Lovejoy recounts a case study in which a nurse, told she could continue using Florence, Anterior’s AI, responded: “Thank God, we’re the lucky ones.” The desired result is a system clinicians want to keep using because it supports their work.

The organizational choice underneath this architecture is what to do with each expert review. Using it only to audit a past answer yields one judgment. Using it to build, audit, and improve the evaluation system also changes how future cases are assessed. That makes the evaluation system itself an ongoing engineering responsibility.

Live production evaluation supplies the cases that offline datasets have not yet captured. High-quality reviewers supply the judgment needed to interpret those cases, and effective tooling lets a small team apply that judgment quickly. Lovejoy’s closing advice is to prioritize reviewer quality over quantity and build custom tools when they improve that work. The aim is a system that estimates performance in real time, responds quickly, and scales at low cost while preserving the expert feedback on which its reliability depends.