AI Engineer Summit 2025
Mission-Critical Evals at Scale: Learnings from 100,000 Medical Decisions
Read the talk
Building live evaluations for mission-critical medical decisions
A plausible MRI answer can miss the clinical distinction that matters. Anterior’s evaluation loop connects expert review, live confidence estimates, and routing before answers reach customers.
From a talk by Christopher Lovejoy
Before you start: Basic familiarity with LLM pipelines and evaluation is helpful; no medical background is required.
When a plausible medical answer is wrong
How do you catch a consequential medical error when an LLM’s answer looks reasonable? Christopher Lovejoy, a medical doctor turned AI engineer, approaches this as a problem of building an evaluation system that can keep working as request volume grows.
Lovejoy reports that Anterior serves insurance providers covering 50 million American lives. The lessons here come from 18 months of development: getting an LLM MVP to work is increasingly easy, but growing request volume exposes edge cases that the MVP never encountered.
Anterior’s prior-authorization product supports decisions about whether a treatment request should be approved or reviewed by a clinician. It receives medical records and guidelines containing questions that must be answered. In the example, a guideline asks whether the patient has had a previous brain MRI suspicious for multiple sclerosis, as part of determining whether they should receive an MRI of the cervical spine.
The AI supplies apparently persuasive evidence: a brain MRI shows hyperintensity in the infratentorial, juxtacortical, and periventricular white matter, noted as consistent with multiple sclerosis. It concludes that this confirms prior findings suspicious for MS. But the patient already has an established MS diagnosis. In Lovejoy’s explanation of this particular guideline question, suspicious implies an unconfirmed diagnosis; established disease does not satisfy that distinction. The answer cites relevant evidence but interprets it incorrectly.
Lovejoy illustrates the scale problem with an error occurring once per 1,000 or 10,000 cases in a system processing more than 100,000 cases daily. Those are illustrative frequencies and volumes, not measured Anterior throughput. Rare mistakes become a recurring operational problem. He also points to lawsuits over inappropriate healthcare AI automation: identifying and handling these failures is a requirement, not an optional quality improvement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn expert critiques into reusable reference answers
Human review is the first layer. Anterior built an internal clinical team and a review dashboard called Scalpel. The interface places the medical record and guideline on the right, accessible without scrolling, and the question and required context on the left. Keeping the evidence alongside the decision lets reviewers assess many questions quickly.
For the MRI example, the reviewer labels the answer incorrect and saves a critique explaining why. That critique states what is wrong; a reference answer states what the correct response should be. Combining the critique with the original answer produces what Lovejoy calls a ground truth, which can then be reused in offline evaluations.
The resulting record separates the guideline question, its Boolean answer, and its reasoning. A compact Python representation of the example is:
python
reference_answer = {
"question": "Has the patient had a prior brain MRI suspicious for MS?",
"is_met": False,
"reasoning": (
"The MRI findings reflect the patient's established MS diagnosis, "
"rather than findings suspicious for an unconfirmed diagnosis."
),
}
Here, is_met = False answers this guideline question; it is not a treatment-denial decision. Preserving the reasoning makes the clinically important distinction available to subsequent evaluation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Why reviewing a fixed fraction stops working
Suppose an MVP processes 1,000 medical decisions a day and the team reviews half of them. Lovejoy’s staffing example assumes each clinician can perform 100 reviews per day. The initial workload is manageable, but maintaining coverage as volume grows rapidly changes the staffing requirement.
| Decisions/day | Fraction reviewed | Reviews/day | Clinicians needed |
|---|---|---|---|
| 1,000 | 50% | 500 | 5 |
| 10,000 | 50% | 5,000 | 50 |
| 10,000 | 5% | 500 | 5 |
| 100,000 | 5% | 5,000 | 50 |
All rows use the same assumed capacity of 100 reviews per clinician per day. At 10,000 decisions, maintaining 50% coverage would require a clinical review team larger than Anterior’s entire company at the time. Reducing coverage to 5% temporarily restores the original staffing requirement, but another increase in volume brings the problem back.
That leaves two distinct questions: Which cases should receive human review? And how did the system perform on the cases nobody reviewed? A smaller sample reduces labor, but does not by itself answer either question.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Offline datasets preserve lessons after failures occur
The reference answers collected through clinical review can populate offline evaluation datasets: reusable cases outside the product against which the team repeatedly runs its pipeline. Gold-standard subsets can cover different enterprises, medical specialties and conditions, difficult questions, complex cases, and ambiguous outcomes. Tracking scores over time gives engineers a way to compare pipeline changes and retain lessons from previously observed failures.
The limitation is when those lessons arrive. A new edge case may reach the customer before a reviewer identifies it and adds it to the dataset. Medical records have an enormous, heterogeneous input space, so production keeps generating unfamiliar cases. Offline evaluation remains useful for iteration, but cannot be the only mechanism for detecting live failures.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluate before a human label exists
Reference-free evaluation, also called label-free evaluation, assesses an answer before its true outcome is known through human review. Removing the need to wait for a reference label makes evaluation possible while cases are being processed, when there is still time to respond.
An LLM-as-judge is a starting point for answering both the review-selection and unreviewed-performance questions:
- Send the inputs through the LLM pipeline being evaluated.
- Pass the resulting output to a judge with an explicit scoring system.
- Have the judge assess the dimensions relevant to the task.
Those dimensions might be helpfulness, concision, brand tone, or confidence that a binary or multiclass answer is correct. The scoring system defines what the evaluator is being asked to assess.
Anterior’s pipeline produces one of two outcomes: approval or escalation for review. Its reference-free evaluation can use an LLM judge, logic-based confidence estimation, or a combination. The evaluator produces a confidence grade ranging from strong belief that the pipeline is correct to such low confidence that it actively predicts the answer is wrong. Applying a threshold then yields a predicted correct output.
These are two different outputs with different uses:
| Evaluator output | Meaning |
|---|---|
| Confidence grade | How strongly the evaluator believes the pipeline answer is correct |
| Predicted correct output | Which answer the evaluator believes should have been returned |
The second is still a prediction, not a human-established reference answer. Keeping the distinction explicit allows the system to estimate performance and prioritize review separately.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Estimate performance, prioritize cases, validate the judge
The predicted correct outputs provide an estimated view of performance across incoming cases, including those without human review. That gives the team a live signal it can respond to and communicate to customers. Where a case receives both automated evaluation and human review, their outputs can be compared to measure alignment and assess how much confidence to place in the evaluator.
The confidence grades serve a different purpose: allocating scarce reviewer attention. Anterior combines them with contextual factors such as procedure cost, risk of bias, and previous error rates. A dynamically ordered queue can then surface the most relevant cases with a high predicted probability of error, rather than treating every case as equally valuable to review.
The evaluator surfaces cases; human review determines accuracy. Those reviews then validate and improve the evaluator, which changes which cases it prioritizes next. Lovejoy describes this cycle as “validating the validator.” It connects review selection to evaluator improvement instead of treating the judge as a fixed, trusted component.
Over repeated iterations, Lovejoy argues, the pool of unfamiliar edge cases shrinks and the system’s ability to detect them improves. This accumulated capability is harder to reproduce than the visible product: it depends on processing substantial volumes of real data and repeatedly incorporating expert feedback.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put a validated evaluator in the delivery path
Once the team has established confidence in the evaluator’s performance, it can become part of the production pipeline itself. Inputs pass through the original pipeline, and its outputs pass through reference-free evaluation before delivery. A sufficiently confident result can return to the customer; a concerning result triggers further action.
Lovejoy describes three alternative routes:
- Another LLM pipeline: Send the case through a different processing path, potentially using more expensive models.
- Internal clinical review: Ask an on-call clinician to review the case before returning the result.
- Customer review: Surface the case in the customer’s dashboard for their own team to assess.
Evaluation now changes what happens to an answer before the customer relies on it. The system can spend additional computation or expert attention on the cases that need it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reported operational results
The operational objective is to grow case volume without continually expanding the clinical review team. Lovejoy reports that an unnamed competitor hired more than 800 nurses for reviews, while Anterior reviews tens of thousands of cases with fewer than 10 clinical experts. He does not specify a time window for that case volume, so this is a staffing account rather than a matched productivity comparison.
After several iterations, Lovejoy reports AI–human review alignment comparable to agreement between human reviewers. The slide reports 94% alignment between AI and human reviews, versus 95% between human reviewers. The talk does not detail the sample or agreement calculation.
Returning to the MRI example, the practical improvement is the ability to identify the incorrect answer and correct it quickly enough to meet customer timing SLAs. Catching an error is useful only if the system can also respond within the time available to deliver the decision.
Lovejoy reports nearly 96% F1 for prior authorization in a recent study. He presents the result as industry-leading, but the study methods and comparison conditions are not provided here. That historical F1 result should not be conflated with a different accuracy metric.
The customer outcome is more personal. Lovejoy recounts a case study in which a nurse, told she could continue using Florence, Anterior’s AI, responded: “Thank God, we’re the lucky ones.” The desired result is a system clinicians want to keep using because it supports their work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make expert review improve the evaluation system
The organizational choice underneath this architecture is what to do with each expert review. Using it only to audit a past answer yields one judgment. Using it to build, audit, and improve the evaluation system also changes how future cases are assessed. That makes the evaluation system itself an ongoing engineering responsibility.
Live production evaluation supplies the cases that offline datasets have not yet captured. High-quality reviewers supply the judgment needed to interpret those cases, and effective tooling lets a small team apply that judgment quickly. Lovejoy’s closing advice is to prioritize reviewer quality over quantity and build custom tools when they improve that work. The aim is a system that estimates performance in real time, responds quickly, and scales at low cost while preserving the expert feedback on which its reliability depends.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
Lovejoy's companion page for this talk, with recording and presentation links.
Explains how drift and unseen cases limit offline datasets, and how live reviews and automated judges complement them.
Practical discussion of constrained outputs, task decomposition, clinical escalation and ongoing evaluation.
Updates since the talk
- Anterior's current prior authorization workflowDocumentation
Current product overview describing approval pathways, clinician escalation and auditable reasoning.
Read the complete timestamped transcript
- 0:00
Hi, my name is Christopher Lovejoy, and I'm a medical doctor turned AI engineer. And in this talk, I'm gonna consider what it means to build an eval system that works at scale, and in particular, one that supports mission-critical decisions like in healthcare where there's no room for error.
- 0:14
Now, this is something we've had to figure out at Anterior as we've scaled to now serve insurance providers covering fifty million American lives. So I'll share what we've learned in the last eighteen months, why real-time reference-free evals can be the special source that enables customer trust, and how you can build them for your company.
- 0:30
So we've all seen that it's pretty easy to create an MVP product powered by LLMs, and it's getting even easier as models get more and more powerful. But what about going from MVP to serving customers at scale?
- 0:40
Now, there are a lot of problems that you just won't see until you hit scale. And as request volume increases, so does the number of edge cases that you've never seen before.
- 0:49
So let's look at an example from the medical industry. At Anterior, our core product supports prior authorization decisions around whether a treatment request should be approved or reviewed by a clinician.
- 1:00
We receive medical records and guidelines which contain various questions. So an example question might be whether a patient has had a previous brain MRI suspicious for multiple sclerosis, and this is then being used to determine whether the patient should receive an MRI of their cervical spine.
- 1:14
So our AI may show something like this, that the medical record shows a brain MRI from this date that demonstrates hyperintensity in the infratentorial, juxtacortical, and periventricular white matter, which is noted to be consistent with multiple sclerosis.
- 1:28
And this confirms prior brain MRI findings suspicious for MS. And on the surface, this looks pretty reasonable, but the problem is that this is missing some key medical nuance.
- 1:38
Now, in a medical context, if I as a doctor say that something is suspicious, I'm implying that the patient doesn't already have a confirmed diagnosis. But in this case, the patient actually did have an existing diagnosis, and therefore, this is not just suspicious, it's confirmed, which means that this answer is actually wrong.
- 1:53
Now, this kind of mistake might happen every thousand cases or even every ten thousand cases. But if you're processing more than a hundred thousand cases every day, then that's a lot of mistakes that you need to pick up.
- 2:04
And the problem is, we just can't make mistakes like this. There are many organizations in US healthcare that are being sued right now for using AI automation inappropriately. So how do you identify and handle failure cases?
- 2:16
Well, the first thing you should consider is performing human reviews of AI outputs. At Anterior, we've built out an internal clinical team and created internal tooling to make this as easy and effective as possible.
- 2:27
So this is our review dashboard, which we've called Scalpel. And on the right-hand side here, we have all of the context that our reviewer needs surfaced in an accessible way without any scrolling required, so they can see the medical record, and they can also see the guideline.
- 2:41
And then on the left-hand side, we have the question that we're answering and the required context, and this empowers our reviewers to review a high number of questions very quickly.
- 2:49
So continuing our example from before, we can ask our reviewers to add a critique saying why this is wrong and label it as incorrect, and then save that into our system.
- 2:59
And one thing we can do with these critiques, which are a statement of what's wrong, is we can generate ground truths, which are a statement, a description of what the correct answer is.
- 3:08
So using that critique and the original answer, we can then generate these ground truths, and we can use those ground truths in offline evaluations, which I'll talk about shortly.
- 3:18
But there's a problem with human reviews, and it's the following. Let's say we've created an MVP. We have our first customer, and we're making around a thousand medical decisions per day.
- 3:27
Well, we want to know how we're doing, so let's say we'll review half of those cases to give us a good estimate.
- 3:33
Um, now reviewing half of those means five hundred re-- human reviews per day. And if every clinician on our team can re-- do about a hundred reviews per day, that means we need five clinicians to do all of these reviews.
- 3:43
And that's okay. That can work. But the problem is when we go beyond MVP and we, you know, we start doing ten thousand medical decisions a day, to maintain the same percentage, we would now have to do five thousand human reviews every day.
- 3:55
So maintaining the same ratio, we now need fifty clinicians, and that's bigger than our entire company is at the moment. Okay, so what we might do is say, "Well, maybe let's, you know, review a smaller subset of cases.
- 4:06
Let's only review five percent." That gets us back down to five hundred human reviews a day, which can be done by five clinicians. But the problem comes as we scale even further.
- 4:15
Let's say we now grow to a hundred thousand medical decisions per day, which is still a very conservative number. Again, we're back at five thousand human reviews and fifty clinicians.
- 4:24
So the problem here is clear. This just doesn't scale. And we're left with these two questions, which is, firstly, which cases should we review? And secondly, of all the cases that we didn't review, what-- how did we perform?
- 4:38
So another component of this is offline eval datasets. And by offline here, I'm referring to datasets that we build that live outside of our product, and we can keep on running evals against them and getting scores.
- 4:49
So we can take the ground truth that we generated from our human reviews to build these datasets. And this can be helpful. We can define some gold standard datasets.
- 4:57
We can segment them by enterprise, by, uh, specific medical type, medical conditions, you know, tough questions, complex cases, ambiguous outcomes. And we can plot those performances over time. We can use them for iterating our, our AI pipelines against, and, and it's helpful.
- 5:11
But the problem is that if you wait until new edge cases are represented in this dataset, which you're building kind of downstream of actually giving this to the customer, it could be too late.
- 5:21
So relying only on offline evals is playing with fire. Um, and the input space for medical records is huge. There's very high heterogeneity. So at scale, you're continually going to see new edge cases that you need to identify and respond to.
- 5:38
And the solution for that, for these two problems, is real-time reference-free evaluation system. So reference-free, also known as label-free, means that you evaluate before you know the true outcome, i.e., before you have done a human review.
- 5:52
And that enables the system to be real time. It enables you to respond to issues immediately as they arise.
- 5:59
So we saw we had these two questions: Which cases should we review? And how did we do on the cases that we couldn't do a human review on?
- 6:07
Well, a great starting point here is using an LLM-as-judge. The way this works is the following. So we have our inputs, they go into our LLM pipeline that we're evaluating, and it gives some kind of outputs.
- 6:16
We then feed that output into an LLM-as-judge along with a scoring system. And this scoring system can be many different things. It could be, uh, how helpful is the output?
- 6:26
How concise is the output? Is the tone of the output on brand? It could be how confident are we that the output is correct? If our, if our output is a binary or a multi-class classification, we can give that confidence level.
- 6:38
So in our case at Anterior, we do have a binary output. Our generated output is either approval, that we think this treatment should be approved, or it's an escalation for review.
- 6:47
And we can take that, we can put that into our reference-free out-- evals, which could be an LLM-as-judge, but can also be other methods, such as confidence estimation using logic-based methods.
- 6:57
And using those methods, either alone or in combination, we can then give an output. And in our case, we use it to give us a confidence grading. How confident are we that our LLM outputs from our actual pipeline here is correct?
- 7:10
We can go all the way from high confidence that it's correct, down to such low confidence that we actively think this is wrong. And then we can use that score and use a threshold to convert that into what the predicted correct output is.
- 7:21
So what do we think the, the right answer is? And these are two pieces of information that we can then use in different ways.
- 7:28
The first thing we can do is we can predict the estimated performance on all of the cases real-time as we're processing them. So we get our medical decisions coming in, we put them to our reference-free evals, and we get our predicted correct outputs.
- 7:40
We can then see across all of these cases, not just the ones that we're doing human reviews on, how do we think we performed? And that's useful because we can then respond to that, and we can feed that back to customers.
- 7:49
We can then take our cases where we did do human reviews, as well as reference-free evals, and we can compare those outputs. Based on that, we can compute an alignment and see how well is our system doing and how much can we trust it.
- 8:02
And another thing that we can do is we can take our confidence grading rather than our predicted outputs from our reference evals, and we can combine those with contextual factors, things like the cost of procedure, the risk of bias, the previous error rates, and we can use those to dynamically prioritize the order for cases.
- 8:18
So we can identify the most relevant cases with the highest probability of error to then prioritize those for human review.
- 8:26
And this creates this virtuous cycle where we can keep on using the human reviews to validate and improve our performance, and then we can prioritize cases dynamically and keep on feeding that back.
- 8:35
So our reference-free evals surface the cases, and then our human review determines accuracy. And we can keep on doing this in a process that's often described as validating the validator.
- 8:44
And over time, the number of edge cases that we've never seen before gets smaller and smaller, and our ability to detect them improves. And now we've built something that's really hard to replicate.
- 8:52
So while a competitor may be able to make a similar product, you can only build this system by processing high volumes of real data and going through a number of data-driven iterations.
- 9:01
And once we're confident in the performance of our system, we can then actually incorporate it into the pipeline itself. So now it looks like the following. We have our inputs, we pass it through our original pipeline, and we generate our outputs.
- 9:11
And we then pass it into our reference-free evals. And depending on what the reference-free eval output is, we can either give it back to the customer because we're confident in the response that we're giving, or we can decide to take a further action.
- 9:22
And this further action might be that we send it off to another LLM pipeline, perhaps with more expensive models. It might be that we want to do a human review internally and give it to an on-call clinician to review it and then return it to the customer.
- 9:33
Or it might be that we want to actually surface it into the customer's review dashboard so that their team can review it. But all together, this becomes a powerful mechanism for us to really ensure that our customer can trust the outputs that we're giving to them.
- 9:45
So what's the impact been for us at Anterior? How has this helped us? Well, the first thing it's enabled us to not do is to hire out an ever-expanding team of expert clinicians to review these cases.
- 9:55
One of our biggest competitors has hired over eight hundred nurses to perform reviews. Now, we haven't needed to do this. Instead, we're able to review tens of thousands of cases with a review team of less than ten clinical experts.
- 10:09
We've been able to achieve very strong alignment after several iterations between our AI and human reviews to a level that is comparable with the alignment we see between our human reviewers.
- 10:19
And we're now able to quickly identify and respond to errors. So using this example from earlier, we can quickly go from this incorrect answer to a correct answer. This means we're able to respond quickly and still meet customer SLAs around time expectations, and we can be confident in the results that we're returning to them.
- 10:35
And the ultimate impact of this is that we now have provably industry-leading performance at prior authorization, with an F1 score of nearly ninety-six percent in a recent study. And this has enabled us to gain customer trust, and beyond even customer trust, this has led customers to love our product.
- 10:52
In a recent case study, we saw that one of the nurses, after they were told they could keep on using Florence, our AI, said, "Thank God, we're the lucky ones."
- 11:02
So the principles that we followed for building our system and what we would recommend is firstly, make sure you build a system. You know, think big. Don't just use review data to, to audit your performance.
- 11:14
Use it to build, audit, and improve your auditing system, your evaluation system. The second thing is evaluating on live production data. Don't rely on offline evals. Identify problems immediately so that you can respond to them quickly.
- 11:27
And thirdly, get the best reviewers and empower them. Prioritize the quality of reviews over quantity, and build your own tooling if that helps you to move faster. This is how we built an evaluation system that gives real-time performance estimates, enabling us to respond, is accurate, that can scale to meet demand while maintaining a low cost, all powered
- 11:47
by a small, focused team of experts. And it's enabled us to go from our MVP to now serving customers and maintaining their trust at scale. And it's how we think you can too.
- 11:58
So thank you for your attention. Um, would love to talk more about this. If you have any thoughts or ideas, please reach out to me. My email here is, is [REDACTED:email_address].
- 12:05
And we're also hiring at the moment, so if you want to be at the cutting edge of LLM application in healthcare, then check out our open posts at anterior.com/company.
- 12:13
Thank you.