How to Leverage Domain Expertise — Chris Lovejoy, Notius Labs

How do you take what a doctor knows and build it into an AI product? Christopher Lovejoy encountered this problem repeatedly after training at Cambridge, working for several years in the NHS, and moving into AI in 2018. His work included clinical AI at Tandem and US prior authorization at Anterior. Across these companies, the challenge was to turn domain expertise into a differentiated product, rather than simply have knowledgeable people nearby.

The system for incorporating domain insights matters more than the sophistication of the models or pipelines. That is Lovejoy’s starting thesis. The last mile is understanding the particular workflows, use cases, and nuances of the customer being served. He reports that his previous talk reached roughly 100,000 people across platforms; the recurring follow-up was organizational: which experts should a company hire, and where should they sit so their knowledge changes the product? A domain-native AI organization is built to answer those questions.

Winning in vertical AI is, in Lovejoy’s framing, an organizational problem. He distinguishes three models for incorporating expertise: Oracle, Evaluator, and Architect. Each gives domain experts a different relationship to assessing and improving the product.

The commercial attraction is that AI can address work previously performed by people, expanding beyond software budgets into labor spending. Lovejoy uses an approximate comparison of a $50 billion vertical SaaS market with a multitrillion-dollar labor market. The related Bessemer analysis supports the broader opportunity, but uses public-company market capitalization and GDP spending shares; those measures do not establish his approximate market-size figure.

Execution is the harder part. Gartner reports that at least 50% of generative AI projects were abandoned after proof of concept by the end of 2025. Lovejoy proposes insufficient understanding of the workflows being automated—and of how experts actually perform them—as one contributing explanation. That is his diagnosis, not a cause measured by the cited statistic. His view is that frontier models are already good enough for many opportunities; organizations need to operationalize expert judgment around them.

Three recurring mistakes turn that gap into a staffing problem:

• Missing or late expertise: nobody with the necessary judgment shapes the product early enough.
• The wrong expert: the credentials do not match the actual task.
• Poor organizational placement: the right person is present but cannot effectively influence the work.

These lead to three practical questions: why do you need an expert, who should that person be, and how should you use their expertise?

Choosing between AI approaches requires a company to recognize good output. That recognition depends on judgment, and domain expertise supplies the relevant judgment. Sometimes it is formal: doctors for healthcare, lawyers for legal work. Sometimes it is informal expertise developed through doing a task and understanding its users. This does not necessarily require a new hire. Someone already inside the company may perform the function and need the authority and organizational support to do it well.

The three roles differ in where they act on a simple loop: assess the product’s current performance, then improve it.

The Oracle embeds expertise directly, the Evaluator makes quality measurable, and the Architect builds a system that learns from use and user interactions.

An Oracle performs both sides of the loop. They inspect outputs, use the product, examine traces, and identify what is going wrong. They then make changes themselves: often editing prompts, but also adding documents, tools, or other sources of domain knowledge. The important feature is the short path from recognizing a problem to changing the application.

An Evaluator makes assessment a repeatable organizational capability. They define meaningful metrics and build a way to collect them. Depending on the product, that might mean a customer-derived north-star metric, clinicians reviewing a sample of medical outputs, or LLM-as-judge evaluations. Engineers predominantly implement improvements using those findings, with the expert helping them understand what works, what fails, and what needs to change.

An Architect steps back from individual assessments and fixes to design a system that performs both with limited human intervention inside the loop. The expert’s contribution is knowing how to structure automated assessment and improvement. Lovejoy points to another talk for the underlying improvement mechanisms; this framework specifies who designs that capability, rather than an implementation recipe for it.

The selection process starts with measurability, then asks about capacity and speed:

1. Can meaningful metrics capture performance? If quality depends primarily on taste and cannot be adequately measured, use an Oracle.
2. Can one Oracle cover the work? If not, divide responsibility among experts who own different subsets of outputs.
3. If quality is measurable, is manual iteration fast enough? If engineers can respond to the findings at the required pace, an Evaluator is sufficient.
4. If manual iteration is too slow, can improvements be automated? That creates a need for Architect capabilities.

The distinction is operational: a measurable output makes an evaluation system possible, while an engineering bottleneck can make automated improvement necessary.

Small startups commonly begin with an Oracle because one person can inspect the product and immediately improve it. Oracle → Evaluator → Architect is not a mandatory maturity ladder. Change the arrangement when the current process breaks at scale, and only when the next approach is feasible. Moving toward evaluation requires a meaningful metric; moving toward architecture requires both an iteration bottleneck and identifiable methods for automating improvement. Another valid direction is simply more Oracles with distinct areas of ownership.

Lovejoy introduces Granola, the AI meeting-notes company, as having recently passed a billion-dollar valuation. His example of an Oracle is Jo, whom he describes as a writer and journalist who joined as the first employee and wrote all the prompts. She researched what makes a good meeting note through extensive reading and conversations with hundreds or thousands of users, becoming the primary gatekeeper of AI quality.

Her work closes the entire loop: examine the notes produced by the current product, decide what needs to improve, and directly revise the prompts. The arrangement fits because there is no objectively perfect meeting note. Human taste remains central, and the product’s concentration on one core output makes a central review-and-improvement function workable even as the company grows.

That does not mean Granola lacks evaluations or contributions from other people. Lovejoy says the company also runs evals and has internal tooling that lets others contribute to prompts. The Oracle model describes where primary judgment and direct iteration remain concentrated, not an absence of supporting infrastructure.

Tandem’s medical AI scribe also generates notes, but from doctors’ consultations. Its first domain expert, Roy, was a doctor with McKinsey experience. In Lovejoy’s account, Roy initially reviewed medical notes and updated prompts himself: the same Oracle loop. As the product scaled, one person could no longer cover the work.

Tandem responded with decentralized Oracles. Additional doctors took responsibility for subsets of the product, and the platform grew support for a long tail of prompt customizations. Specialties, countries, note types, and use cases all introduced differences that a single centrally managed prompt could not adequately address. Doctors familiar with those contexts could make the corresponding changes.

Medical expertise was essential, but it did not remove subjectivity: there was still no single perfect medical note. A local expert could own a customer relationship or specialty in a particular location, understand its requirements, adjust the prompt, and make that version live for that context. Lovejoy describes thousands or more prompt variations available across different contexts. Here, scaling expertise meant distributing direct control while providing the platform support needed to keep those variations usable.

Anterior’s prior authorization product addresses a different clinical workflow. A doctor requests a treatment or scan—for example, an MRI—and the request goes to an insurer, where nurses or doctors assess whether it is appropriate. Lovejoy describes himself as Anterior’s first technical employee: he built the initial code and prompts, reviewed the outputs using his clinical judgment, and changed the implementation when a decision was inappropriate.

More customers, demand, and variation made that personal review loop insufficient. He moved into Evaluator work: defining metrics and failure modes, building a review dashboard, and hiring clinicians to review outputs. Their assessments produced performance information the engineering team could use to make changes. Assessment had become a system rather than one person’s available attention.

The next bottleneck was fixing the problems. Organizations interpreted policies and rules differently, so manual engineering iteration could not keep up with the variation. Lovejoy describes needing to learn at the edge, where those customer-specific interpretations appeared. That pushed his role toward designing methods for automated improvement, with the implementation details again deferred to another talk.

The progression made sense because, in his account, the relevant decision quality was measurable against medical evidence. The product’s outputs were approval or escalation for clinician review, not approval or denial. Determining whether the chosen action was appropriate still required clinical reasoning. Variation in policy interpretation then created a reason to adapt from usage rather than rely entirely on centrally implemented fixes. Finally, the earlier Oracle work supplied an advantage: firsthand knowledge of failure modes helped determine what the evaluation system should assess and what the improvement system needed to change.

For an Oracle, relevant expertise means direct experience of the use case. A medical qualification alone may not qualify someone to shape a medical coding product. Have they done medical coding? Do they understand its process and failure modes? The same specificity applies to legal and other professional work. Prompting and context engineering are helpful but comparatively learnable. Depth of insight, attention to detail, and customer communication matter because the expert must discover requirements and translate subtle quality judgments into product changes—the kind of detailed work illustrated by Jo at Granola.

An Evaluator needs domain expertise plus data science intuition: choosing useful metrics, collecting them reliably, and making them actionable. The supporting skills depend on how the evaluation system operates:

• Statistics: analyze measurements as review volume grows.
• Industry connections: recruit qualified reviewers from a relevant network.
• Leadership: manage the review team.
• Product management: translate findings into effective collaboration with engineers.

These skills support ownership of a measurement system, rather than merely the ability to score individual outputs.

An Architect additionally needs experience with LLM-powered products and the available ways to improve their performance. The earlier skills remain relevant, while engineering ability helps the expert implement changes or steer their development. Domain expertise remains the baseline: technical knowledge is useful because it gives the expert more ways to turn that judgment into an improving product.

Define a principal domain expert: one person ultimately accountable for AI quality, with authority to make decisions. This avoids consensus by committee, where shared responsibility leaves nobody clearly responsible. It also gives someone the time and space to understand performance deeply enough to make informed decisions quickly.

Give that person ownership beyond an advisory role. They should be in the room when product decisions are made. Having an expert review outputs and hand the results to someone else captures only part of their value; the larger opportunity is to let them shape the system through which the organization measures and improves quality.

Lovejoy describes an unnamed company with two senior clinicians but no clear principal expert. Final authority was ambiguous, their roles were largely advisory, and progress on the wider quality-improvement system was slow. Both clinicians left after roughly 12–18 months. He suspects insufficient ownership contributed to their departures; the organizational loss was also the accumulated context they took with them.

The third principle is to hire for breadth without expecting one person to possess every adjacent skill. Domain expertise is the foundation; seek as much relevant breadth as practical, then pair the expert with complementary specialists. If statistical expertise becomes necessary, for example, a statistician can collaborate with them.

Breadth creates room for the role to evolve. Someone hired solely for narrow domain knowledge may struggle to take on evaluation-system or architecture responsibilities if the company later needs them. Preserving continuity is valuable because Oracle work builds detailed knowledge of the product: which failures recur, which distinctions matter, and what should be measured. That accumulated understanding makes an experienced Oracle well placed to become an Evaluator.

The practical staffing recommendation is to establish a principal expert early, give them ownership, and usually begin with direct Oracle work. As the product and organization change, that person may enable decentralized Oracles or move toward Evaluator and Architect responsibilities. The appropriate direction depends on the use case, scale, and bottleneck; the organizational playbook is still developing. Lovejoy closes by offering the slides on his website.