# Understanding AI Stakes to Break Production Code

Philip Rathle, Neo4j

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/gWLVu9OGvBg-understanding-ai-stakes-break-production-code) · [Video](https://www.youtube.com/watch?v=gWLVu9OGvBg)

## Understanding AI Stakes to Break Production Code

Production AI requires more than a convincing prototype: the consequences of error determine where reasoning belongs, how systems are evaluated, and what users must be able to inspect.

Familiarity with language models, vector search, and basic application pipelines will help; the Python example uses only the standard library.

### Why does an impressive prototype stall?

[0:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=23s)

What changes between an impressive AI demonstration and an application people can safely depend on? The answer starts with **application stakes**: what happens when the system is wrong. Philip Rathle opens this production roundtable by asking participants to compare the hurdles they have solved with those they have not. The discussion operates under the [Chatham House Rule](https://www.chathamhouse.org/about-us/chatham-house-rule), with audience contributions kept anonymous. Its organizing premise is that higher stakes bring higher acceptance bars—and different engineering obstacles.

Generative AI initially looks like a hockey stick: capability rises quickly, and compelling prototypes are easy to build. Actual projects encounter a production barrier. Adding **retrieval-augmented generation**, or RAG, with vector search can move some applications across it. Others still fall short.

![Slide titled “Actual Project Experience with GenAI” shows an LLMs curve below a yellow line labeled “GenAI Go-to-production Barrier.”](https://ai.engineer/api/talk-essay-media/725acef11d8bc44f6d012badfbc5187b22a662a3213b21bfef69fb4b736f191c)

Actual project experience: the GenAI go-to-production barrier.

Rathle presents knowledge graphs and **GraphRAG** as a further step in Neo4j’s approach. But even this progression is better understood as successive S curves than as unlimited acceleration: a technique unlocks a class of applications, then encounters another limit. The practical question is which barrier matters for the application being built.

| Application position | Example or consequence | What changes the stakes |
| --- | --- | --- |
| Lower stakes | Bedtime stories; some text summaries | Several answers may be acceptable |
| Intermediate | A business copilot | Human oversight mitigates consequential errors |
| Higher stakes | An autonomous pilot | Errors can affect money, reputation, health, or safety |

These are positions on a spectrum, not fixed labels for entire product categories. Regulation, fines, and bias also raise the consequences of failure. A human in the loop can move a consequential application toward the middle: the system assists a decision instead of owning it.

### An apple, a vector, and a knowledge graph

[3:51](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=231s)

Higher-stakes applications often need enterprise-specific facts and up-to-the-moment information that a model cannot reliably supply from its parameters. Retrieval brings that information into the answer-generation process. It can also reduce hallucinations, but Rathle’s metaphor of repeatedly loading the dice captures the remaining problem: improving the odds does not establish certainty.

The acceptable residual risk depends on the task. Rathle asks how tightly to fasten bolts on a 737 MAX 9. In Rathle’s hypothetical aircraft-maintenance example, even a 0.001% chance of error is unacceptable. That figure expresses his safety concern, rather than a measured model error rate or an established aviation requirement.

Now consider an apple. A person’s concept of it combines appearance, taste, texture, and other associations. A vector represents it as coordinates. Cosine similarity or spatial distance can use those coordinates to retrieve related fruit; that is a useful operation, but it is not the whole concept of an apple. Rathle characterizes LLMs and vector RAG as statistical ways of approximating the world through patterns such as word proximity and frequency.

His right-brain analogy describes the resulting behavior: creative, often right, occasionally very wrong, and difficult to explain. Knowledge graphs provide the complementary representation. Explicit structure makes information understandable to people and available for machine reasoning—the left-brain side of his analogy. These are metaphors for system behavior, not a neurological account of language models.

**Structured knowledge and language generation have different jobs.** A graph cannot do everything an LLM can. Combined with an LLM, however, it can supply facts, relationships that support reasoning and discernment, and persistent information that serves as long-term memory. The proposal is to combine those capabilities rather than expect either representation to solve every problem.

### Put exact reasoning in the right component

[6:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=368s)

The discussion turns to a practical question: which lesson would have saved time if it had been learned sooner? Rathle’s first answer is to understand the stakes early without letting planning stop iteration. Start building, but let the consequences of failure influence tool selection.

His term for the next decision is **locus of reasoning**. If a question has an exact answer, a database or another reasoning engine may be the right place to compute it. The LLM can handle the language on either side:

1. Translate the user’s question into a query.
2. Have the database or reasoning engine compute the result.
3. Express that result in language appropriate to the audience.

Deterministic execution does not make the whole pipeline correct: the generated query must still express the intended question, and the final response must preserve the result. But it moves the exact computation into a component designed for that work.

Audience adaptation then becomes a separate task from determining the answer. A response to a regulator should be serious and joke-free. An outbound marketing message might use different language for a high-school student and a retiree. The facts need not change when the presentation does.

An audience member offers a concrete version of this separation: instead of pasting Excel data into an LLM and asking it to identify an outlier, ask it to write Python that finds the outlier. The resulting computation can run without keeping the LLM in the execution loop. The model’s ability to write a procedure can be more useful than its ability to answer the numerical question directly.

For a small illustrative dataset, an explicit interquartile-range rule makes that division of labor visible. The values and rule below are a worked example of the suggested approach:

```python
from statistics import quantiles

values = [10, 11, 12, 12, 13, 14, 50]
q1, _, q3 = quantiles(values, n=4, method="inclusive")
iqr = q3 - q1
lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr

outliers = [
    value
    for value in values
    if value < lower_bound or value > upper_bound
]

print(outliers)  # [50]
```

Here the program applies a specified definition of an outlier. Choosing whether that definition fits the data remains a separate decision; the benefit is that the calculation itself is inspectable and repeatable.

### Semantic search still meets keyword habits

[8:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=532s)

A team can build good retrieval and still misunderstand how people will use it. One participant describes investing in RAG, indexing, and semantic search, then discovering in production that customers continued entering keywords. The team had expected natural-language questions; users brought the habits of a familiar search box.

That mismatch required work to interpret or rewrite the input: was it a keyword query, or a question? The team had spent heavily on making search work before asking whether this was how customers wanted to search. Retrieval quality alone did not resolve the interaction problem.

Rathle asks whether a curriculum could help users discover that they can ask more than keyword queries. The participant acknowledges that onboarding had initially received little attention: the assumption was that everyone would simply start asking questions. Easing users into the new behavior became part of the product work, alongside making the underlying search function well.

### Choose projects that can meet the acceptance bar

[10:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=613s)

Another production lesson is to kill more projects during experimentation. Running many experiments is useful only if the team eventually concentrates effort on the few that can meet users’ expectations. Some projects will not reach the required accuracy even with substantial additional work. The participant would rather focus on one or two that can make it all the way to production.

Rathle raises both achievability and business value as selection criteria. The response brings the discussion back to the user’s acceptance bar. A participant uses 90–95% accuracy as an illustrative level that may be insufficient for one use case, while 90% may be acceptable for another. No task, dataset, or accuracy definition accompanies those figures; they illustrate why expensive commercialization should follow a match between achievable performance and user needs.

A brief contribution sharpens the same point: expect less and use more common sense. That does not mean abandoning ambition. It means avoiding a product plan that depends on capabilities the system has not demonstrated.

The economic version of the problem begins with a customer asking for the best model at the fastest possible speed. Those are not sufficient requirements. Cost–quality and cost–speed trade-offs depend on what the application actually needs to do and how quickly it needs to respond. Without that understanding, a deployment can spend far more than necessary.

A small laboratory question set naturally encourages maximizing accuracy; production exposes the cost of doing so repeatedly. Rathle imagines a deployment consuming a year’s budget in a day to illustrate the danger of ignoring scale. His suggested response includes smaller models, with the choice driven by required quality and latency rather than by an unconditional demand for the strongest model.

### Ship now without freezing the model stack

[12:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=746s)

Rapid model improvement complicates the investment decision. The moderator tentatively recalls Bloomberg training a financial-data model, perhaps based on **Llama 2**, that performed well in evaluations. The moderator recalls **Llama 3** outperforming that financial fine-tune, but the model identity and comparison are unverified. The anecdote raises a useful question without settling it: how much work will a subsequent base-model release make unnecessary?

Waiting is not a complete product strategy either. Enterprises want something working today, and teams want the competitive advantage of shipping. Rathle initially frames the wait in years; participants push back that meaningful advances are arriving on the scale of weeks. The engineering problem is to deliver value while retaining the ability to benefit from those advances.

One participant’s approach is to establish the full path through the system first, then improve its stages in parallel. Deliver an end-to-end result for the customer. If the first data-engineering step will not scale, assign work to improve it. If acceptable model performance currently requires an expensive option such as **Claude 3.5 Sonnet**, keep that stage replaceable as model capabilities and economics change. These model names describe the choices discussed at the time of the recording.

**A working pipeline makes component improvements usable.** Once the steps and their boundaries are understood, better components can be substituted into the path rather than requiring the product to be reinvented. Rathle connects this to the rising capability curves from the opening: improvements become valuable when the system has somewhere to put them.

That approach has a limit the participant has not solved. More agentic behavior makes orchestration itself a moving target. The team is looking for better tools, and it has already had to slow down because too many layers were changing at once. Interfaces and contracts between layers are necessary so that improving one stage does not unexpectedly break a downstream stage. The participant presents this as ongoing work, not a finished orchestration solution.

### Validate capability before optimizing it away

[14:55](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=895s)

The next question is how long an optimization should take. If a latency improvement requires months of engineering, a new model may arrive before the work finishes. A participant proposes putting a ceiling on implementation time, without specifying a universal number of months. Rathle’s two-year-project example pushes the concern to an extreme: a long project aimed at a temporary model limitation may become obsolete before it delivers value.

A different participant warns about optimizing too early. Teams that begin with a small fine-tuned model can incorrectly conclude that a task is impossible. Starting with a more capable model helps distinguish a genuine product or capability problem from a limitation introduced by the initial cost constraint. Production latency still matters, but it should not prevent the team from first learning whether the desired behavior is achievable.

The resulting sequence is:

1. **Validate the product need and capability.** Establish whether a strong model can perform the task well enough.
2. **Account for proprietary information.** Determine whether missing domain data calls for retrieval or fine-tuning.
3. **Optimize the working behavior.** Move toward faster, cheaper models, potentially using larger-model outputs as fine-tuning data, while checking that accuracy holds.

Rathle names **GPT-4o** and Claude 3.5 as starting points for that feasibility check. His heuristic is that if the stronger model cannot do the task, a smaller model probably will not—while acknowledging that highly proprietary data changes what the model must be given.

### Evaluation starts with the data you will need

[16:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1006s)

Consider a pipeline in which an LLM converts unstructured material into structured data for a traditional machine-learning model. The application needs a way to determine whether that conversion is good enough and whether subsequent changes improve it. A participant’s lesson is to build the evaluation pipeline from the start—and first ask whether the necessary evaluation data exists.

If suitable data is missing, begin generating it on day zero. Otherwise, a team may reach the point of having an LLM embedded in its application without being able to tell whether a prompt change made the system better or worse. Designing evaluation only then creates a long delay: the application exists, but the evidence needed to guide its development does not.

Rathle places this alongside the usual operational framework. **DevOps, evaluation, and data quality are connected responsibilities.** The ability to deploy a change is not the ability to judge it; the evaluation framework, in turn, depends on having the right data at sufficient quality.

### Make generated answers recognizable and inspectable

[18:06](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1086s)

One team deliberately avoids hiding AI behind an apparently ordinary interface. A participant describes gradually displaying generated responses as a cue for users to be vigilant, along with distinct colors and icons that identify AI interactions. The intent is to make the nature of the output visible before users treat it as authoritative.

The interface also gives users tools to inspect an answer: what sources were used, what inputs went into it, and what might have gone wrong. The team does not simply abstract generation away and hope for the best. Rathle connects visible generation to moderated expectations: seeing an answer emerge can remind a user that it deserves consideration rather than automatic acceptance. These are the team’s reported interface choices; the discussion does not establish how much they change user vigilance.

### Human review needs more than a citation list

[19:05](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1145s)

Explainability becomes especially concrete when the output could influence medical treatment or equipment maintenance. In response to Rathle’s question about regulation, a behavioral-health practitioner describes the team’s current operating principle: always retain a human in the loop. They are building a copilot. The participant’s remark about never becoming a pilot is offered as reassurance to others, rather than as a firm technical prediction.

For that human review to mean something, the reviewer needs access to the inputs that produced the output. Recommendations and selected case information must be traceable to root data. Rathle asks whether showing inputs is enough to earn trust, then points out the practical burden of references: users may not read three 50-page papers just to check an answer. **Evidence must be available in a form that fits the review task.**

![Slide asks “is explainability a blocker?” and lists “regulation, Q/A, stakeholder buy-in, ...” with the presenter inset at lower left.](https://ai.engineer/api/talk-essay-media/01d5d64f16d4835dd13d27b66569f72dedce415fc0bff88cedec2478839572ac)

Is explainability a blocker? Regulation, Q/A, and stakeholder buy-in.

The practitioner describes two review arrangements:

- **Knowledge lookup:** show the retrieved source documentation beside the answer, with citations.
- **Therapy-session summary:** show the collected session data beside the generated summary, paragraph by paragraph.

For summaries, the user can change and edit the resulting paragraphs. The system supports a reviewable draft; the final summary becomes the user’s work rather than an untouchable model artifact.

Rathle then asks whether human judgments are used to improve the producer. The practitioner describes several checks and feedback channels:

1. A separate, quick evaluation stage checks whether the output matches the inputs, including a safety-oriented check.
2. User edits and thumbs-up or thumbs-down responses provide feedback.
3. The team performs its own clinical evaluation of outputs.

This describes layered evaluation and feedback collection, not a specified training procedure. The team inserts checks wherever it needs additional confidence in what the system is producing.

### When a human reviewer cannot be included

[22:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1333s)

The final example tests the limit of human oversight. A participant working on regulated personal-loan recommendations cannot include a human reviewer in the workflow. Instead of declaring a single loan best, the team gives the end user three options with explanations. The participant describes this as less deterministic in the sense that the product presents choices rather than one definitive recommendation, and reports that it helps address regulatory concerns; the discussion does not establish compliance with any particular rule.

Rathle checks who receives the options: an intermediary reviewer or the end user? They go directly to the end user because intermediary review is unavailable. The participant adds that the selection uses logic beyond raw LLM output, although the logic itself is not specified. The final design therefore changes the decision the AI is allowed to make: it supplies explained alternatives for the user to consider, rather than presenting one generated answer as the decision.

## Resources

- [The GraphRAG manifesto](https://neo4j.com/blog/genai/graphrag-manifesto/) — Philip Rathle explains graph-enhanced retrieval, contrasts vector and graph representations, and develops his argument for enterprise GraphRAG.
- [Neo4j GraphRAG Python user guide](https://neo4j.com/docs/neo4j-graphrag-python/current/user_guide_rag.html) — Current examples for vector search, hybrid retrieval, graph traversal, and natural-language-to-Cypher pipelines.
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) — Foundational research combining pretrained language generation with retrieved Wikipedia passages.
- [BloombergGPT: A Large Language Model for Finance](https://arxiv.org/abs/2303.17564) — Bloomberg's original account of its financial language model, training data, and evaluation methodology.
- [Chatham House Rule](https://www.chathamhouse.org/about-us/chatham-house-rule) — Official explanation of the rule permitting information sharing while protecting participant identities and affiliations.

## Slides

### Presenting Sponsor

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=5s) [Microsoft presenting sponsor card.](https://assets.ai.engineer/sites/aiecode2025/sha256/41458fcb702ed14e0a2c14bc7d4a7f9af33564d53de1b6f45d83d20f51dfee38.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and Platinum Sponsors

AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=10s) [AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/666dee55a1ad420a213956a4788c082983571775efe5b60faf62e72fdcf57577.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### Roundtable Goal

The roundtable asks participants to learn from others’ successes and apply those insights to their own challenges getting to production.

[0:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=43s) [The goal is to gain insights into getting-to-production challenges by learning how others are succeeding.](https://assets.ai.engineer/sites/aiecode2025/sha256/3c7fcc1fbc3e64b3bbfb8c9a1d245898b437b25a73bbd094b1c1e351841f4f2d.webp)

Goal: Come away with insights about your “getting to production” challenges by learning from others about how they are succeeding

AIE
aws

### AI Round Table Rules

The roundtable asks participants to figure things out together and follow Chatham House Rules: what happens in Nob Hill stays in Nob Hill.

[0:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=59s) [The rules are to figure things out together and follow Chatham House Rules, with a footnote about keeping discussions in Nob Hill.](https://assets.ai.engineer/sites/aiecode2025/sha256/7089d11f0195fbcb64ea836c261317d68a5024596ee496f850764a14c438eb96.webp)

Two rules of the AI Round Table:
1. We all figure this out together
2. Chatham House Rules*

*What happens in Nob Hill stays in Nob Hill

AIE
Microsoft
smol ai

### Personal learnings about getting to production and stakes

The speaker introduces his experience getting to production, focusing on how the stakes of a problem relate to the bars and obstacles teams encounter.

[1:06](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=66s) [Section title: personal learnings about getting to production and stakes.](https://assets.ai.engineer/sites/aiecode2025/sha256/b22f18a639cd4999b0859e2d2170e43eb2591cc2bcd33aa3dadf640382c20482.webp)

My personal learnings about
“getting to production” and stakes

AIE

[1:19](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=79s) [Section title introducing personal learnings about getting to production and stakes.](https://assets.ai.engineer/sites/aiecode2025/sha256/3912821dd0a9619b03d4df2a71242668e68d8d39ad7093b2642368b56844a508.webp)

My personal learnings about
“getting to production” and stakes

AIE
aws

### GenAI is a hockey stick

The LLM curve starts flat and turns sharply upward, illustrating a hockey-stick expectation for GenAI.

[1:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=114s) [A steep upward curve labeled LLMs illustrates a hockey-stick expectation for GenAI.](https://assets.ai.engineer/sites/aiecode2025/sha256/c5568141707352636ad1c09f50de4bea5fc0f225ba466f0185b0adcbb22767d1.webp)

GenAI is a hockey stick - woo hoo!

LLMs

AIE
aws

### Actual project experience with GenAI

The LLM curve rises but remains below the go-to-production barrier. The speaker says impressive prototypes are easy to build, while production has a higher bar.

[2:00](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=120s) [The LLM curve sits below a horizontal GenAI go-to-production barrier.](https://assets.ai.engineer/sites/aiecode2025/sha256/7b4df73104614a13bbc04c0f43dcfdc47752c61a21c45216cf9e60afef1e919a.webp)

Actual Project Experience with GenAI

GenAI Go-to-production Barrier

LLMs

AIE
Microsoft
smol ai

### Now it’s a hockey stick

The slide sketches a second steep rise with RAG and vectors after LLMs and fine-tuning. The speaker says this can reach production for some use cases, but others still need more.

[2:15](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=135s) [Curves labeled LLMs plus fine tuning and RAG with vectors show a second steep rise.](https://assets.ai.engineer/sites/aiecode2025/sha256/fecdb56bf95d68371db6e3b925cfd110eb9571ef240f3b9c3afbcdd768a785a1.webp)

NOW it’s a hockey stick!!

LLMs +
Fine Tuning

RAG with Vectors

AIE
Microsoft

### Actual project experience with GenAI

The slide shows LLMs and vector RAG leveling off below a higher bar. The speaker says vector RAG reaches production for some use cases but falls short for others.

[2:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=146s) [The LLM and RAG curves level off below a horizontal barrier.](https://assets.ai.engineer/sites/aiecode2025/sha256/baf1210ff9cbc0d386de7a7b216153f7d173d38d091af9838f1c9cc5e15734f7.webp)

Actual Project Experience with GenAI

LLMs +
Fine Tuning

RAG with Vectors

AIE
Microsoft
smol ai

### Next Unlock: Knowledge Graphs

The speaker presents knowledge graphs and GraphRAG as another potential unlock after LLMs and vector RAG, sketching progress as a series of S-curves.

[2:31](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=151s) [A stepped chart progresses from LLMs to vector RAG and GraphRAG; the presenter inset covers part of the lower left.](https://assets.ai.engineer/sites/aiecode2025/sha256/d9d87acbcf5c56acc7a281c44a1fc974b91115aafa53b2f6365d60a2f21bde5b.webp)

Next Unlock:
Knowledge Graphs

GraphRAG
RAG with Vector
LLMs

AIE
aws

### How are we using data to represent the world?

An apple introduces the question of how data represents the world. The speaker asks us to consider the multidimensional, multisensory way humans conceptualize it.

[4:41](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=281s) [A red apple beneath a question about representing the world with data.](https://assets.ai.engineer/sites/aiecode2025/sha256/8fc03dd699e5d3f40d553b8ac17d637e481085aa8cb8cc7586ebd61e96178302.webp)

How are we using data to represent the world?

AIE
Microsoft
smol ai

### Human and vector views of an apple

The speaker contrasts a multisensory human view of an apple with a vector of numbers, which can be compared using cosine similarity or spatial distance to find other fruit.

[4:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=294s) [An apple image is compared with a block of numerical vector values.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c835a06c21230f389abf2d22255f8d50d63224596322e64c9ca8251100812aa.webp)

Human View of an Apple

Vector View of an Apple
[illegible]

AIE

### LLMs with vector-based RAG

The speaker likens vector-based RAG to a right brain: creative, but sometimes wrong in ways that are hard to explain. He describes its word-model view in terms of word proximity and frequency statistics.

[5:16](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=316s) [A right-brain illustration accompanies the slide's descriptions of vector-based RAG as creative, nondeterministic, lacking critical thinking, and a black box.](https://assets.ai.engineer/sites/aiecode2025/sha256/e3daee2a61aced00478aadf9fae51b3db67213c4b1c6fc5f8a3fa4edf98bb840.webp)

LLMs with Vector-Based RAG
a ‘word model’ view

Right Brain

Creative
Spontaneous / Nondeterministic
Lacks critical thinking
Black box

AIE
Microsoft
smol ai

### Human, vector and knowledge graph views of an apple

The apple appears as a picture, a numerical vector and a graph of relationships. The speaker presents the graph as a structured representation humans can understand and machines can reason over.

[5:47](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=347s) [An apple image and numerical vector are compared with a graph linking an apple to its origin, structure, kind, shape, size and color.](https://assets.ai.engineer/sites/aiecode2025/sha256/e61527ca15afe03109dd25accd839787ca01b97868c5ca4a2fb687f875f65283.webp)

Human View of an Apple

Vector View of an Apple
[illegible]

Knowledge Graph View of an Apple
Apple
origin
structure
kind
tree
body
stem
fruit
shape
size
color
round
10cm
red

AIE
Microsoft
smol ai

### Knowledge Graph: A World Model View

The speaker describes knowledge graphs as structured representations that humans can understand and machines can reason over, complementing LLMs with facts, reasoning, discernment and long-term memory.

[5:56](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=356s) [A brain illustration labeled left brain points to facts, reasoning, discernment and long-term memory.](https://assets.ai.engineer/sites/aiecode2025/sha256/233fc387068f557ff1fac558b2b94eec31926ff8540aef19b0ccd440af3f5b50.webp)

Knowledge Graph
a ‘world model’ view

Left Brain

Facts
Reasoning
Discernment
Long-term Memory

AIE
Microsoft
smol ai

### Round Table Discussion

The speaker opens the round table by asking attendees to share a learning that would have saved them time.

[6:22](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=382s) [Title slide reading Round Table Discussion.](https://assets.ai.engineer/sites/aiecode2025/sha256/6baa538c4f9c90cd2101560b4477b59e65e2f73d9990df4e248115c8ad1d8278.webp)

Round Table Discussion

AIE
aws

### What Is One Learning That Would Have Saved You a Lot of Time?

The speaker opens the discussion by asking for lessons learned. His example: assess an application’s stakes early and use an LLM to turn questions into queries for systems that provide deterministic answers.

[6:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=388s) [Question asking what one learning would have saved the audience a lot of time if known sooner.](https://assets.ai.engineer/sites/aiecode2025/sha256/332827dac2a4e7bdae9b9a1a664641aaee8053a66eb33fda6d385db9c5246031.webp)

Question: what is one learning that would have saved you a lot of time if you had known it sooner?

AIE

[7:31](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=451s) [Slide asks what one learning would have saved the audience a lot of time if they had known it sooner.](https://assets.ai.engineer/sites/aiecode2025/sha256/dc9b14917dc8b03b0c93bfe21f5feb8a0760752dfbc3483bcd111ee89197ff2d.webp)

Question: what is one learning that would have saved you a lot of time if you had known it sooner?

AIE

### One learning you wish you had known sooner

Panelists recommend validating the product with a capable model before optimizing, building an evaluation pipeline alongside the app, and making AI-generated answers visible and verifiable to users.

[16:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=961s) [A slide asks what one learning would have saved the audience time if known sooner.](https://assets.ai.engineer/sites/aiecode2025/sha256/5cfa71f240e8517278e83c664f0b14b280b0b2205177a2fcd358442e5739abc6.webp)

Question: what is one learning that would have saved you a lot of time if you had known it sooner?

AIE
Microsoft
smol ai

[18:38](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1118s) [Slide asks what one learning would have saved the viewer time if known sooner.](https://assets.ai.engineer/sites/aiecode2025/sha256/a3973f3b1876364a2195d3ebf4c9edff51602cc4767778a83530145e31c6ee53.webp)

Question: what is one learning that would have saved you a lot of time if you had known it sooner?

AIE

### Is explainability a blocker?

A behavioral health team keeps a human in the loop and shows source data beside summaries. Another participant offers three explained loan options in a regulated use case.

[19:14](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1154s) [Slide asks whether explainability is a blocker, citing regulation, Q/A and stakeholder buy-in.](https://assets.ai.engineer/sites/aiecode2025/sha256/2aeac48651d6381945cd92ee2dfc2f6d4babacd8193063043cd48d21ca3a01ac.webp)

Question: is explainability a blocker?

(regulation, Q/A, stakeholder buy-in, ...)

AIE
Microsoft
smol ai

[22:22](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1342s) [Slide asks whether explainability is a blocker for regulation, Q/A, and stakeholder buy-in.](https://assets.ai.engineer/sites/aiecode2025/sha256/5e7bfd578280054642572040ced2d257717b807bd4eeb80faa8b2686f187de63.webp)

Question: is explainability a blocker?

(regulation, Q/A, stakeholder buy-in, ...)

AIE

## Full transcript

[0:00](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=0s) [upbeat music] So the idea here is a lot of you are solving some of the most leading-edge AI, GenAI problems in the world in, in, in this room, and then a lot of you are trying ambitious things.

[0:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=23s) So main goal was to get folks, um, learning from each other. So think of this as a logical roundtable, uh, not, not very round physically. Um, so come away with insights about specifically your getting to production challenges by learning from each other.

[0:40](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=40s) So two rules of this AI roundtable. One is we're all-- we'll all be stronger if we figure this out together. Number two is Chatham House Rules. Um, that's for Peter's benefit.

[0:51](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=51s) Um, for us [REDACTED:origin] in the room, what happens in Nob Hill stays in Nob Hill.

[0:57](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=57s) Um, so l-let me actually kick it off by sharing some of my personal learnings. I'll go through maybe f- uh, five, five minutes or so of what I've learned in relation to getting to production, but specifically around the topic of stakes.

[1:11](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=71s) Um, and by stakes I mean, like, how high stakes is the problem that you're trying to solve. And I found there's a relationship between the bars and the obstacles that you run into as the stakes increase.

[1:21](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=81s) So I'll just share my own journey, and hopefully this is useful and food for thought. But as I'm going through it, think about what you're experiencing and the hurdles, 'cause what I'm gonna ask after I go through my own section is what, uh, what are some of the hurdles you are running into?

[1:38](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=98s) What are some of the ones that you haven't solved? What are some of the ones you have solved? So that we can learn from each other.

[1:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=105s) All right, so it all started like this. GenAI is a h-- GenAI is a hockey stick, right? Um, and then we kind of ended up with actual project experience where you can do, and I think Lucas said this earlier, lots of amazing prototypes.

[1:58](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=118s) It's so easy to build an amazing prototype, but then getting to production is harder. You have a higher bar. And so what a lot of folks have figured out is you can use RAG with vectors and all right, now it looks like a hockey stick.

[2:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=132s) And in some cases it does. In some cases you can get to production with this. But then there's still a class of use cases where that's not enough. And so what, you know, you'll hear from the Neo4j folks here, and if you go to the booth and attend Emil's talk and some of the other ones later, is

[2:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=148s) there's another unlock here with knowledge graphs and with GraphRAG. Um, again, like, will this be a hockey stick? I think the world moves in S curves, so this is probably more realistic and there's probably gonna be some other thing.

[2:40](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=160s) I don't know what that looks like. Um, but why this pattern? Um, so I-I've come to see a spectrum of application stakes, and it's a whole spectrum, right? But to kind of paint opposite ends and maybe something in the middle, if something is low stakes, then maybe it's more business peripheral.

[2:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=179s) Um, it's things like summarizing text. Here I have someone, you know, a children's bedtime story. Um, there's no single right answer. Uh, the stakes are a bit lower. Um, and then on the high end of things, like a wrong answer can have significant impact in terms of dollar value, brand or reputational impact, health and human safety, regulation,

[3:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=203s) fines, bias, all that stuff. And then, um, and then there's this in-between zone where, you know, it might be higher business stakes, but I've got some mitigations. Like, I've got a human in the loop.

[3:35](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=215s) Um, so, uh, so here you have a difference between at the far right, um, uh, pilot and in the middle, co-pilot. Uh, so there's kind of an interesting relationship between pilot, co-pilot, and, and stakes.

[3:51](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=231s) And, you know, the problem with high-stakes use cases is you bump into the meme, right? Which you've all seen. Um, and specifically it's things like needing enterprise domain knowledge, up-to-the-moment knowledge.

[4:02](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=242s) This is what RAG solves. Um, solving for h-hallucinations. Again, RAG solve this and depending on how you solve it, you're, you're loading the dice, loading the dice, loading the dice.

[4:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=253s) And then for-- depending on the stakes, your intolerance for error, you know, could be, you know, potentially gets very high, right? How t-- how tight should I tighten my bolts on my 737 MAX 9?

[4:24](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=264s) Like, let's not have a 0.001% chance of error, 'cause that's not good enough. Um, cool. So let's look at how we're representing data. So here's an apple, and think about how we conceptualize an apple as a human.

[4:39](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=279s) It's, you know, it's this multidimensional, multisensory kinda thing. Um, now let's look at a vector view of an apple. It looks something like this. [chuckles] Um, so I can find other fruit by doing a, you know, cosine similarity or spatial distance, um, operation with, uh,

[5:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=301s) w-with this representation of an apple. It's useful. Again, good for certain things, good-- not, not as good for other things. And if I characterize the behavior that you get with LLMs generally and with vector-based RAG, it's this, you know, kinda world of statistically computing, uh, the real world based on, um, word proximity statistics and word frequency statistics.

[5:25](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=325s) So you get this behavior that's kind of a little bit like the right brain. You know, it's creative, it's mostly right, sometimes really, really wrong. I don't really know why it's wrong, it just does its thing.

[5:35](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=335s) Uh, a bit of a black box. Enter knowledge graphs as a way to bump up the, the stakes on things. So this gives you a, um, more structured representation that is understandable by humans, can be reasoned upon by machines, and this gives you a bit more of left brain behavior.

[5:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=354s) So it can't do all the things that LLMs can by any means, but when you combine it with an LLM, you, you-- being able to bring in facts, reasoning, discernment, long-term memory, um, can create a better together.

[6:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=368s) So that's my quick- Um, sharing around stakes and unlocking things in different levels. So let me turn it over to folks in the room, and Peter will be coming around with a microphone.

[6:21](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=381s) So let me start with the question: Does anyone have... What's one learning that would've saved you a lot of time if you'd known it sooner?

[6:33](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=393s) Come on, all at once.

[6:34](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=394s) Anyone, just raise your hand.

[6:36](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=396s) Yeah.

[6:36](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=396s) Blurt it out if you're-

[6:37](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=397s) Well, maybe if, if no one's got one-

[6:39](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=399s) Yeah

[6:39](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=399s) ... maybe you have one, like, just off the cuff. It might inspire other people to have a story-

[6:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=405s) Um-

[6:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=405s) ... get us going.

[6:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=406s) I mean, it, it comes back to stakes and, you know, un- understanding what those are upfront, um, for different applications. So w- we've heard a lot today about iteration and how important that is.

[6:58](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=418s) So definitely iterate. You know, don't overthink things initially. But, you know, to also think and, think and plan ahead, and that can, that can guide things like what tools you use.

[7:09](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=429s) Also locus of reasoning is, is something I think about, is if, if there's something-- if there's a question that has an exact answer and it's a der- deterministic question, then, you know, maybe you can use the LLM, which is inherently non-deterministic, to complement some other technology that can give you a deterministic answer, which you have.

[7:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=448s) You've got databases, um, and, and other kinds of reasoning engines. And then use the LLM to do what's, it's exceptionally good at, which is maybe take the question, uh, turn it into, um, a query.

[7:41](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=461s) Take that query, formulate it back into some text, and then maybe modulate that text depending on who the audience is. If it's going to regulator, then very serious. No jokes.

[7:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=470s) If it's going to, you know, a, an outbound marketing email, then it can, uh, maybe go to, uh, be phrased differently depending on who the target audience is. Is it someone in high school?

[8:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=481s) Is it a retiree? Um, yes, I see a hand, though-

[8:04](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=484s) Oh

[8:04](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=484s) ... though over there.

[8:04](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=484s) Cool. Let's go. [laughs] Here we go.

[8:07](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=487s) Going around.

[8:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=488s) Ah, so I'll go here.

[8:10](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=490s) Um, I've, uh... One, one learning that I wish I had had before is instead of asking the LLM to solve a problem, right, like paste in Excel and say which one's the outlier, just ask it to write the code to solve that problem, which it can easily do, and then you can leave the LLM out of the

[8:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=507s) loop. Um, right. You say, "Find the outlier." It will do some Python outlier machine learning thing, and then you have the answer.

[8:35](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=515s) Cool. Yeah. So the LLM might not be good at certain kinds of reasoning, but it can write code that will do the reasoning, which is a pretty amazing hack, actually.

[8:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=523s) Thanks. Um, I think you were-

[8:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=525s) Oh, everyone's getting involved now

[8:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=526s) ... I know. I'll, you, you, you work out who's next.

[8:48](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=528s) I'll... Wait, I'm sorry. Who had their hands up? Oh, let's go here.

[8:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=532s) Yeah. So one of the things we learned was, um, when we were kind of getting excited about the RAG and indexing and doing the semantic search, and when we put it to production, that our customers were not used to doing queries like this.

[9:07](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=547s) They always resorted back to keywords. It's almost like they were-

[9:11](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=551s) Mm-hmm

[9:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=552s) ... like-

[9:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=552s) Like search

[9:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=552s) ... trained to do this.

[9:14](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=554s) Yeah.

[9:14](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=554s) And so getting them along, uh, and so we had to write, rewrite, like is this question actually a query for keywords or is it like the, like a query they ask like a, a question?

[9:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=566s) So that was a learning that, you know, we, we made, uh, while putting it into production. And, uh, we put a lot of effort in making the search well, [laughs] but we, we, we didn't ask upfront like, well, is this actually how you wanna use it?

[9:39](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=579s) And so that was one learning there.

[9:41](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=581s) And so did you have a sort of curriculum to help people understand how to ask questions and that they can do more than just keywords?

[9:49](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=589s) Uh, so yeah, we, we didn't put attention to this. We were just-

[9:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=592s) Yeah

[9:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=592s) ... assuming like, yeah, everybody's now gonna use like-

[9:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=594s) Yeah. Yeah

[9:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=594s) ... putting questions in. So that was, you know, the surprise and the learning there, like-

[9:58](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=598s) Yeah, that's cool. Yeah

[9:58](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=598s) ... we needed to ease them in.

[9:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=599s) You need to understand what the technology's capable of, and they can do more than just-

[10:02](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=602s) Yeah. But it's like the-

[10:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=603s) ... what you're used to. Like it's not a Google search box

[10:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=603s) ... adapt the user to how the new thing is supposed to work, uh-

[10:07](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=607s) Yeah

[10:07](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=607s) ... in a way. Like that was a learning.

[10:09](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=609s) Cool.

[10:10](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=610s) Thanks.

[10:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=613s) Um, I think the lesson is kill more projects in experimentation. So right now, I think one of the big lessons we learned is we did a lot more experimentation, and r- if I have to look back, I will focus on few that will go all the way to production.

[10:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=629s) Yeah.

[10:30](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=630s) Because a lot of these experimentation, even with the best effort, will never meet the user expectation on accuracy. So if we can kill and... a lot of projects, but focus on one or two that can, we can take all the way to production is probably where I will focus a lot more.

[10:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=646s) And people spend too much time, uh, on not being focused.

[10:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=650s) And when, when deciding what project to focus on, I imagine there's an element of how achievable is it and how valuable is it. H-how, how do you-

[10:57](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=657s) The thing is for us, like it, it... Can we get to the level of accuracy that the user is expecting?

[11:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=663s) Yeah.

[11:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=663s) Commercialization is expensive, but we can do it to 90, 95. That's not good enough. Why are we spending time on it now? But work on the use cases that will go to them, where 90% is good enough.

[11:15](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=675s) Yeah. So matching up with user expectation of accuracy, and then those projects are bound to succeed and go with those. Cool. Thanks.

[11:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=683s) Okay.

[11:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=683s) My learning is, uh, expect less and use more common sense.

[11:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=688s) Yes.

[11:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=688s) It's a funny one.

[11:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=689s) Yeah. I like that.

[11:37](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=697s) Thank you.

[11:38](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=698s) One learning that I've experienced from our customers when they're building deployments is, you know, it's really easy to walk in and say, "I want the best model, and I want it to run as fast as possible."

[11:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=710s) But when you're making, you know, actual cost to quality trade-offs and cost to speed trade-offs, you know, you can, you can really overspend versus what you really need if you don't have a great understanding of what you're actually trying to build, how fast it actually needs to run, that kind of stuff.

[12:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=728s) Yeah. Yeah. That in, in a lab with a small number of questions, you're naturally gonna go for ... the best accuracy, but then enter production if it, you know, if, if, if you're burning a year's budget within a day, which is very possible with this stuff, then maybe you need smaller models or multiple...

[12:24](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=744s) Yeah. Gr- great one.

[12:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=746s) Actually, I just had one pop up in my own mind, um, that I remember seeing, which was when Bloomberg were, uh, training, I think it was Llama 2 at the time-

[12:36](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=756s) Mm

[12:36](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=756s) ... to handle a lot of the financial data that they had, and it passed really well on all the evals and everything. And then [laughs] they found out when Llama 3 came out, it actually outperformed their fine-tuned model they'd spent all this time producing.

[12:47](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=767s) Um, and so I'm not entirely sure what the takeaway from that is, but it's almost this idea that if you just wait for the models to get better, then you kind of save work.

[12:56](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=776s) But then I guess that's not necessarily gonna fly in the enterprise where [laughs] it's like, "We want something working today."

[13:00](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=780s) Well, th- this is a great... I'd love to hear what people in the room are experiencing, 'cause there's this real tension of, all right, if I just wait a couple years, like, this stuff is all gonna figure itself out and it'll be way better, but who wants to wait a couple year?

[13:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=792s) Like, we, we wanna get out and get competitive advantage. [laughs]

[13:16](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=796s) Not a couple years, right? Like, we're seeing crazy advances.

[13:18](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=798s) Yeah, yeah. You wait, you wait a few [laughs] weeks.

[13:20](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=800s) So like, we're, we're just focused on, like, the full path and getting it end to end, and then, like, in parallel tracks, optimizing every part of that, that journey, right?

[13:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=809s) Yeah.

[13:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=809s) So trying to end-to-end ship customer value, and then, oh, our data engineering part of that was like a kludge and that's not gonna scale, so let's get people working on that.

[13:37](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=817s) And it's like an interface between every step of this chain that we're putting together. And so, like, if I encounter an issue where my LLMs aren't performing and it's like, "Oh, no, we gotta use, like, 3.5 Sonnet and it's gonna cost us a lot," well, just wait, you know?

[13:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=830s) It's gonna come down a little bit.

[13:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=832s) Yeah.

[13:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=832s) And then kind of just keep optimizing as these improvements happen across that stack.

[13:56](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=836s) Yeah, yeah. The, the, the bars I was showing earlier are naturally all rising, like, across all those dimensions, and if you understand what the pipeline looks like, then you can just drop stuff in, right?

[14:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=848s) Figure out your pipeline first-

[14:09](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=849s) Yeah

[14:09](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=849s) ... and then you can just iterate on each one of those components.

[14:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=852s) Yeah.

[14:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=852s) That's, that's what's been working really well for us.

[14:14](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=854s) Cool. I like... So o- o- o- one question to drill into that is, I can see pipelines getting more complex with more agentic stuff and so on. Like, how do you rec- like, that one is a moving target too.

[14:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=866s) How do you see handling-

[14:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=867s) It's, it's-

[14:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=867s) How are you handling that?

[14:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=868s) It's not something we've figured out yet. Um, obviously looking for tool, better tools to help orchestrate that.

[14:34](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=874s) Yeah.

[14:34](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=874s) We know we are making improvements on various parts of this thing, but, like, I've had to slow down the team. Like, we're making too many changes at once now across the different things.

[14:42](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=882s) Mm-hmm.

[14:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=883s) So it sounds like we have to slow down a little bit with interfaces and contracts between the layers so that we can iterate on one side and not affect the downstream on another.

[14:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=890s) So I don't know that yet.

[14:52](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=892s) Cool. Great. Anyone else on this question?

[14:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=894s) Just, just to piggyback.

[14:55](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=895s) Yes.

[14:55](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=895s) Um, when you're thinking of the latency improvements you can get if you have a solution that might take you months to implement, you have to evaluate that with the trade-off and say, "By the time we finish this, will a new model have come out?"

[15:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=908s) Yes.

[15:08](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=908s) So I think the time it takes to implement new solutions, there should be, like, a max, like a ceiling. Like, if it takes more than however many months, you're just wasting your time because it's gonna be throwaway.

[15:18](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=918s) I love that. Yeah. If it's a two-year project, then you're-

[15:21](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=921s) Yeah

[15:21](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=921s) ... w- why bother? Like anything-

[15:22](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=922s) Basically-

[15:23](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=923s) ... you do now is gonna be irrelevant.

[15:24](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=924s) Right. I need another solution.

[15:25](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=925s) Yeah.

[15:25](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=925s) Basically.

[15:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=926s) Yeah. Love it.

[15:30](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=930s) I'm gonna dash over here.

[15:31](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=931s) I could also just shout.

[15:32](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=932s) Yeah. It sometimes works. But, um- [laughs]

[15:35](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=935s) Thank you. Yeah, one, I've-- all of that I really resonate with, and one thing that we've seen is companies that have succeeded have avoided premature optimization. So obviously you want to be sort of aware of latency in production and, you know, make something that'll be realistic.

[15:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=950s) But the teams that have sort of started with that and gone, "We're gonna use a small fine-tune model," often incorrectly conclude that a task isn't possible. And also there...

[15:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=959s) people are often able to take data from a larger model and fine-tune it later. And so I always think of, like, moving down the model size stack or fine-tuning as optimizations, and you want to validate the product need and whether it works, whether it's possible, usually with a better model first.

[16:14](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=974s) At least we've seen that be much more successful.

[16:16](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=976s) That's a great... I, I've, I've heard that too, like start with GPT-4o or I guess the, now, now maybe Claude 3.5 and, um, if that can't do it, then the smaller models probably won't.

[16:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=988s) And, and I guess you have an exception for data that's highly proprietary, and that's, that's where fine-tuning and RAG come in.

[16:34](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=994s) Yeah. We do the same thing. Just optimize the best out of the box, and then we kind of slowly bring it down-

[16:40](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1000s) Yeah

[16:40](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1000s) ... to more performant, cheaper models, hopefully trying to get the same accuracy out.

[16:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1006s) Cool.

[16:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1006s) I think building up on, on something that you mentioned as well earlier when, for example, you're using a, a, a large language model to turn unstructured data to structured data to then feed it into, like, traditional machine learning models.

[16:56](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1016s) Mm-hmm.

[16:57](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1017s) One of the key learnings for us was if we're, um, if we're building our eval pipeline from the get-go, which, uh, is really starting to be echoed, you know, trying to figure out what your eval should be, start, get it, get it ready from the start, is to understand do you have the right data to build your

[17:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1032s) eval pipeline? And if not, start from day zero to make sure that you generate that data for your eval pipeline. Because if you leave it to stay too long and then realize, oh, we've, we've built an LLM model and it's in our app, but we cannot evaluate whether, uh, the fine-tuning on the prompts makes it better or

[17:31](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1051s) worse, you're then spending a lot of sort of long lag time on, on figuring out what your eval pipeline should be. So our key learning was really start, as, as you start building your app, also start with your eval pipeline.

[17:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1063s) Make sure that you have the data that's there.

[17:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1065s) Cool. That, that's maybe a corollary of the, okay, you, you need your DevOps kind of framework, but then you need your eval framework, and then your data framework is closely related to that, is making sure you have the right data.

[17:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1079s) Yeah, and the quality's there.

[18:06](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1086s) Um, one of the things we learned is not hiding AI from the end user. So we're- Like explicitly, for example, generating the responses slowly, so as a hint for the user-

[18:19](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1099s) Hmm

[18:19](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1099s) ... that they, they should be more vili- vigilant. We're using specific colors in the UI to make it clear, uh, the icons that they click on. So kind of make sure that they understand that there's a special thing they need to do.

[18:32](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1112s) And then when they wanna check, we give them as much tools for them to validate the answer that was generated to see what, what went wrong, what were the sources, what kind of, uh, went in there.

[18:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1123s) So that's one of the lessons that, uh, not to abstract this away and kind of hope for the best. But, uh-

[18:48](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1128s) This is related to your moderate expectations point.

[18:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1130s) Yeah.

[18:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1130s) If the user can actually see the thing coming out, then that's a trigger that, um, all right, it's not instant, it's coming from some machine, I need to consider-

[18:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1139s) Yeah

[18:59](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1139s) ... things. Cool. I have, um, I wanna ask if anyone,

[19:05](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1145s) um, dealing with regulation or something where explainability is a big issue, and if so, how are you solving that?

[19:17](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1157s) What sort of industry are we talking? Like, it, like a medical something or-

[19:19](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1159s) Um, yeah, I, I could think about medical, like if you're trying to pre- if, if the end game is prescribe something or provide some treatment.

[19:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1166s) Yeah.

[19:26](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1166s) I can think about, um, equipment maintenance is another one.

[19:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1169s) We work in the behavioral health space, um, and a lot of their... There's a lot of regula- regulation, and the one way we've determined to solve it as a kind of a core principle is just always human in the loop right now.

[19:42](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1182s) Yeah.

[19:42](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1182s) Like, we're only in the copilot stage. In fact, I tell people we'll never get to the pilot stage, but that's just to not scare them. But, uh-

[19:49](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1189s) Yeah

[19:49](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1189s) ... we're, we're, we're very focused on copilot right now, uh, and making sure there's a human in the loop. Um, but as part of that we have to give transparency to the inputs that generated the outputs, so they can kinda see things side by side.

[20:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1201s) Um, if we make recommendations or we pick some data out that we think is interesting, uh, for the particular case, uh, that they're working with, um, they can, they can trace it back to the, the root data.

[20:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1213s) And so do you find the inputs are sufficient for getting the humans in the loop to trust the answer enough? 'Cause I, I... So you've got inputs. Another way to do it is here are the references that, you know-

[20:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1227s) Yeah

[20:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1227s) ... but then they're not gonna read the three 50-page papers maybe.

[20:30](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1230s) We, we, we... So if it's, if it's doing a, a lookup, a knowledge lookup, we're showing the source documentation that we find based on their query, uh, so they can see it kinda side by side-

[20:41](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1241s) Yeah

[20:41](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1241s) ... um, with those citations. Um, if it's doing a generation, uh, so let's say we're summarizing data that was collected in a therapy session, um, we show the data side by side with the actual summarization of that.

[20:51](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1251s) Uh, it's kinda like on a paragraph by paragraph basis so that the users have complete control of the, the resultant summary. They can change it, edit it. It's theirs at that point.

[21:02](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1262s) And then are you... Last question. Are you using the, um, judgment of the human in the loop as like [REDACTED:username] to improve the producer?

[21:12](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1272s) Uh, yeah, we run our own eval on any of our output with a separate, um, stage just to make sure that, like, did the input... Does the, does the output match kinda the inputs?

[21:21](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1281s) And that's just a quick one. It's also for a, a safety perspective. But then we use the, whether or not the user edited or, and/or just gave us thumbs up, thumbs down.

[21:29](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1289s) Yeah.

[21:30](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1290s) Uh, and then we do our own clinical evaluation at the end, uh, on our own, with our own team, just making sure, like we're just... Everywhere we need to, we're putting in that eval stage to make sure that we, we know what, uh...

[21:43](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1303s) we're comfortable with what's coming out.

[21:45](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1305s) Cool.

[21:46](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1306s) Uh, I'm just gonna go over here. Just one thing affects, uh, whether we can do one more or multiple more is, is our next speaker here? 'Cause I've not-

[21:54](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1314s) Yes.

[21:55](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1315s) Oh, fantastic. Cool. [laughs] Okay. Right, unfortunately this will have to be the last one then for now, but, uh, I assume this, this discussion-

[22:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1321s) Yeah

[22:01](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1321s) ... can continue at lunchtime as well, 'cause it seems everyone, there's so many shared concerns, um, and stories that people have got in the room, so it would be a great thing for people to discuss at lunch.

[22:11](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1331s) Yeah.

[22:11](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1331s) Absolutely. Here we go.

[22:13](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1333s) I'll keep this short. So a hu- human in the loop is clearly a solution, but that's not always possible. So what we do is give options. So this is, uh, uh, in a, one of our use cases is personal loan recommendation.

[22:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1347s) Hmm.

[22:27](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1347s) It's regulated.

[22:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1348s) Yeah.

[22:28](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1348s) And then so we give, rather than, um, ever say, "This is the best loan for you," we give three options and give explanation for these, and so it becomes less deterministic and less, uh, and, and that, that's how we take care of the regulation.

[22:42](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1362s) That helps.

[22:42](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1362s) Okay, cool. Yeah. So g- give a few options, and then do you do that, do that directly to the end user or to the human in the loop?

[22:47](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1367s) It's end user because we cannot have human in the loop.

[22:49](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1369s) Right. Right.

[22:49](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1369s) But I think the-

[22:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1370s) Yeah

[22:50](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1370s) ... the three options that we se- select are clearly t- show some more logic to it than just coming, that's coming out of the LLM.

[22:58](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1378s) Cool. Thanks. Well, let's definitely keep this one alive throughout the day as you're-

[23:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1383s) Thank you

[23:03](https://www.youtube.com/watch?v=gWLVu9OGvBg&t=1383s) ... mingling with each other and lunch and so on. But, uh, thanks. I appreciate the participation. [upbeat music]
