# Your PR Says the Tests Passed. Prove It. - Stephen Chin & Cassandra Chin

Stephen Chin, Neo4j · Cassandra Chin

[Talk page](https://ai.engineer/talks/gbeuC0ud1p4-your-pr-says-tests-passed-prove-it) · [Video](https://www.youtube.com/watch?v=gbeuC0ud1p4)

## Your PR Says the Tests Passed. Prove It.

A session log can contain both a test command and a type check and still leave a reviewer wondering what passed. Cassandra Chin and Stephen Chin follow a Tracetoon pull request from that gap to a description that identifies which code was checked and where to start reading.

### What sent the pull request back

[0:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=0s)

Cassandra Chin built Tracetoon after seeing good code turned away in review because its pull request description got the facts wrong. The tool writes a description and can turn the change into a comic. In the story she and Stephen Chin tell, Alex works in a token factory, Pip looks after the evidence, finance watches the bill, and Alex would dearly like to earn an AI engineer hoodie.

Alex submits a long ledger of changes. What could a reviewer still need? The redlines, drawn from a real PR, ask for exact preview links, attribution for the commands, and a qualification about work done earlier. Each request gives the reviewer something to check. They can open the particular preview, trace a command to its source, and distinguish earlier work from what this PR delivers.

Cassandra used Tracetoon to write PRs for Tracetoon itself, so each change was another opportunity to read the output and improve it. In the factory, every trip back through review sends Alex into the logs to reconstruct the evidence again. The growing bill was the part Cassandra could vouch for personally. Building and testing Tracetoon had earned her a token billionaire card.

### Recovering the work behind a branch

[4:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=276s)

Ask a coding agent to write a PR at the end of a long session and it can draw on the conversation it just had. Two things can fall out of that account. The agent may have compressed its context along the way, losing particulars, and some of the work may have happened in another session altogether. Tracetoon reads traces across sessions on the developer’s computer to recover what happened on the branch.

The idea came from an education study Cassandra supported. A student could hand in a working program while leaving much of the intellectual work, including the prompts that shaped it, behind in an agent session. A code diff has a similar blind spot. It can show a change to how a tool handles the shell’s current working directory (`CWD`). The history can explain why the directory recorded *before* `cd` runs matters to someone investigating how test data from one repository got associated with another.

Searching more sessions also gives the writer more opportunities to pick up a fact from the wrong PR. Tracetoon pairs the agent context with the code and gives writing and review to independent agents. The comic is optional, though Cassandra started there and worked backward to improving the descriptions. One early comic accompanied a PR about giving each episode new characters.

### Two commands in a trace, one unsupported claim

[8:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=502s)

The original Tracetoon gathered the traces for a branch, researched them and drafted a description. A reviewer could ask for more context, sending the writer back for another pass. Both Cassandra’s experience and the project’s PR Bench comparisons favored these descriptions over the ones produced without Tracetoon. She suspected that the final revision, when the writer tried to incorporate still more context, introduced some of the remaining mistakes.

Consider the passing-run dispute. The original PR said there was no record of a later passing run. Alex’s account nevertheless credited the change with passing tests and completing all the checks. What did the trace establish? It put `bun test` and the TypeScript type check at the same time. In this case the two checks needed to run sequentially for the test to pass. The trace accounted for both invocations, but their timing left the claimed successful sequence unproven.

Finance suggested using a cheaper writer. When the team tried that, description quality fell. The writer still had to reconstruct what happened and decide how far the evidence supported its wording. A lower price per token left that work intact.

### One neighborhood brief for the writer and reviewers

[11:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=675s)

The next version moved evidence gathering ahead of drafting. Relate, an open-source embedded graph database written in Rust, stores relationships that an agent would otherwise have to rediscover in the logs. A query can follow several connections to collect related facts about a change. The in-memory graph uses local compute and storage. Following those connections takes no model tokens.

Tracetoon calls the result a *neighborhood brief*. For the change at hand, it brings the rationale, the code and the checks into one packet, preserving their sources and how they relate. What reaches the reviewer? The writer supplies the draft, and the brief reaches the reviewer directly as well. Those two inputs let the reviewer check in both directions, tracing a sentence back to its source or finding an important fact that never made it into the draft.

Cassandra had expected the reviewer to need the stronger model. Giving both roles the brief changed where the team could economize.

- **Writing.** The team kept a stronger writer, such as **Sonnet**, to turn the evidence into an explanation a human could follow. With a weaker writer, successive rounds of feedback tended to turn the description into a checklist.
- **Reviewing.** Several cheaper reviewers, such as **Haiku** models, could compare claims with the brief and return specific findings to the writer for revision. Cross-checking a claim against a known source was a narrower job than independently researching the branch.

Which inputs reach the reviewer?
Reviewers receive the writer’s account and the source material used to draft it. A finding can therefore point the writer back to the evidence behind a correction.
- Neighborhood brief — Relate brings the rationale, code and checks together with their sources.
- Writer’s draft — A stronger writer explains the change and what was checked.
- Focused reviewers — Reviewers compare claims with the sources and look for omissions.
- Writer’s revision — The writer incorporates the findings into the description for human review.
- Neighborhood brief → Writer’s draft (Facts for drafting)
- Neighborhood brief → Focused reviewers (Same facts for verification)
- Writer’s draft → Focused reviewers (Claims to check)
- Focused reviewers → Writer’s revision (Findings to address)

### What the revised PR lets a human check

[15:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=925s)

The revised description returns to the working-directory problem. A log entry records where the shell was *before* it executes a leading `cd`. One case the description calls out is a leading `cd` into the repository. The shell can start outside the repository and still do its work inside it, so the initial directory alone could lead a reviewer to attribute the command to the wrong project. The PR points to the code that handles this case.

What passed, and on which code? The revised output reports a successful type check and 71 passing tests on the submitted content. An earlier pass would describe an earlier version. A human reviewer now has both a verification result that applies to this submission and a specific place to begin reading its implementation, the leading-`cd` exception in `correlate.ts`.

### What changed on PR Bench

[16:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=988s)

How much did the brief improve descriptions across PRs? The team built PR Bench to compare agents on different pull requests. Language-model judges assessed descriptions against audit traces and code, with a human also checking accuracy. Across the Claude and Codex comparisons, the judges generally preferred Tracetoon with Relate. On a 1–100 quality scale, the two reported changes were about 74 to 76 and 82 to 86, gains of two and four points in the team’s development runs.

Generating those comparisons was expensive. Cassandra initially let one benchmark run finish before starting the next. Here the cases were independent, unlike the two checks within Alex’s PR, so the team could run them in parallel. The workload could saturate Stephen’s high-end gaming computer. It was also a remarkably quick way to spend tokens.

For generating a description, token use moved in the other direction. In the team’s comparisons, Tracetoon with the graph used roughly half as many tokens as the previous workflow. Precomputing the evidence spared the writer and reviewers repeated searches while leaving the stronger writer in place. Cassandra had been skeptical about adding a graph. The reduction in spend persuaded her to keep it.

### Where the pattern travels

[18:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1133s)

Stephen’s suggestion for other projects starts with a question about repeated work. Which records does an agent have to find and connect every time someone asks it to write or check a claim? Those relationships are candidates for a local graph. In Tracetoon, retrieving the facts connected to the submitted change gave the writer a compact brief and the reviewers a common set of sources. Alex gets his hoodie. The token billionaire card finds a second use as a bookmark.

For developers working on a pull request, Cassandra’s open-source Tracetoon works with Claude and Codex and includes the optional comic. For developers building agents of their own, Stephen invited them to try Relate as the embedded database for connecting those agents’ evidence. [Try Tracetoon](https://tracetoon.com/).

Cassandra and Stephen closed by inviting families at the San Francisco conference to Saturday’s AI Engineer Kids Day. After a brief disagreement about whether robots were on the program, they confirmed both activities. Kids would use AI to program their own games and to control little soda-can robots.

## Key takeaways

- A passing-test claim needs a command, a result and the version of the code it checked. When checks must run sequentially, their timing is part of the evidence too.
- A reviewer with the writer’s source brief can check in both directions. Does the evidence support each claim? Did the draft leave out a finding the evidence supports?
- Tracetoon kept a stronger writer and used cheaper models for focused review. With evidence assembled locally before drafting, description scores rose by about two and four points in the team’s two PR Bench comparisons, while the workflow used roughly half as many tokens as before.

## Slides

### Warning: This Pull Request Contains Graphic Content

A paper scroll unrolls across the factory floor, giving the pull request in the title a comic sense of scale. Stephen and Cassandra Chin introduce a graph as a way to make a pull request easier to review.

[0:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=2s) [The subtitle reads “A PR crisis. A graph. A better way to review.” Beside it, a character holds a long paper scroll at a factory gate.](https://assets.ai.engineer/sites/aiecode2025/sha256/90d25281210ee4b1940facb38346471428321ab3ef5c18cd0d6536a835369770.webp)

Warning: This Pull Request Contains Graphic Content

A PR crisis. A graph. A better way to review.

Stephen Chin · Cassandra Chin

Interface text:
neo4j
Stephen Chin

### Make the next PR easy to trust. Easy to review.

The comic shows the team gathered around a sealed envelope. Chin suggests using a comic in a pull request to help reviewers see how the evidence fits together. He points to TraceToon and to Relate, an embedded graph database.

[21:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1275s) [“Connect the evidence. Give the reviewer a useful story.” Two resource links appear below. In the illustration, people, animals and robots gather around an envelope.](https://assets.ai.engineer/sites/aiecode2025/sha256/b730adacf4fd6a3cadda4156fd3a0f08092fcad9173440d190a4a86a4b5c8044.webp)

Make the next PR easy to trust. Easy to review.

Connect the evidence. Give the reviewer a useful story.

tracetoon.com
relatedb.ai

AI Engineer

Interface text:
neo4j
Stephen Chin

## Full transcript

[0:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=0s) Hello and welcome to our ag engineer San

[0:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=3s) Hello and welcome to our ag engineer San

[0:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=3s) Hello and welcome to our ag engineer San Francisco talk. My name is Steven Chin.

[0:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=5s) Francisco talk. My name is Steven Chin.

[0:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=5s) Francisco talk. My name is Steven Chin. I'm joined by Cassandra Chin who's the

[0:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=7s) I'm joined by Cassandra Chin who's the

[0:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=7s) I'm joined by Cassandra Chin who's the creator of the t trace tune framework.

[0:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=12s) creator of the t trace tune framework.

[0:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=12s) creator of the t trace tune framework. Hi, I'm Cassandra Chin. I built trace

[0:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=14s) Hi, I'm Cassandra Chin. I built trace

[0:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=14s) Hi, I'm Cassandra Chin. I built trace tune because I wanted to improve PRs. I

[0:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=18s) tune because I wanted to improve PRs. I

[0:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=18s) tune because I wanted to improve PRs. I see a lot of developers doing great

[0:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=20s) see a lot of developers doing great

[0:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=20s) see a lot of developers doing great things. The speed of innovation is as

[0:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=22s) things. The speed of innovation is as

[0:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=22s) things. The speed of innovation is as fast as it has ever been with AI

[0:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=24s) fast as it has ever been with AI

[0:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=24s) fast as it has ever been with AI technologies. But a lot of poll requests

[0:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=27s) technologies. But a lot of poll requests

[0:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=27s) technologies. But a lot of poll requests are getting rejected. And it's not

[0:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=29s) are getting rejected. And it's not

[0:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=29s) are getting rejected. And it's not because the code is bad. It's because

[0:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=31s) because the code is bad. It's because

[0:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=31s) because the code is bad. It's because the PRs often contain incorrect

[0:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=34s) the PRs often contain incorrect

[0:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=34s) the PRs often contain incorrect information or are written poorly.

[0:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=37s) information or are written poorly.

[0:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=37s) information or are written poorly. So hopefully with Tune, we can improve

[0:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=39s) So hopefully with Tune, we can improve

[0:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=39s) So hopefully with Tune, we can improve that. It also has the option to add a

[0:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=42s) that. It also has the option to add a

[0:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=42s) that. It also has the option to add a fun comic to help explain your pull

[0:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=44s) fun comic to help explain your pull

[0:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=44s) fun comic to help explain your pull requests in a funny way.

[0:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=47s) requests in a funny way.

[0:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=47s) requests in a funny way. Cool. And um

[0:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=48s) Cool. And um

[0:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=48s) Cool. And um what we're going to do today is we're

[0:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=50s) what we're going to do today is we're

[0:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=50s) what we're going to do today is we're going to go through a little bit about

[0:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=52s) going to go through a little bit about

[0:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=52s) going to go through a little bit about kind of the common issues of PRs, how to

[0:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=54s) kind of the common issues of PRs, how to

[0:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=54s) kind of the common issues of PRs, how to improve the quality, how to improve

[0:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=56s) improve the quality, how to improve

[0:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=56s) improve the quality, how to improve token efficiency on this, and hopefully

[0:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=58s) token efficiency on this, and hopefully

[0:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=58s) token efficiency on this, and hopefully help everyone to be a better AI engineer

[1:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=62s) help everyone to be a better AI engineer

[1:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=62s) help everyone to be a better AI engineer for your organization. Like our main

[1:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=65s) for your organization. Like our main

[1:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=65s) for your organization. Like our main character, Alex,

[1:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=67s) character, Alex,

[1:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=67s) character, Alex, who all he wants in life is to get an AI

[1:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=70s) who all he wants in life is to get an AI

[1:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=70s) who all he wants in life is to get an AI engineer hoodie.

[1:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=73s) engineer hoodie.

[1:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=73s) engineer hoodie. That's a really neat hoodie. I have a

[1:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=75s) That's a really neat hoodie. I have a

[1:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=75s) That's a really neat hoodie. I have a lot of AI engineer t-shirts, but I don't

[1:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=78s) lot of AI engineer t-shirts, but I don't

[1:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=78s) lot of AI engineer t-shirts, but I don't have the hoodie yet.

[1:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=81s) have the hoodie yet.

[1:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=81s) have the hoodie yet. Get switched to make this AI engineer

[1:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=82s) Get switched to make this AI engineer

[1:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=82s) Get switched to make this AI engineer hoodie. Everyone

[1:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=84s) hoodie. Everyone

[1:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=84s) hoodie. Everyone everyone needs an

[1:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=85s) everyone needs an

[1:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=85s) everyone needs an ready. And um this is enough motivation

[1:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=88s) ready. And um this is enough motivation

[1:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=88s) ready. And um this is enough motivation that hopefully he's he's going to do an

[1:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=90s) that hopefully he's he's going to do an

[1:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=90s) that hopefully he's he's going to do an amazing talk at the conference about his

[1:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=93s) amazing talk at the conference about his

[1:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=93s) amazing talk at the conference about his use case study on what he's doing in his

[1:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=95s) use case study on what he's doing in his

[1:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=95s) use case study on what he's doing in his company to improve the quality of PRs.

[1:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=99s) company to improve the quality of PRs.

[1:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=99s) company to improve the quality of PRs. Here

[1:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=100s) Here

[1:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=100s) Here is a picture of the token factory where

[1:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=104s) is a picture of the token factory where

[1:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=104s) is a picture of the token factory where Alex works. So, um, Alex works together

[1:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=108s) Alex works. So, um, Alex works together

[1:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=108s) Alex works. So, um, Alex works together with a bunch of agents. He has, um, of

[1:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=112s) with a bunch of agents. He has, um, of

[1:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=112s) with a bunch of agents. He has, um, of course, finance looking over his

[1:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=113s) course, finance looking over his

[1:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=113s) course, finance looking over his shoulder because we we all spend too

[1:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=114s) shoulder because we we all spend too

[1:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=114s) shoulder because we we all spend too much tokens and they want to make sure

[1:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=116s) much tokens and they want to make sure

[1:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=116s) much tokens and they want to make sure we're keeping up with costs. Um, his

[1:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=119s) we're keeping up with costs. Um, his

[1:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=119s) we're keeping up with costs. Um, his buddy there, Pip, he's walking with

[2:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=121s) buddy there, Pip, he's walking with

[2:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=121s) buddy there, Pip, he's walking with carries all the evidence and looks into

[2:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=123s) carries all the evidence and looks into

[2:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=123s) carries all the evidence and looks into the the archives and what's actually

[2:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=125s) the the archives and what's actually

[2:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=125s) the the archives and what's actually happening. And then, u Meera does all

[2:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=127s) happening. And then, u Meera does all

[2:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=127s) happening. And then, u Meera does all the reviews. So, she's she's keeping

[2:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=129s) the reviews. So, she's she's keeping

[2:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=129s) the reviews. So, she's she's keeping track of things.

[2:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=131s) track of things.

[2:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=131s) track of things. It looks like Pip left his station empty

[2:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=139s) part of the factory

[2:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=141s) part of the factory

[2:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=141s) part of the factory until until he sees what Alex produced

[2:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=144s) until until he sees what Alex produced

[2:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=144s) until until he sees what Alex produced for a PR. Oh, no.

[2:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=147s) for a PR. Oh, no.

[2:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=147s) for a PR. Oh, no. That's a very long ledger.

[2:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=150s) That's a very long ledger.

[2:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=150s) That's a very long ledger. A lot to review.

[2:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=153s) A lot to review.

[2:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=153s) A lot to review. A lot of code. So, it's it's hard. And

[2:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=156s) A lot of code. So, it's it's hard. And

[2:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=156s) A lot of code. So, it's it's hard. And Merror's Merror's going through

[2:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=158s) Merror's Merror's going through

[2:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=158s) Merror's Merror's going through everything for the fine tune comb. She

[2:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=159s) everything for the fine tune comb. She

[2:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=159s) everything for the fine tune comb. She actually is red lines. Found some issues

[2:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=163s) actually is red lines. Found some issues

[2:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=163s) actually is red lines. Found some issues with the PR. And this is a a real PR. I

[2:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=168s) with the PR. And this is a a real PR. I

[2:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=168s) with the PR. And this is a a real PR. I think this is actually one of your trace

[2:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=169s) think this is actually one of your trace

[2:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=169s) think this is actually one of your trace tune PRs where

[2:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=171s) tune PRs where

[2:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=171s) tune PRs where it needs to have exact preview links for

[2:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=173s) it needs to have exact preview links for

[2:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=174s) it needs to have exact preview links for the URLs, command attribution,

[2:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=177s) the URLs, command attribution,

[2:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=177s) the URLs, command attribution, make sure the earlier work is um

[2:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=179s) make sure the earlier work is um

[2:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=179s) make sure the earlier work is um qualified as well and explain all of

[3:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=181s) qualified as well and explain all of

[3:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=182s) qualified as well and explain all of this in detail in the PR record to be

[3:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=184s) this in detail in the PR record to be

[3:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=184s) this in detail in the PR record to be accepted by Mera's very critical review.

[3:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=187s) accepted by Mera's very critical review.

[3:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=187s) accepted by Mera's very critical review. This was interesting cuz as I was

[3:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=189s) This was interesting cuz as I was

[3:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=189s) This was interesting cuz as I was building trace tune, I was actually like

[3:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=192s) building trace tune, I was actually like

[3:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=192s) building trace tune, I was actually like as I was building it, I was using trace

[3:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=194s) as I was building it, I was using trace

[3:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=194s) as I was building it, I was using trace tune to test it. So I would have trace

[3:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=197s) tune to test it. So I would have trace

[3:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=197s) tune to test it. So I would have trace tune build the PRs and as I improved the

[3:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=199s) tune build the PRs and as I improved the

[3:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=199s) tune build the PRs and as I improved the PRs it would like get better.

[3:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=203s) PRs it would like get better.

[3:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=203s) PRs it would like get better. You got amazing comics along the way too

[3:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=205s) You got amazing comics along the way too

[3:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=205s) You got amazing comics along the way too just to have fun while you were building

[3:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=208s) just to have fun while you were building

[3:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=208s) just to have fun while you were building stuff.

[3:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=209s) stuff.

[3:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=209s) stuff. Addictive.

[3:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=211s) Addictive.

[3:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=211s) Addictive. [laughter]

[3:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=214s) So um

[3:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=215s) So um

[3:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=216s) So um you can see that the repeated PRs which

[3:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=219s) you can see that the repeated PRs which

[3:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=219s) you can see that the repeated PRs which Alex is doing to try to get good quality

[3:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=221s) Alex is doing to try to get good quality

[3:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=221s) Alex is doing to try to get good quality results are not working out well. is

[3:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=224s) results are not working out well. is

[3:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=224s) results are not working out well. is working out for his token billionaire

[3:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=226s) working out for his token billionaire

[3:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=226s) working out for his token billionaire card. Do do you know anything about the

[3:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=227s) card. Do do you know anything about the

[3:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=227s) card. Do do you know anything about the token billionaire program?

[3:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=229s) token billionaire program?

[3:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=229s) token billionaire program? Yeah, so actually I am a token

[3:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=231s) Yeah, so actually I am a token

[3:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=231s) Yeah, so actually I am a token billionaire. I spent lots and lots of

[3:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=234s) billionaire. I spent lots and lots of

[3:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=234s) billionaire. I spent lots and lots of tokens to build trace tune.

[3:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=239s) That's kind of unfortunate. Was that was

[4:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=241s) That's kind of unfortunate. Was that was

[4:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=241s) That's kind of unfortunate. Was that was that all my money?

[4:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=243s) that all my money?

[4:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=243s) that all my money? Uh don't sweat the details.

[4:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=245s) Uh don't sweat the details.

[4:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=246s) Uh don't sweat the details. Okay.

[4:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=247s) Okay.

[4:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=247s) Okay. So, um I I think there's a lot of good

[4:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=250s) So, um I I think there's a lot of good

[4:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=250s) So, um I I think there's a lot of good uses for for tokens and for spending

[4:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=252s) uses for for tokens and for spending

[4:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=252s) uses for for tokens and for spending more AI work. Um maybe we're doing

[4:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=255s) more AI work. Um maybe we're doing

[4:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=255s) more AI work. Um maybe we're doing repeated PRs is not where you want to

[4:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=256s) repeated PRs is not where you want to

[4:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=256s) repeated PRs is not where you want to spend your token

[4:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=259s) spend your token

[4:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=259s) spend your token and definitely the finance department's

[4:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=260s) and definitely the finance department's

[4:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=260s) and definitely the finance department's not happy with this. So, he wants a

[4:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=262s) not happy with this. So, he wants a

[4:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=262s) not happy with this. So, he wants a results and he wants to make sure that

[4:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=264s) results and he wants to make sure that

[4:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=264s) results and he wants to make sure that the PRs are accurate, are working and he

[4:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=268s) the PRs are accurate, are working and he

[4:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=268s) the PRs are accurate, are working and he has they don't have need to keep getting

[4:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=269s) has they don't have need to keep getting

[4:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=269s) has they don't have need to keep getting repeated again and again by reviewing

[4:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=272s) repeated again and again by reviewing

[4:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=272s) repeated again and again by reviewing the logs and reconstructing the evidence

[4:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=274s) the logs and reconstructing the evidence

[4:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=274s) the logs and reconstructing the evidence repeatedly.

[4:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=275s) repeatedly.

[4:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=276s) repeatedly. And it's interesting cuz normally when

[4:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=279s) And it's interesting cuz normally when

[4:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=279s) And it's interesting cuz normally when you're just like uh coding on your

[4:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=281s) you're just like uh coding on your

[4:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=281s) you're just like uh coding on your agent, you're in your session and you

[4:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=283s) agent, you're in your session and you

[4:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=283s) agent, you're in your session and you ask it to write a PR, it just reads your

[4:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=288s) ask it to write a PR, it just reads your

[4:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=288s) ask it to write a PR, it just reads your session log to try to figure out what it

[4:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=292s) session log to try to figure out what it

[4:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=292s) session log to try to figure out what it should have done using its inflated

[4:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=294s) should have done using its inflated

[4:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=294s) should have done using its inflated context window.

[4:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=296s) context window.

[4:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=296s) context window. It actually often like as it's churning

[4:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=299s) It actually often like as it's churning

[4:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=299s) It actually often like as it's churning it compresses its context window. So it

[5:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=301s) it compresses its context window. So it

[5:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=301s) it compresses its context window. So it gets more and more inaccurate as it goes

[5:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=303s) gets more and more inaccurate as it goes

[5:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=303s) gets more and more inaccurate as it goes along.

[5:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=305s) along.

[5:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=305s) along. So instead, what we're doing here is

[5:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=306s) So instead, what we're doing here is

[5:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=306s) So instead, what we're doing here is with trace tune, we're having it read

[5:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=309s) with trace tune, we're having it read

[5:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=309s) with trace tune, we're having it read through uh different sessions on your

[5:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=312s) through uh different sessions on your

[5:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=312s) through uh different sessions on your computer to try to really figure out

[5:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=314s) computer to try to really figure out

[5:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=314s) computer to try to really figure out what you built and what happened.

[5:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=318s) what you built and what happened.

[5:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=318s) what you built and what happened. Yeah. And I think like like that concept

[5:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=320s) Yeah. And I think like like that concept

[5:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=320s) Yeah. And I think like like that concept of um looking through the audit trails,

[5:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=322s) of um looking through the audit trails,

[5:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=322s) of um looking through the audit trails, looking through like what the agents

[5:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=324s) looking through like what the agents

[5:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=324s) looking through like what the agents have actually done is a lot of what you

[5:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=326s) have actually done is a lot of what you

[5:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=326s) have actually done is a lot of what you were doing with the Showtale project you

[5:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=328s) were doing with the Showtale project you

[5:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=328s) were doing with the Showtale project you were doing for education.

[5:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=331s) were doing for education.

[5:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=331s) were doing for education. Yeah. I actually started

[5:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=333s) Yeah. I actually started

[5:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=333s) Yeah. I actually started this project with this showtail project

[5:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=336s) this project with this showtail project

[5:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=336s) this project with this showtail project because there was this like academic

[5:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=338s) because there was this like academic

[5:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=338s) because there was this like academic research study I was supporting. But

[5:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=341s) research study I was supporting. But

[5:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=341s) research study I was supporting. But basically the issue they hit is students

[5:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=343s) basically the issue they hit is students

[5:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=343s) basically the issue they hit is students were creating programs but there was no

[5:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=346s) were creating programs but there was no

[5:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=346s) were creating programs but there was no way to trace back the actual work and

[5:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=348s) way to trace back the actual work and

[5:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=348s) way to trace back the actual work and effort they did cuz with AI coding the

[5:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=352s) effort they did cuz with AI coding the

[5:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=352s) effort they did cuz with AI coding the what you submit is the final work the

[5:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=354s) what you submit is the final work the

[5:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=354s) what you submit is the final work the program but most of the work is the

[5:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=357s) program but most of the work is the

[5:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=357s) program but most of the work is the prompting and everything that came

[5:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=359s) prompting and everything that came

[5:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=359s) prompting and everything that came before and that doesn't get submitted

[6:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=361s) before and that doesn't get submitted

[6:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=361s) before and that doesn't get submitted with the program and I use the

[6:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=364s) with the program and I use the

[6:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=364s) with the program and I use the technology and theory behind that to be

[6:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=367s) technology and theory behind that to be

[6:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=367s) technology and theory behind that to be like well I can also improve poll

[6:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=369s) like well I can also improve poll

[6:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=369s) like well I can also improve poll requests with all this data collection.

[6:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=374s) requests with all this data collection.

[6:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=374s) requests with all this data collection. Cool. And I think that improvement in

[6:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=377s) Cool. And I think that improvement in

[6:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=377s) Cool. And I think that improvement in pull requests is something that benefits

[6:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=378s) pull requests is something that benefits

[6:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=378s) pull requests is something that benefits everybody because now you can find the

[6:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=381s) everybody because now you can find the

[6:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=381s) everybody because now you can find the evidence like um logging the change

[6:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=383s) evidence like um logging the change

[6:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=384s) evidence like um logging the change directory before the or the CWD before

[6:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=386s) directory before the or the CWD before

[6:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=386s) directory before the or the CWD before the CD runs. explaining why maybe other

[6:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=389s) the CD runs. explaining why maybe other

[6:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=389s) the CD runs. explaining why maybe other test data is leaking into other repos

[6:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=392s) test data is leaking into other repos

[6:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=392s) test data is leaking into other repos and explaining what's happening behind

[6:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=394s) and explaining what's happening behind

[6:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=394s) and explaining what's happening behind the scenes on the PR rather than what

[6:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=397s) the scenes on the PR rather than what

[6:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=397s) the scenes on the PR rather than what most AIPRs read like which is a line by

[6:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=400s) most AIPRs read like which is a line by

[6:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=400s) most AIPRs read like which is a line by line explanation of the diffs since

[6:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=402s) line explanation of the diffs since

[6:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=402s) line explanation of the diffs since that's what they build the PRs off of

[6:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=408s) but incorrect facts can still pollute

[6:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=411s) but incorrect facts can still pollute

[6:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=411s) but incorrect facts can still pollute the information um can cause a problem

[6:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=414s) the information um can cause a problem

[6:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=414s) the information um can cause a problem with conflicting PRs and then still give

[6:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=417s) with conflicting PRs and then still give

[6:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=417s) with conflicting PRs and then still give you back inaccurate results.

[7:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=420s) you back inaccurate results.

[7:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=420s) you back inaccurate results. So maybe we have a solution for this.

[7:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=422s) So maybe we have a solution for this.

[7:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=422s) So maybe we have a solution for this. What What do you What do you think? Do

[7:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=424s) What What do you What do you think? Do

[7:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=424s) What What do you What do you think? Do you have a solution, Cassandra?

[7:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=426s) you have a solution, Cassandra?

[7:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=426s) you have a solution, Cassandra? Yeah, everyone should use Trace June

[7:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=428s) Yeah, everyone should use Trace June

[7:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=428s) Yeah, everyone should use Trace June because of the comics.

[7:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=431s) because of the comics.

[7:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=431s) because of the comics. So because of the comics, not because it

[7:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=433s) So because of the comics, not because it

[7:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=433s) So because of the comics, not because it gives you better PRs, right?

[7:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=435s) gives you better PRs, right?

[7:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=435s) gives you better PRs, right? Well, I started with the comics and I

[7:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=437s) Well, I started with the comics and I

[7:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=437s) Well, I started with the comics and I kind of went backwards and like, oh, I

[7:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=440s) kind of went backwards and like, oh, I

[7:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=440s) kind of went backwards and like, oh, I need to actually make the PRs better.

[7:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=443s) need to actually make the PRs better.

[7:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=443s) need to actually make the PRs better. Well, hopefully better PRs, better

[7:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=446s) Well, hopefully better PRs, better

[7:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=446s) Well, hopefully better PRs, better comics, and maybe better token savings

[7:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=450s) comics, and maybe better token savings

[7:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=450s) comics, and maybe better token savings if by the end of this. Maybe

[7:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=453s) if by the end of this. Maybe

[7:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=453s) if by the end of this. Maybe it should be.

[7:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=455s) it should be.

[7:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=456s) it should be. So, um, trade students an open source

[7:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=458s) So, um, trade students an open source

[7:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=458s) So, um, trade students an open source project. It lets you improve the quality

[7:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=460s) project. It lets you improve the quality

[7:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=460s) project. It lets you improve the quality of your PRs by pulling in agent context

[7:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=463s) of your PRs by pulling in agent context

[7:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=463s) of your PRs by pulling in agent context by pulling in information about the the

[7:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=466s) by pulling in information about the the

[7:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=466s) by pulling in information about the the code and then having an independent

[7:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=468s) code and then having an independent

[7:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=468s) code and then having an independent writer independent reviewer build better

[7:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=470s) writer independent reviewer build better

[7:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=470s) writer independent reviewer build better quality PRs plus optionally giving you

[7:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=474s) quality PRs plus optionally giving you

[7:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=474s) quality PRs plus optionally giving you an amazing comic which what do you think

[7:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=477s) an amazing comic which what do you think

[7:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=477s) an amazing comic which what do you think the best feature is here Cassandra?

[7:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=479s) the best feature is here Cassandra?

[7:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=479s) the best feature is here Cassandra? I think it's obviously the comics.

[8:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=482s) I think it's obviously the comics.

[8:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=482s) I think it's obviously the comics. This is one of the earlier comics from

[8:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=484s) This is one of the earlier comics from

[8:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=484s) This is one of the earlier comics from when I was using traune to build it.

[8:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=490s) That's good. And then you you you made

[8:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=492s) That's good. And then you you you made

[8:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=492s) That's good. And then you you you made sure that there's new characters every

[8:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=493s) sure that there's new characters every

[8:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=493s) sure that there's new characters every episode, not none of this repeating

[8:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=495s) episode, not none of this repeating

[8:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=495s) episode, not none of this repeating characters.

[8:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=496s) characters.

[8:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=496s) characters. Yeah, that's what this PR was about.

[8:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=502s) Okay. And then this is a description of

[8:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=504s) Okay. And then this is a description of

[8:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=504s) Okay. And then this is a description of the original trace tune flow. So it

[8:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=507s) the original trace tune flow. So it

[8:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=507s) the original trace tune flow. So it finds all the traces for a particular

[8:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=508s) finds all the traces for a particular

[8:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=508s) finds all the traces for a particular branch. It researches them, reviews the

[8:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=511s) branch. It researches them, reviews the

[8:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=511s) branch. It researches them, reviews the description, adds more context, and kind

[8:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=513s) description, adds more context, and kind

[8:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=513s) description, adds more context, and kind of goes in a loop until it gets all of

[8:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=516s) of goes in a loop until it gets all of

[8:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=516s) of goes in a loop until it gets all of the evidence it needs to make a a

[8:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=518s) the evidence it needs to make a a

[8:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=518s) the evidence it needs to make a a quality PR. And um subjectively, but

[8:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=522s) quality PR. And um subjectively, but

[8:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=522s) quality PR. And um subjectively, but also through um PRB bench, which is

[8:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=524s) also through um PRB bench, which is

[8:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=524s) also through um PRB bench, which is testing framework, um gives you better

[8:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=527s) testing framework, um gives you better

[8:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=527s) testing framework, um gives you better quality, better research, lower um rate

[8:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=531s) quality, better research, lower um rate

[8:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=531s) quality, better research, lower um rate of inaccuracies in the PRs. I think at

[8:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=533s) of inaccuracies in the PRs. I think at

[8:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=533s) of inaccuracies in the PRs. I think at this point the PRs were already better

[8:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=536s) this point the PRs were already better

[8:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=536s) this point the PRs were already better than if you don't use trace dune, but I

[8:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=538s) than if you don't use trace dune, but I

[8:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=538s) than if you don't use trace dune, but I found that it still makes some mistakes

[9:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=540s) found that it still makes some mistakes

[9:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=540s) found that it still makes some mistakes occasionally.

[9:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=542s) occasionally.

[9:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=542s) occasionally. You can see at the end that it tries to

[9:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=544s) You can see at the end that it tries to

[9:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=544s) You can see at the end that it tries to revise with more context. I think that's

[9:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=546s) revise with more context. I think that's

[9:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=546s) revise with more context. I think that's where the mistakes come in.

[9:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=549s) where the mistakes come in.

[9:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=549s) where the mistakes come in. [snorts]

[9:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=550s) [snorts]

[9:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=550s) [snorts] Okay. Well, Alex Alex thinks it's good

[9:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=552s) Okay. Well, Alex Alex thinks it's good

[9:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=552s) Okay. Well, Alex Alex thinks it's good enough. He's going to try to solve the

[9:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=554s) enough. He's going to try to solve the

[9:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=554s) enough. He's going to try to solve the token crisis the factory with trace tune

[9:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=556s) token crisis the factory with trace tune

[9:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=556s) token crisis the factory with trace tune and see if this will solve all of his

[9:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=557s) and see if this will solve all of his

[9:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=557s) and see if this will solve all of his problems. Polish checked. Alex looks

[9:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=562s) problems. Polish checked. Alex looks

[9:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=562s) problems. Polish checked. Alex looks very very happy

[9:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=564s) very very happy

[9:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=564s) very very happy but Mera doesn't look that happy.

[9:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=566s) but Mera doesn't look that happy.

[9:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=566s) but Mera doesn't look that happy. It looks like we still hit a mistake

[9:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=568s) It looks like we still hit a mistake

[9:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=568s) It looks like we still hit a mistake here like I was saying earlier. So the

[9:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=571s) here like I was saying earlier. So the

[9:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=571s) here like I was saying earlier. So the original PR claim was no later passing

[9:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=574s) original PR claim was no later passing

[9:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=574s) original PR claim was no later passing run is recorded.

[9:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=576s) run is recorded.

[9:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=576s) run is recorded. And the problem here is the in the

[9:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=579s) And the problem here is the in the

[9:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=579s) And the problem here is the in the actual trace it shows it running a bun

[9:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=583s) actual trace it shows it running a bun

[9:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=583s) actual trace it shows it running a bun test at the same time as the bun xtsc

[9:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=588s) test at the same time as the bun xtsc

[9:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=588s) test at the same time as the bun xtsc and you can't run those two at the same

[9:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=590s) and you can't run those two at the same

[9:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=590s) and you can't run those two at the same time. They have to be run sequentially

[9:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=592s) time. They have to be run sequentially

[9:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=592s) time. They have to be run sequentially for the test to pass.

[9:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=595s) for the test to pass.

[9:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=595s) for the test to pass. Yeah, exactly. So he claims test passing

[9:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=598s) Yeah, exactly. So he claims test passing

[9:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=598s) Yeah, exactly. So he claims test passing and all the checks were run and actually

[10:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=601s) and all the checks were run and actually

[10:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=601s) and all the checks were run and actually that's not what the code did. So poor

[10:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=604s) that's not what the code did. So poor

[10:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=604s) that's not what the code did. So poor Alex. Another failed PR, a very angry

[10:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=608s) Alex. Another failed PR, a very angry

[10:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=608s) Alex. Another failed PR, a very angry badger with all the token cost. Um kind

[10:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=612s) badger with all the token cost. Um kind

[10:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=612s) badger with all the token cost. Um kind of kind of makes me feel like Monday

[10:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=613s) of kind of makes me feel like Monday

[10:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=614s) of kind of makes me feel like Monday morning with the Neo Forj finance

[10:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=615s) morning with the Neo Forj finance

[10:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=615s) morning with the Neo Forj finance department looking at my token spend

[10:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=617s) department looking at my token spend

[10:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=617s) department looking at my token spend each week. But they're they're very

[10:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=619s) each week. But they're they're very

[10:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=619s) each week. But they're they're very forgiving, [laughter]

[10:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=621s) forgiving, [laughter]

[10:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=621s) forgiving, [laughter] especially when it's a important project

[10:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=623s) especially when it's a important project

[10:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=624s) especially when it's a important project like like this. So maybe Alex can save

[10:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=628s) like like this. So maybe Alex can save

[10:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=628s) like like this. So maybe Alex can save some cost by using cheaper writers is

[10:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=630s) some cost by using cheaper writers is

[10:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=630s) some cost by using cheaper writers is the number one thing finance asked me to

[10:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=631s) the number one thing finance asked me to

[10:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=631s) the number one thing finance asked me to do. They're like, "Hey, hey, no more

[10:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=633s) do. They're like, "Hey, hey, no more

[10:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=633s) do. They're like, "Hey, hey, no more opus on it. Haiku, do something cheaper,

[10:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=636s) opus on it. Haiku, do something cheaper,

[10:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=636s) opus on it. Haiku, do something cheaper, please. Does this work?"

[10:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=639s) please. Does this work?"

[10:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=640s) please. Does this work?" I feel like the quality is a bit lower

[10:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=642s) I feel like the quality is a bit lower

[10:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=642s) I feel like the quality is a bit lower when you do it like that. Just reducing

[10:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=644s) when you do it like that. Just reducing

[10:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=644s) when you do it like that. Just reducing the cost doesn't quite help.

[10:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=646s) the cost doesn't quite help.

[10:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=646s) the cost doesn't quite help. Yeah. So, we were trying to get better

[10:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=648s) Yeah. So, we were trying to get better

[10:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=648s) Yeah. So, we were trying to get better quality and we we ended up getting worse

[10:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=650s) quality and we we ended up getting worse

[10:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=650s) quality and we we ended up getting worse quality by just using cheaper writers.

[10:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=653s) quality by just using cheaper writers.

[10:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=653s) quality by just using cheaper writers. And

[10:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=655s) And

[10:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=655s) And obviously, Alex is in a bind here,

[10:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=657s) obviously, Alex is in a bind here,

[10:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=657s) obviously, Alex is in a bind here, right? He's he's spent too much on

[10:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=658s) right? He's he's spent too much on

[10:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=658s) right? He's he's spent too much on revisions. He doesn't have a solution

[11:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=661s) revisions. He doesn't have a solution

[11:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=661s) revisions. He doesn't have a solution for reducing token costs. He's not

[11:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=662s) for reducing token costs. He's not

[11:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=662s) for reducing token costs. He's not getting the results he wants out of his

[11:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=663s) getting the results he wants out of his

[11:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=664s) getting the results he wants out of his PRs. But

[11:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=667s) PRs. But

[11:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=667s) PRs. But maybe there's a there's something Pip

[11:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=669s) maybe there's a there's something Pip

[11:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=669s) maybe there's a there's something Pip can help us with. Pip Pip has a friend.

[11:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=672s) can help us with. Pip Pip has a friend.

[11:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=672s) can help us with. Pip Pip has a friend. I wonder who this friend could be.

[11:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=675s) I wonder who this friend could be.

[11:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=675s) I wonder who this friend could be. Oh, well, I I think I think you know

[11:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=678s) Oh, well, I I think I think you know

[11:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=678s) Oh, well, I I think I think you know from my previous talks that that this

[11:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=680s) from my previous talks that that this

[11:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=680s) from my previous talks that that this friend is definitely going to be a

[11:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=682s) friend is definitely going to be a

[11:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=682s) friend is definitely going to be a graph.

[11:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=684s) graph.

[11:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=684s) graph. You're the graph guy. We're doing

[11:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=686s) You're the graph guy. We're doing

[11:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=686s) You're the graph guy. We're doing another graph again.

[11:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=688s) another graph again.

[11:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=688s) another graph again. Yeah. Yeah. There's always a new graph,

[11:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=690s) Yeah. Yeah. There's always a new graph,

[11:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=690s) Yeah. Yeah. There's always a new graph, but this is not the same old graph. This

[11:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=693s) but this is not the same old graph. This

[11:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=693s) but this is not the same old graph. This is not the the Neo Forj database. This

[11:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=696s) is not the the Neo Forj database. This

[11:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=696s) is not the the Neo Forj database. This is our new friend Archie,

[11:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=699s) is our new friend Archie,

[11:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=699s) is our new friend Archie, who is a embedded database written in

[11:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=703s) who is a embedded database written in

[11:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=703s) who is a embedded database written in Rust called relate that actually is

[11:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=706s) Rust called relate that actually is

[11:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=706s) Rust called relate that actually is being launched this week

[11:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=709s) being launched this week

[11:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=709s) being launched this week at the nodes conference and also this

[11:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=711s) at the nodes conference and also this

[11:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=711s) at the nodes conference and also this talk at a engineer in San Francisco.

[11:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=714s) talk at a engineer in San Francisco.

[11:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=714s) talk at a engineer in San Francisco. And it's an open- source um database

[11:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=718s) And it's an open- source um database

[11:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=718s) And it's an open- source um database which allows us to get a lot of the

[11:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=719s) which allows us to get a lot of the

[11:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=719s) which allows us to get a lot of the benefits of more accuracy, multihop

[12:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=723s) benefits of more accuracy, multihop

[12:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=723s) benefits of more accuracy, multihop queries, more explainable traces without

[12:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=727s) queries, more explainable traces without

[12:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=727s) queries, more explainable traces without having the heavyweight approach. So it's

[12:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=729s) having the heavyweight approach. So it's

[12:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=729s) having the heavyweight approach. So it's kind of an agent focused graph database

[12:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=732s) kind of an agent focused graph database

[12:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=732s) kind of an agent focused graph database specifically for these sort of embedded

[12:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=734s) specifically for these sort of embedded

[12:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=734s) specifically for these sort of embedded use cases.

[12:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=738s) Okay, so let's see how Archie can help

[12:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=740s) Okay, so let's see how Archie can help

[12:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=740s) Okay, so let's see how Archie can help us out. So before we were relying a lot

[12:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=743s) us out. So before we were relying a lot

[12:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=743s) us out. So before we were relying a lot on on agents and loops of agents to

[12:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=745s) on on agents and loops of agents to

[12:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=745s) on on agents and loops of agents to resolve the problem. But graph databases

[12:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=748s) resolve the problem. But graph databases

[12:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=748s) resolve the problem. But graph databases can store relationships can find like

[12:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=751s) can store relationships can find like

[12:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=751s) can store relationships can find like like different things which are

[12:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=753s) like different things which are

[12:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=753s) like different things which are clustered build a brief which connects

[12:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=757s) clustered build a brief which connects

[12:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=757s) clustered build a brief which connects all this information together and then

[12:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=759s) all this information together and then

[12:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=759s) all this information together and then give the agents even a cheaper agent

[12:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=761s) give the agents even a cheaper agent

[12:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=761s) give the agents even a cheaper agent better information and more context to

[12:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=763s) better information and more context to

[12:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=763s) better information and more context to give a good drafting decision. And the

[12:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=765s) give a good drafting decision. And the

[12:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=765s) give a good drafting decision. And the good thing about Archie is he has zero

[12:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=767s) good thing about Archie is he has zero

[12:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=767s) good thing about Archie is he has zero LLM costs. It's all local compute cost

[12:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=771s) LLM costs. It's all local compute cost

[12:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=771s) LLM costs. It's all local compute cost and storage and it's a fully immemory

[12:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=773s) and storage and it's a fully immemory

[12:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=773s) and storage and it's a fully immemory database and rest so it's highly

[12:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=774s) database and rest so it's highly

[12:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=774s) database and rest so it's highly optimized. So we get this providence

[12:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=778s) optimized. So we get this providence

[12:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=778s) optimized. So we get this providence about all these submitted changes and

[13:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=780s) about all these submitted changes and

[13:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=780s) about all these submitted changes and this evidence graph basically for free

[13:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=783s) this evidence graph basically for free

[13:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=783s) this evidence graph basically for free compared to all the LLM and token costs

[13:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=785s) compared to all the LLM and token costs

[13:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=785s) compared to all the LLM and token costs which we were doing before. And this

[13:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=787s) which we were doing before. And this

[13:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=788s) which we were doing before. And this means our our buddy Alex doesn't need to

[13:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=790s) means our our buddy Alex doesn't need to

[13:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=790s) means our our buddy Alex doesn't need to research. She doesn't need to go through

[13:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=791s) research. She doesn't need to go through

[13:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=791s) research. She doesn't need to go through and find all the facts again and go

[13:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=793s) and find all the facts again and go

[13:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=793s) and find all the facts again and go through the sources

[13:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=795s) through the sources

[13:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=795s) through the sources because relate is going to do all that

[13:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=798s) because relate is going to do all that

[13:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=798s) because relate is going to do all that hard work for him and build an evidence

[13:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=800s) hard work for him and build an evidence

[13:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=800s) hard work for him and build an evidence packet that he can use for trace tune.

[13:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=803s) packet that he can use for trace tune.

[13:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=803s) packet that he can use for trace tune. So all sounds really great. Maybe we do

[13:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=805s) So all sounds really great. Maybe we do

[13:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=805s) So all sounds really great. Maybe we do need more graphs.

[13:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=808s) need more graphs.

[13:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=808s) need more graphs. That's I'm surprised to hear that from

[13:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=809s) That's I'm surprised to hear that from

[13:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=810s) That's I'm surprised to hear that from you after all the graph talks you've had

[13:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=811s) you after all the graph talks you've had

[13:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=811s) you after all the graph talks you've had to put up with. Okay, but

[13:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=814s) to put up with. Okay, but

[13:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=814s) to put up with. Okay, but we can do this and make it even better

[13:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=817s) we can do this and make it even better

[13:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=817s) we can do this and make it even better with the neighborhood brief. So

[13:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=818s) with the neighborhood brief. So

[13:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=818s) with the neighborhood brief. So basically taking all of the the Y the

[13:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=820s) basically taking all of the the Y the

[13:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=820s) basically taking all of the the Y the code and the checks putting attributions

[13:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=823s) code and the checks putting attributions

[13:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=823s) code and the checks putting attributions and relationships between those and then

[13:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=825s) and relationships between those and then

[13:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=825s) and relationships between those and then returning back a connected set of

[13:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=826s) returning back a connected set of

[13:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=826s) returning back a connected set of information as a packet. And by doing

[13:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=829s) information as a packet. And by doing

[13:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=829s) information as a packet. And by doing this then when the agent is drafting up

[13:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=831s) this then when the agent is drafting up

[13:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=831s) this then when the agent is drafting up the PR it has all the connected facts

[13:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=834s) the PR it has all the connected facts

[13:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=834s) the PR it has all the connected facts with their sources together in a single

[13:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=837s) with their sources together in a single

[13:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=837s) with their sources together in a single source. And that makes the writer draft

[14:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=841s) source. And that makes the writer draft

[14:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=841s) source. And that makes the writer draft more accurate with fewer passes and it

[14:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=844s) more accurate with fewer passes and it

[14:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=844s) more accurate with fewer passes and it also makes the reviewer's job much

[14:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=846s) also makes the reviewer's job much

[14:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=846s) also makes the reviewer's job much easier. So we don't need an expensive

[14:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=848s) easier. So we don't need an expensive

[14:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=848s) easier. So we don't need an expensive reviewer. We can save by using a few

[14:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=851s) reviewer. We can save by using a few

[14:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=851s) reviewer. We can save by using a few haiku reviewers instead of a um sonnet

[14:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=854s) haiku reviewers instead of a um sonnet

[14:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=854s) haiku reviewers instead of a um sonnet reviewer or or a luna in the case of

[14:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=856s) reviewer or or a luna in the case of

[14:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=856s) reviewer or or a luna in the case of codeex to find come up with the findings

[14:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=859s) codeex to find come up with the findings

[14:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=859s) codeex to find come up with the findings check those against the draft redo the

[14:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=861s) check those against the draft redo the

[14:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=861s) check those against the draft redo the writing and then we retain the higher

[14:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=863s) writing and then we retain the higher

[14:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=864s) writing and then we retain the higher quality writer a a sonnet or terror

[14:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=867s) quality writer a a sonnet or terror

[14:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=867s) quality writer a a sonnet or terror class writer to make sure we have a high

[14:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=869s) class writer to make sure we have a high

[14:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=869s) class writer to make sure we have a high quality brief that will pass human

[14:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=870s) quality brief that will pass human

[14:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=870s) quality brief that will pass human approval.

[14:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=873s) approval.

[14:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=873s) approval. When we were building this, it felt very

[14:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=875s) When we were building this, it felt very

[14:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=875s) When we were building this, it felt very counterintuitive to have the writer be

[14:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=877s) counterintuitive to have the writer be

[14:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=877s) counterintuitive to have the writer be high higher quality than the reviewers.

[14:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=880s) high higher quality than the reviewers.

[14:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=880s) high higher quality than the reviewers. It actually it it's common that people

[14:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=883s) It actually it it's common that people

[14:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=883s) It actually it it's common that people always want to have the planner, the

[14:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=884s) always want to have the planner, the

[14:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=884s) always want to have the planner, the reviewer, the checker to be the highest

[14:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=886s) reviewer, the checker to be the highest

[14:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=886s) reviewer, the checker to be the highest quality model. But if you're doing high

[14:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=890s) quality model. But if you're doing high

[14:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=890s) quality model. But if you're doing high quality writing, a lower model, no

[14:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=892s) quality writing, a lower model, no

[14:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=892s) quality writing, a lower model, no matter how many iterations and how much

[14:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=894s) matter how many iterations and how much

[14:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=894s) matter how many iterations and how much feedback you give it, can't do it. It

[14:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=896s) feedback you give it, can't do it. It

[14:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=896s) feedback you give it, can't do it. It just turns in things into a checklist

[14:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=898s) just turns in things into a checklist

[14:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=898s) just turns in things into a checklist and makes it very mechanical. So to give

[15:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=900s) and makes it very mechanical. So to give

[15:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=900s) and makes it very mechanical. So to give writing which feels humanlike, you do

[15:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=903s) writing which feels humanlike, you do

[15:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=903s) writing which feels humanlike, you do need a higher quality model. And it's

[15:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=905s) need a higher quality model. And it's

[15:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=905s) need a higher quality model. And it's it's actually the the reviewing and the

[15:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=906s) it's actually the the reviewing and the

[15:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=906s) it's actually the the reviewing and the cross-checking step which is more

[15:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=908s) cross-checking step which is more

[15:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=908s) cross-checking step which is more mechanical. So it's a little

[15:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=909s) mechanical. So it's a little

[15:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=909s) mechanical. So it's a little counterintuitive, but that works out

[15:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=911s) counterintuitive, but that works out

[15:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=911s) counterintuitive, but that works out quite well. It's also a little

[15:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=914s) quite well. It's also a little

[15:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=914s) quite well. It's also a little counterintuitive that um our badger has

[15:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=917s) counterintuitive that um our badger has

[15:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=917s) counterintuitive that um our badger has a thumbs up coming out of his chest, but

[15:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=919s) a thumbs up coming out of his chest, but

[15:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=919s) a thumbs up coming out of his chest, but let's let's ignore the high quality AI

[15:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=921s) let's let's ignore the high quality AI

[15:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=921s) let's let's ignore the high quality AI artwork.

[15:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=925s) Okay, so we've checked all the claims

[15:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=926s) Okay, so we've checked all the claims

[15:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=926s) Okay, so we've checked all the claims against the evidence. We've looked for

[15:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=928s) against the evidence. We've looked for

[15:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=928s) against the evidence. We've looked for missing findings with our cheaper

[15:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=931s) missing findings with our cheaper

[15:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=931s) missing findings with our cheaper reviewer. Did the focus work and then

[15:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=935s) reviewer. Did the focus work and then

[15:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=935s) reviewer. Did the focus work and then found the things which were connected

[15:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=937s) found the things which were connected

[15:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=937s) found the things which were connected information to give a better draft and

[15:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=939s) information to give a better draft and

[15:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=940s) information to give a better draft and review

[15:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=941s) review

[15:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=941s) review for mirror.

[15:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=944s) for mirror.

[15:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=944s) for mirror. And here's the output. So it has the the

[15:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=946s) And here's the output. So it has the the

[15:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=946s) And here's the output. So it has the the Y. So log CWD and that describes the

[15:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=949s) Y. So log CWD and that describes the

[15:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=949s) Y. So log CWD and that describes the shell before the CD runs. What's the

[15:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=952s) shell before the CD runs. What's the

[15:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=952s) shell before the CD runs. What's the exception?

[15:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=953s) exception?

[15:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=954s) exception? So the CD targets in repo or the command

[15:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=956s) So the CD targets in repo or the command

[15:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=956s) So the CD targets in repo or the command mentions a route. What passed? It has

[15:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=958s) mentions a route. What passed? It has

[15:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=958s) mentions a route. What passed? It has type check plus 71 test checked on

[16:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=961s) type check plus 71 test checked on

[16:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=961s) type check plus 71 test checked on submitted content not before.

[16:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=964s) submitted content not before.

[16:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=964s) submitted content not before. And then it starts in correlate.ts

[16:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=966s) And then it starts in correlate.ts

[16:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=966s) And then it starts in correlate.ts inspecting the leading CD exception.

[16:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=969s) inspecting the leading CD exception.

[16:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=970s) inspecting the leading CD exception. And Meera looks pretty happy with these

[16:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=972s) And Meera looks pretty happy with these

[16:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=972s) And Meera looks pretty happy with these results, right?

[16:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=974s) results, right?

[16:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=974s) results, right? Yeah. This might be the first scene

[16:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=976s) Yeah. This might be the first scene

[16:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=976s) Yeah. This might be the first scene where she's genuinely happy.

[16:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=979s) where she's genuinely happy.

[16:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=979s) where she's genuinely happy. I think she was kind of happy with the

[16:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=981s) I think she was kind of happy with the

[16:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=981s) I think she was kind of happy with the the AI engineer sweatshirt before she

[16:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=983s) the AI engineer sweatshirt before she

[16:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=983s) the AI engineer sweatshirt before she realized that Alex actually was a was a

[16:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=985s) realized that Alex actually was a was a

[16:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=985s) realized that Alex actually was a was a goof. Okay, so here are some of the

[16:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=988s) goof. Okay, so here are some of the

[16:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=988s) goof. Okay, so here are some of the actual data and results from the

[16:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=992s) actual data and results from the

[16:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=992s) actual data and results from the sideby-side analysis using PRBench,

[16:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=994s) sideby-side analysis using PRBench,

[16:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=994s) sideby-side analysis using PRBench, which is a um pull request benchmark,

[16:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=997s) which is a um pull request benchmark,

[16:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=997s) which is a um pull request benchmark, which uses

[16:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=998s) which uses

[16:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=998s) which uses audit traces plus code and um LM judges

[16:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1002s) audit traces plus code and um LM judges

[16:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1002s) audit traces plus code and um LM judges to review the quality of the work. Of

[16:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1005s) to review the quality of the work. Of

[16:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1005s) to review the quality of the work. Of course, reviewed by a human as well for

[16:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1007s) course, reviewed by a human as well for

[16:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1007s) course, reviewed by a human as well for accuracy. And you can see that on both

[16:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1009s) accuracy. And you can see that on both

[16:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1009s) accuracy. And you can see that on both Claude and Codeex, you get a significant

[16:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1012s) Claude and Codeex, you get a significant

[16:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1012s) Claude and Codeex, you get a significant quality improvements on a scale of 1 to

[16:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1014s) quality improvements on a scale of 1 to

[16:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1014s) quality improvements on a scale of 1 to 100. The judges by and large almost

[16:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1017s) 100. The judges by and large almost

[16:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1017s) 100. The judges by and large almost unilaterally liked the trace tune plus

[17:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1021s) unilaterally liked the trace tune plus

[17:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1021s) unilaterally liked the trace tune plus relate quality better. Um on a 1 to 100

[17:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1025s) relate quality better. Um on a 1 to 100

[17:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1025s) relate quality better. Um on a 1 to 100 scale it was like 74 to 76 82 to 86. So

[17:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1028s) scale it was like 74 to 76 82 to 86. So

[17:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1028s) scale it was like 74 to 76 82 to 86. So plus two plus 4 point.

[17:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1030s) plus two plus 4 point.

[17:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1030s) plus two plus 4 point. These tests contributed to my token

[17:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1032s) These tests contributed to my token

[17:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1032s) These tests contributed to my token billionaire card because we were running

[17:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1035s) billionaire card because we were running

[17:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1035s) billionaire card because we were running lots and lots of tests. We created this

[17:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1037s) lots and lots of tests. We created this

[17:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1037s) lots and lots of tests. We created this side project called PRB bench just for

[17:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1039s) side project called PRB bench just for

[17:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1039s) side project called PRB bench just for testing and it we would have different

[17:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1042s) testing and it we would have different

[17:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1042s) testing and it we would have different agents test over different poll requests

[17:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1044s) agents test over different poll requests

[17:24](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1044s) agents test over different poll requests and compare the results. Initially I had

[17:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1048s) and compare the results. Initially I had

[17:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1048s) and compare the results. Initially I had it like running like sequentially. It'll

[17:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1052s) it like running like sequentially. It'll

[17:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1052s) it like running like sequentially. It'll do one test and finish it then do the

[17:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1054s) do one test and finish it then do the

[17:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1054s) do one test and finish it then do the next but then we decided to run them all

[17:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1056s) next but then we decided to run them all

[17:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1056s) next but then we decided to run them all in parallel. It's a very quick way to

[17:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1059s) in parallel. It's a very quick way to

[17:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1059s) in parallel. It's a very quick way to reach high spend.

[17:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1060s) reach high spend.

[17:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1060s) reach high spend. Yeah. So that's kind of funny. I I at at

[17:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1063s) Yeah. So that's kind of funny. I I at at

[17:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1063s) Yeah. So that's kind of funny. I I at at at at the current testing rate with the

[17:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1065s) at at the current testing rate with the

[17:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1065s) at at the current testing rate with the number of use cases and tests and the

[17:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1067s) number of use cases and tests and the

[17:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1067s) number of use cases and tests and the benchmark um we can fully saturate my my

[17:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1071s) benchmark um we can fully saturate my my

[17:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1071s) benchmark um we can fully saturate my my high-end gaming computer with as many

[17:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1074s) high-end gaming computer with as many

[17:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1074s) high-end gaming computer with as many threads as Linux can handle

[17:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1076s) threads as Linux can handle

[17:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1076s) threads as Linux can handle simultaneously

[17:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1077s) simultaneously

[17:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1077s) simultaneously and and of course the token spend is is

[18:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1080s) and and of course the token spend is is

[18:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1080s) and and of course the token spend is is slight slightly higher than what we'd

[18:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1082s) slight slightly higher than what we'd

[18:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1082s) slight slightly higher than what we'd like. But it's all for the good of

[18:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1084s) like. But it's all for the good of

[18:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1084s) like. But it's all for the good of improving the quality of pull requests

[18:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1086s) improving the quality of pull requests

[18:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1086s) improving the quality of pull requests for the world. So you all get the

[18:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1088s) for the world. So you all get the

[18:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1088s) for the world. So you all get the benefit of all these test runs and

[18:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1090s) benefit of all these test runs and

[18:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1090s) benefit of all these test runs and having the highest quality improvements

[18:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1093s) having the highest quality improvements

[18:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1093s) having the highest quality improvements and you save on tokens. So we've spent

[18:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1095s) and you save on tokens. So we've spent

[18:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1095s) and you save on tokens. So we've spent the tokens for you in advance because

[18:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1099s) the tokens for you in advance because

[18:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1099s) the tokens for you in advance because trace tune with the graph solution takes

[18:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1102s) trace tune with the graph solution takes

[18:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1102s) trace tune with the graph solution takes about half as many tokens as it did

[18:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1105s) about half as many tokens as it did

[18:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1105s) about half as many tokens as it did previously and which is an amazing

[18:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1108s) previously and which is an amazing

[18:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1108s) previously and which is an amazing result from the graph because now you're

[18:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1109s) result from the graph because now you're

[18:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1109s) result from the graph because now you're premputing you're getting all this

[18:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1111s) premputing you're getting all this

[18:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1111s) premputing you're getting all this evidence graph construct in advance.

[18:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1114s) evidence graph construct in advance.

[18:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1114s) evidence graph construct in advance. The writing's cheaper, the reviewing's

[18:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1117s) The writing's cheaper, the reviewing's

[18:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1117s) The writing's cheaper, the reviewing's cheaper with a higher quality result.

[18:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1121s) cheaper with a higher quality result.

[18:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1121s) cheaper with a higher quality result. I wasn't sure about the whole graph

[18:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1123s) I wasn't sure about the whole graph

[18:43](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1123s) I wasn't sure about the whole graph thing initially, but after seeing the

[18:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1125s) thing initially, but after seeing the

[18:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1125s) thing initially, but after seeing the cost tank almost in half, like it's not

[18:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1128s) cost tank almost in half, like it's not

[18:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1128s) cost tank almost in half, like it's not even a question, it's making this

[18:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1130s) even a question, it's making this

[18:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1130s) even a question, it's making this program work.

[18:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1133s) program work.

[18:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1133s) program work. And this is something we did for the

[18:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1135s) And this is something we did for the

[18:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1135s) And this is something we did for the Trey Stone project, but you can do this

[18:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1136s) Trey Stone project, but you can do this

[18:56](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1136s) Trey Stone project, but you can do this for pretty much any project where you

[18:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1139s) for pretty much any project where you

[18:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1139s) for pretty much any project where you have data, you have relationships, you

[19:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1141s) have data, you have relationships, you

[19:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1141s) have data, you have relationships, you have evidence, you have things which

[19:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1142s) have evidence, you have things which

[19:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1142s) have evidence, you have things which you're trying to to compare because

[19:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1145s) you're trying to to compare because

[19:06](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1146s) you're trying to to compare because agents are not that good at doing the

[19:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1148s) agents are not that good at doing the

[19:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1148s) agents are not that good at doing the data analysis at doing all of the

[19:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1150s) data analysis at doing all of the

[19:10](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1150s) data analysis at doing all of the evaluation and the curation. So if you

[19:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1153s) evaluation and the curation. So if you

[19:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1153s) evaluation and the curation. So if you give them a help at the front end by

[19:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1155s) give them a help at the front end by

[19:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1155s) give them a help at the front end by creating a a custom graph layer with an

[19:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1158s) creating a a custom graph layer with an

[19:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1158s) creating a a custom graph layer with an in-memory database like relate suddenly

[19:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1161s) in-memory database like relate suddenly

[19:21](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1161s) in-memory database like relate suddenly your costs go down your quality goes up

[19:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1163s) your costs go down your quality goes up

[19:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1163s) your costs go down your quality goes up and you too

[19:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1166s) and you too

[19:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1166s) and you too can be a token billionaire like our

[19:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1168s) can be a token billionaire like our

[19:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1168s) can be a token billionaire like our friend Alex. Well I guess it's a now

[19:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1170s) friend Alex. Well I guess it's a now

[19:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1170s) friend Alex. Well I guess it's a now he's using it as a bookmark.

[19:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1173s) he's using it as a bookmark.

[19:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1173s) he's using it as a bookmark. I think that's a good use for it. It's a

[19:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1174s) I think that's a good use for it. It's a

[19:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1174s) I think that's a good use for it. It's a really high quality bookmark. It's

[19:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1176s) really high quality bookmark. It's

[19:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1176s) really high quality bookmark. It's heavier than a credit card.

[19:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1180s) heavier than a credit card.

[19:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1180s) heavier than a credit card. It is. It is a nice surprise for um

[19:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1182s) It is. It is a nice surprise for um

[19:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1182s) It is. It is a nice surprise for um spending all those tokens on PR Bench.

[19:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1186s) spending all those tokens on PR Bench.

[19:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1186s) spending all those tokens on PR Bench. So, finance is happy. Lower drafting

[19:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1189s) So, finance is happy. Lower drafting

[19:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1189s) So, finance is happy. Lower drafting costs that they can reasonably fund.

[19:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1191s) costs that they can reasonably fund.

[19:52](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1192s) costs that they can reasonably fund. Alex is happy. He has his new bookmark.

[19:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1194s) Alex is happy. He has his new bookmark.

[19:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1194s) Alex is happy. He has his new bookmark. key is now has a great case study

[19:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1197s) key is now has a great case study

[19:57](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1197s) key is now has a great case study with the connections doing neighborhood

[19:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1199s) with the connections doing neighborhood

[20:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1200s) with the connections doing neighborhood retrievable of facts so that it's a

[20:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1202s) retrievable of facts so that it's a

[20:02](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1202s) retrievable of facts so that it's a better brief focusing the review so the

[20:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1205s) better brief focusing the review so the

[20:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1205s) better brief focusing the review so the reviewers get the same compact brief ma

[20:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1207s) reviewers get the same compact brief ma

[20:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1207s) reviewers get the same compact brief ma gets exactly the evidence she needs and

[20:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1209s) gets exactly the evidence she needs and

[20:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1209s) gets exactly the evidence she needs and then here's Alex on stage at um AI

[20:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1213s) then here's Alex on stage at um AI

[20:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1213s) then here's Alex on stage at um AI engineer somewhere in the world better

[20:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1215s) engineer somewhere in the world better

[20:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1215s) engineer somewhere in the world better PR's lower drafting cost with a graph

[20:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1220s) PR's lower drafting cost with a graph

[20:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1220s) PR's lower drafting cost with a graph plus open-source

[20:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1222s) plus open-source

[20:22](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1222s) plus open-source trace to builds, which also gives you

[20:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1225s) trace to builds, which also gives you

[20:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1225s) trace to builds, which also gives you amazing comics.

[20:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1230s) Okay, so big round of applause for our

[20:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1233s) Okay, so big round of applause for our

[20:33](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1233s) Okay, so big round of applause for our friend Alex, who's now earned the AI

[20:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1236s) friend Alex, who's now earned the AI

[20:36](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1236s) friend Alex, who's now earned the AI engineer hoodie.

[20:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1238s) engineer hoodie.

[20:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1238s) engineer hoodie. I think that's definitely a wardrobe

[20:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1240s) I think that's definitely a wardrobe

[20:40](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1240s) I think that's definitely a wardrobe upgrade for him.

[20:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1242s) upgrade for him.

[20:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1242s) upgrade for him. Yeah, I don't think the pocket protector

[20:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1244s) Yeah, I don't think the pocket protector

[20:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1244s) Yeah, I don't think the pocket protector look was that great for him. So, good

[20:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1247s) look was that great for him. So, good

[20:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1247s) look was that great for him. So, good for him. I think we'll see Swixs on

[20:49](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1249s) for him. I think we'll see Swixs on

[20:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1250s) for him. I think we'll see Swixs on stage with one of these hoodies soon. He

[20:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1251s) stage with one of these hoodies soon. He

[20:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1251s) stage with one of these hoodies soon. He he can't he can't not do it now that we

[20:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1254s) he can't he can't not do it now that we

[20:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1254s) he can't he can't not do it now that we shamed him into doing it at his own

[20:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1255s) shamed him into doing it at his own

[20:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1255s) shamed him into doing it at his own conference. The token billion program is

[20:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1258s) conference. The token billion program is

[20:58](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1258s) conference. The token billion program is amazing. A big kudos to the team for

[21:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1260s) amazing. A big kudos to the team for

[21:00](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1260s) amazing. A big kudos to the team for coming up with ways to incentivize and

[21:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1263s) coming up with ways to incentivize and

[21:03](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1263s) coming up with ways to incentivize and reward people who are on the bleeding

[21:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1264s) reward people who are on the bleeding

[21:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1264s) reward people who are on the bleeding edge of AI usage and learning how to

[21:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1267s) edge of AI usage and learning how to

[21:07](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1267s) edge of AI usage and learning how to make it really effective for the rest of

[21:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1269s) make it really effective for the rest of

[21:09](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1269s) make it really effective for the rest of us. And um

[21:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1272s) us. And um

[21:12](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1272s) us. And um now you too can have better PRs. Impress

[21:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1276s) now you too can have better PRs. Impress

[21:16](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1276s) now you too can have better PRs. Impress your co-workers by having

[21:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1279s) your co-workers by having

[21:19](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1279s) your co-workers by having wellressearched

[21:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1280s) wellressearched

[21:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1280s) wellressearched effectively written and lowerc cost PRs

[21:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1283s) effectively written and lowerc cost PRs

[21:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1283s) effectively written and lowerc cost PRs with a comic

[21:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1285s) with a comic

[21:25](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1285s) with a comic with a comic. So for more information go

[21:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1287s) with a comic. So for more information go

[21:27](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1287s) with a comic. So for more information go to trace tune.com

[21:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1289s) to trace tune.com

[21:29](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1289s) to trace tune.com learn about trace tune. Give it a try.

[21:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1291s) learn about trace tune. Give it a try.

[21:31](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1291s) learn about trace tune. Give it a try. It works with cloud. It works with

[21:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1292s) It works with cloud. It works with

[21:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1292s) It works with cloud. It works with codecs. It works with all your favorite

[21:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1294s) codecs. It works with all your favorite

[21:34](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1294s) codecs. It works with all your favorite AI tools. And if you want to do these

[21:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1297s) AI tools. And if you want to do these

[21:37](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1297s) AI tools. And if you want to do these same sort of amazing results on your own

[21:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1299s) same sort of amazing results on your own

[21:39](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1299s) same sort of amazing results on your own project, try out Relate, our new

[21:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1302s) project, try out Relate, our new

[21:42](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1302s) project, try out Relate, our new Rustbased embedded open-source graph

[21:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1304s) Rustbased embedded open-source graph

[21:44](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1304s) Rustbased embedded open-source graph database, which brings context to your

[21:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1307s) database, which brings context to your

[21:47](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1307s) database, which brings context to your agents and helps you build more

[21:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1308s) agents and helps you build more

[21:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1308s) agents and helps you build more efficient applications that just work

[21:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1311s) efficient applications that just work

[21:51](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1311s) efficient applications that just work better, are cheaper, and give you better

[21:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1313s) better, are cheaper, and give you better

[21:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1313s) better, are cheaper, and give you better results. So, thank you all for joining

[21:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1315s) results. So, thank you all for joining

[21:55](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1315s) results. So, thank you all for joining our AI engineer San Francisco talk. And

[21:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1319s) our AI engineer San Francisco talk. And

[21:59](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1319s) our AI engineer San Francisco talk. And if you're watching this talk this week

[22:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1321s) if you're watching this talk this week

[22:01](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1321s) if you're watching this talk this week at San Francisco,

[22:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1324s) at San Francisco,

[22:04](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1324s) at San Francisco, what should you do this weekend with

[22:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1325s) what should you do this weekend with

[22:05](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1325s) what should you do this weekend with your kids on Saturday? Cassandra,

[22:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1328s) your kids on Saturday? Cassandra,

[22:08](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1328s) your kids on Saturday? Cassandra, we are running an AI engineer kids day.

[22:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1331s) we are running an AI engineer kids day.

[22:11](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1331s) we are running an AI engineer kids day. So, anyone who's available, just sign

[22:13](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1333s) So, anyone who's available, just sign

[22:14](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1334s) So, anyone who's available, just sign your kids up. We're teaching kids

[22:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1335s) your kids up. We're teaching kids

[22:15](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1335s) your kids up. We're teaching kids workshops. Kids will be learning how to

[22:17](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1337s) workshops. Kids will be learning how to

[22:18](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1338s) workshops. Kids will be learning how to do AI programming to make their own

[22:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1340s) do AI programming to make their own

[22:20](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1340s) do AI programming to make their own games. And I think we're also doing a

[22:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1343s) games. And I think we're also doing a

[22:23](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1343s) games. And I think we're also doing a No, it's just the gaming one.

[22:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1346s) No, it's just the gaming one.

[22:26](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1346s) No, it's just the gaming one. No, we'll do we'll do the robots, too.

[22:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1348s) No, we'll do we'll do the robots, too.

[22:28](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1348s) No, we'll do we'll do the robots, too. Okay. Yeah, we'll also be doing robots.

[22:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1350s) Okay. Yeah, we'll also be doing robots.

[22:30](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1350s) Okay. Yeah, we'll also be doing robots. So, kids will get to control these

[22:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1352s) So, kids will get to control these

[22:32](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1352s) So, kids will get to control these little soda can robots using AI.

[22:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1355s) little soda can robots using AI.

[22:35](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1355s) little soda can robots using AI. Yeah. So, a lot of fun. Physical AI

[22:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1358s) Yeah. So, a lot of fun. Physical AI

[22:38](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1358s) Yeah. So, a lot of fun. Physical AI gaming AI all in one place. Um, sign up

[22:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1361s) gaming AI all in one place. Um, sign up

[22:41](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1361s) gaming AI all in one place. Um, sign up at AI engineer kids day and um, you

[22:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1365s) at AI engineer kids day and um, you

[22:45](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1365s) at AI engineer kids day and um, you know, bring your kids, bring your

[22:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1366s) know, bring your kids, bring your

[22:46](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1366s) know, bring your kids, bring your nieces, bring your nephews, and let's

[22:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1368s) nieces, bring your nephews, and let's

[22:48](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1368s) nieces, bring your nephews, and let's train the next generation of AI

[22:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1370s) train the next generation of AI

[22:50](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1370s) train the next generation of AI engineers.

[22:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1373s) engineers.

[22:53](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1373s) engineers. All right. Thanks so much and enjoy the

[22:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1374s) All right. Thanks so much and enjoy the

[22:54](https://www.youtube.com/watch?v=gbeuC0ud1p4&t=1374s) All right. Thanks so much and enjoy the rest of the conference.