Your PR Says the Tests Passed. Prove It. - Stephen Chin & Cassandra Chin

Cassandra Chin built Tracetoon after seeing good code turned away in review because its pull request description got the facts wrong. The tool writes a description and can turn the change into a comic. In the story she and Stephen Chin tell, Alex works in a token factory, Pip looks after the evidence, finance watches the bill, and Alex would dearly like to earn an AI engineer hoodie.

Alex submits a long ledger of changes. What could a reviewer still need? The redlines, drawn from a real PR, ask for exact preview links, attribution for the commands, and a qualification about work done earlier. Each request gives the reviewer something to check. They can open the particular preview, trace a command to its source, and distinguish earlier work from what this PR delivers.

Cassandra used Tracetoon to write PRs for Tracetoon itself, so each change was another opportunity to read the output and improve it. In the factory, every trip back through review sends Alex into the logs to reconstruct the evidence again. The growing bill was the part Cassandra could vouch for personally. Building and testing Tracetoon had earned her a token billionaire card.

Ask a coding agent to write a PR at the end of a long session and it can draw on the conversation it just had. Two things can fall out of that account. The agent may have compressed its context along the way, losing particulars, and some of the work may have happened in another session altogether. Tracetoon reads traces across sessions on the developer’s computer to recover what happened on the branch.

The idea came from an education study Cassandra supported. A student could hand in a working program while leaving much of the intellectual work, including the prompts that shaped it, behind in an agent session. A code diff has a similar blind spot. It can show a change to how a tool handles the shell’s current working directory (CWD). The history can explain why the directory recorded before cd runs matters to someone investigating how test data from one repository got associated with another.

Searching more sessions also gives the writer more opportunities to pick up a fact from the wrong PR. Tracetoon pairs the agent context with the code and gives writing and review to independent agents. The comic is optional, though Cassandra started there and worked backward to improving the descriptions. One early comic accompanied a PR about giving each episode new characters.

The original Tracetoon gathered the traces for a branch, researched them and drafted a description. A reviewer could ask for more context, sending the writer back for another pass. Both Cassandra’s experience and the project’s PR Bench comparisons favored these descriptions over the ones produced without Tracetoon. She suspected that the final revision, when the writer tried to incorporate still more context, introduced some of the remaining mistakes.

Consider the passing-run dispute. The original PR said there was no record of a later passing run. Alex’s account nevertheless credited the change with passing tests and completing all the checks. What did the trace establish? It put bun test and the TypeScript type check at the same time. In this case the two checks needed to run sequentially for the test to pass. The trace accounted for both invocations, but their timing left the claimed successful sequence unproven.

Finance suggested using a cheaper writer. When the team tried that, description quality fell. The writer still had to reconstruct what happened and decide how far the evidence supported its wording. A lower price per token left that work intact.

The next version moved evidence gathering ahead of drafting. Relate, an open-source embedded graph database written in Rust, stores relationships that an agent would otherwise have to rediscover in the logs. A query can follow several connections to collect related facts about a change. The in-memory graph uses local compute and storage. Following those connections takes no model tokens.

Tracetoon calls the result a neighborhood brief. For the change at hand, it brings the rationale, the code and the checks into one packet, preserving their sources and how they relate. What reaches the reviewer? The writer supplies the draft, and the brief reaches the reviewer directly as well. Those two inputs let the reviewer check in both directions, tracing a sentence back to its source or finding an important fact that never made it into the draft.

Cassandra had expected the reviewer to need the stronger model. Giving both roles the brief changed where the team could economize.

• Writing. The team kept a stronger writer, such as Sonnet, to turn the evidence into an explanation a human could follow. With a weaker writer, successive rounds of feedback tended to turn the description into a checklist.
• Reviewing. Several cheaper reviewers, such as Haiku models, could compare claims with the brief and return specific findings to the writer for revision. Cross-checking a claim against a known source was a narrower job than independently researching the branch.

The revised description returns to the working-directory problem. A log entry records where the shell was before it executes a leading cd. One case the description calls out is a leading cd into the repository. The shell can start outside the repository and still do its work inside it, so the initial directory alone could lead a reviewer to attribute the command to the wrong project. The PR points to the code that handles this case.

What passed, and on which code? The revised output reports a successful type check and 71 passing tests on the submitted content. An earlier pass would describe an earlier version. A human reviewer now has both a verification result that applies to this submission and a specific place to begin reading its implementation, the leading-cd exception in correlate.ts.

How much did the brief improve descriptions across PRs? The team built PR Bench to compare agents on different pull requests. Language-model judges assessed descriptions against audit traces and code, with a human also checking accuracy. Across the Claude and Codex comparisons, the judges generally preferred Tracetoon with Relate. On a 1–100 quality scale, the two reported changes were about 74 to 76 and 82 to 86, gains of two and four points in the team’s development runs.

Generating those comparisons was expensive. Cassandra initially let one benchmark run finish before starting the next. Here the cases were independent, unlike the two checks within Alex’s PR, so the team could run them in parallel. The workload could saturate Stephen’s high-end gaming computer. It was also a remarkably quick way to spend tokens.

For generating a description, token use moved in the other direction. In the team’s comparisons, Tracetoon with the graph used roughly half as many tokens as the previous workflow. Precomputing the evidence spared the writer and reviewers repeated searches while leaving the stronger writer in place. Cassandra had been skeptical about adding a graph. The reduction in spend persuaded her to keep it.

Stephen’s suggestion for other projects starts with a question about repeated work. Which records does an agent have to find and connect every time someone asks it to write or check a claim? Those relationships are candidates for a local graph. In Tracetoon, retrieving the facts connected to the submitted change gave the writer a compact brief and the reviewers a common set of sources. Alex gets his hoodie. The token billionaire card finds a second use as a bookmark.

For developers working on a pull request, Cassandra’s open-source Tracetoon works with Claude and Codex and includes the optional comic. For developers building agents of their own, Stephen invited them to try Relate as the embedded database for connecting those agents’ evidence. Try Tracetoon.

Cassandra and Stephen closed by inviting families at the San Francisco conference to Saturday’s AI Engineer Kids Day. After a brief disagreement about whether robots were on the program, they confirmed both activities. Kids would use AI to program their own games and to control little soda-can robots.