# Beyond the Prototype: Using AI to Write High-Quality Code

Josh Albrecht, Imbue

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/x_1EumTaXeE-beyond-prototype-using-ai-write-high-quality) · [Video](https://www.youtube.com/watch?v=x_1EumTaXeE)

## Beyond the Prototype: Using AI to Write High-Quality Code

Shipping agent-generated code requires evidence about its behavior. Sculptor explores immediate checks, explicit specifications, and safer testing to make that evidence part of development.

Familiarity with code diffs, automated tests, and coding agents will help; the Python example uses sets and a dataclass.

### Generated code is only the beginning

[0:19](https://www.youtube.com/watch?v=x_1EumTaXeE&t=19s)

You give a coding agent a task. It produces a pile of code, and the prototype looks finished. What would make that code ready to ship into a larger, established codebase? That gap motivates [Sculptor](https://imbue.com/blog/sculptor), the experimental coding-agent environment introduced by Josh Albrecht, CTO of Imbue. The version shown in this recording was a research preview, with features still subject to change.

There are many possible places to improve an agent: larger context windows, lower cost, faster responses, or more reliable output parsing. Albrecht recommends concentrating instead on problems specific to the developer’s domain. His expectation was that upstream model and agent improvements would address many generic limitations over roughly three to twenty-four months. That is a forecast about where to invest engineering effort, much like choosing an existing database instead of building one yourself. The durable work is the part that matters specifically to your product and organization.

![Slide titled “Focus on the Right Problem(s)” with a large circle labeled “What you could build” and a small blue dot indicated by “What you should build.”](https://ai.engineer/api/talk-essay-media/3dcaa2e350942cef964344b82523c3d3f2e860abcbb932d080dd5e36719f44f2)

What you should build is a small part of what you could build.

### What is wrong with this diff?

[2:42](https://www.youtube.com/watch?v=x_1EumTaXeE&t=162s)

An agent reports that it added 59 lines. That tells you how much code changed, but little about whether the change is good. You can inspect every line yourself, or merge and hope. Sculptor explores a third option: have another AI system examine the change for concrete problems, such as race conditions or an exposed API key. **Generating code and building trust in code are separate jobs.** The illustrated Slack extraction change makes the missing evidence visible: a diff summary is not a quality assessment.

![Slide titled “The Problem We Focused On” shows “Implement Slack content extraction and saving to JSON,” +59 and -0, and the question “What is wrong with this diff?”](https://ai.engineer/api/talk-essay-media/0de2cd4fbf3a341e114c44a8e8e08a09eb4819e3dec38e9e4703957b66a54ab1)

A Slack extraction change prompts the question: “What is wrong with this diff?”

The working definition of quality here is defect-oriented: how many problems exist, how long they take to fix, and how many a particular technique catches. That makes the timing of feedback consequential. Sculptor is positioned inside the editing loop, rather than as a pull-request review tool. As soon as an agent generates code—or a developer changes a line—the environment should surface problems while the relevant context is still available. Immediate feedback makes correction easier for both the developer and the agent.

The prevention side of that loop begins with four practices: learn what already exists, plan before implementing, write specifications, and enforce a strict style guide. Each gives the agent a better-defined problem before it starts producing changes.

### Research first, then separate planning from implementation

[4:51](https://www.youtube.com/watch?v=x_1EumTaXeE&t=291s)

Before asking an agent to implement something, ask what technologies already solve the problem and how other developers have approached similar work. Sculptor supports questions and research as part of development, so discovering an existing solution can happen before unnecessary reimplementation begins.

The Scrabble solver demonstration then separates deciding what to build from writing it:

1. Give the agent the solver task, with a system prompt requiring a plan and forbidding code generation.
2. Let it produce the plan.
3. Change the system prompt to permit implementation.
4. Extend the workflow with a checking stage after code generation.

The important control is the ability to change the agent’s operating instructions between phases. Albrecht presents editing the system prompt as stronger steering than simply adding a request to the conversation. Customized agents can make the sequence repeatable: plan, implement, then run checks.

### Make project intent explicit

[6:08](https://www.youtube.com/watch?v=x_1EumTaXeE&t=368s)

Specifications and documentation are expensive when someone must continually keep them synchronized with implementation. But agents need the context those documents provide. They may not have access to the email or Slack conversation where a decision happened; even with access, they may not infer its intended consequences for the code. Writing that intent down makes it available as a development input.

Sculptor tries to reduce the maintenance burden by detecting drift between code, documentation, and docstrings, and helping fix the inconsistencies. It can also flag requirements that conflict with one another. These are different checks: one asks whether implementation matches its description; the other asks whether the description is internally coherent before implementation begins.

A strict style guide supplies another kind of context: constraints on how the team wants software constructed. In the demonstrated suggestion, Sculptor recommends making a class immutable, reflecting Imbue’s preference for functional, immutable code to avoid classes of errors involving shared mutable state. The guidance applies to human teammates as well as agents. An additional style guide tailored specifically to common AI-agent mistakes was still under development.

### Detect new problems in an imperfect repository

[8:03](https://www.youtube.com/watch?v=x_1EumTaXeE&t=483s)

Prevention will not eliminate mistakes. The next layer combines three detection methods: automated analysis, tests, and LLM checks. The named Python tools include [Ruff](https://docs.astral.sh/ruff/faq/), [mypy](https://mypy.readthedocs.io/en/stable/getting_started.html), [Pylint](https://pylint.readthedocs.io/en/stable/), and Pyre. Collectively these are linters and static type checkers, with complementary responsibilities. Small diagnostics can create tedious work for developers, but they also give an agent specific issues to fix. Sculptor connects detection to automatic repair so the developer need not handle every minor violation.

An established repository may already contain many diagnostics. Requiring the agent to make the entire repository clean would mix the requested task with unrelated cleanup. Sculptor instead records which issues existed before the agent started and compares them with the issues afterward. **The immediate requirement is to avoid introducing new errors.** Existing debt does not have to block that improvement.

For example, the comparison can be expressed in Python using normalized diagnostic identities:

```python
from dataclasses import dataclass


@dataclass(frozen=True)
class Diagnostic:
    path: str
    symbol: str
    rule: str


before = {
    Diagnostic("legacy.py", "load_config", "unused-import"),
}
after = {
    Diagnostic("legacy.py", "load_config", "unused-import"),
    Diagnostic("solver.py", "solve", "undefined-name"),
}

introduced = after - before
resolved = before - after

assert introduced == {
    Diagnostic("solver.py", "solve", "undefined-name"),
}
assert resolved == set()
```

The old `legacy.py` issue remains, while the new `solver.py` issue becomes a repair target. A real integration needs diagnostic matching that survives ordinary edits; shifted line numbers alone should not make an old issue appear new.

### Use generated tests to protect behavior

[9:54](https://www.youtube.com/watch?v=x_1EumTaXeE&t=594s)

Test generation changes the economics of preserving behavior. Writing and maintaining tests used to require substantial effort after the implementation was already working. Agents reduce the initial writing cost. For code already assumed correct, Albrecht suggests generating tests, discarding those that fail, and retaining those that pass. That captures accepted behavior; it does not independently establish that the implementation is correct.

The reason to retain those tests becomes stronger when agents can make broad changes quickly. A regression suite expresses behavior you do not want an agent to alter unexpectedly. Albrecht invokes Google’s testing slogan to make the point: behavior that matters deserves a test, because confidence in today’s implementation does not protect it from tomorrow’s edit.

![Slide titled “Why write tests?” shows a boxed quotation, “If you liked it, you shoulda put a test on it,” attributed to Software Engineering at Google: Lessons Learned from Programming Over Time.](https://ai.engineer/api/talk-essay-media/350b289038ec2b85a658ae2d9efc0783f74fd84247d3042fc99a7c732d3a9121)

“If you liked it, you shoulda put a test on it.”

### Isolate side effects before searching for failures

[10:59](https://www.youtube.com/watch?v=x_1EumTaXeE&t=659s)

Good tests start with code that is easy and safe to exercise. Put most logic into functional transformations without side effects, and isolate the operations that touch external systems. Albrecht’s cautionary example is a test with access to a live Gmail account: one mistake could delete real email. Separating decision logic from external actions lets an agent explore the logic without exposing the live account to every experiment.

Different tests then serve different purposes:

| Test type | Purpose | Suggested emphasis |
| --- | --- | --- |
| Happy path | Show expected operation | A small representative set |
| Unhappy path | Find bugs and unusual behavior | Broad exploration of inputs |

For functional code, Albrecht proposes generating hundreds or thousands of candidate inputs, running them, and asking an LLM which outputs look abnormal. An abnormal result is a lead to investigate, not proof by itself. Once confirmed as a bug, the input supplies a concrete, reproducible test case.

### Keep the tests whose behavior matters

[12:18](https://www.youtube.com/watch?v=x_1EumTaXeE&t=738s)

Cheap generation also changes which unit tests deserve permanent maintenance. A generated suite may encode incidental behavior that the team does not care about preserving. Albrecht suggests considering whether to discard those tests, regenerate them later, or consolidate them into a more maintainable suite. Tests that insist on obsolete behavior can confuse an agent asked to change that behavior. The distinction is between durable regression protection and disposable exploration: preserve the contracts you care about, and deliberately curate the rest.

### Test what users can do

[13:04](https://www.youtube.com/watch?v=x_1EumTaXeE&t=784s)

Integration tests move the focus from individual functions to observable user behavior. A shopping-cart test plan can state that clicking to add an item makes it appear in the cart. The complementary plan says that removing the item makes it disappear. Writing these plans first gives the agent a meaningful behavioral target to translate into tests. The developer can reason about the product interaction instead of specifying every implementation detail of the test.

Passing tests are only one part of the evidence. When Claude Code writes a change, you also need to ask whether enough of that change is tested. In Albrecht’s hypothetical merge decision, 100% test coverage, passing tests, and a review finding the tests reasonable provide more confidence than a line-count summary alone. This is not a measured Sculptor result or a correctness guarantee: coverage records execution, while useful assertions and missing edge cases still need scrutiny.

Finally, make tests easy to run in sandboxes and without secrets wherever possible. Isolation reduces the chance that testing will cause accidental external effects, and helps avoid dependencies that make tests flaky. It also makes the repeated execution needed for agent-driven repairs more practical.

### Check intent, architecture, and missing requirements

[14:48](https://www.youtube.com/watch?v=x_1EumTaXeE&t=888s)

The third detection method is to ask an LLM questions that go beyond conventional diagnostics:

- **Change review:** Are there issues in the current uncommitted change or elsewhere in the branch?
- **Intent:** Does the requested task make sense?
- **Design constraints:** Does the implementation violate the style guide or architecture documents?
- **Specification completeness:** Are necessary details missing, requirements unimplemented, or behavior insufficiently tested?

These checks compare code with the project’s intended meaning, not just the language’s rules. Sculptor’s extension goal is to let teams add their own checks so codebase-specific practices are examined continually.

### Make the failure clear, then try repairs safely

[15:29](https://www.youtube.com/watch?v=x_1EumTaXeE&t=929s)

Once a problem has been identified precisely, repair becomes easier. Albrecht spends comparatively little time on this stage because, in his experience, coding agents can often fix an issue once they understand what went wrong. A specific defect gives them a concrete target instead of an open-ended instruction to improve the code.

Repeated attempts are one repair strategy; Albrecht even suggests trying a hundred times with a different agent. He provides no measured success rate for that suggestion. The enabling mechanism is **sandboxing**: safely isolated agents can attempt repairs in parallel, subject to cost constraints, and a successful candidate can supply the solution. That makes clear failure detection valuable twice—first for identifying the problem, then for judging whether an attempted fix resolves it.

### Extend the loop beyond code generation

[16:24](https://www.youtube.com/watch?v=x_1EumTaXeE&t=984s)

The same approach can extend past implementation and deployment. Debugging, logging, tracing, and profiling offer further places for specialized tools to help. Automated QA agents can navigate a website and check whether a user can complete an intended task. Other tools can generate code from visual designs, while better contextual search can serve both developers and their agents.

Better base models are part of this picture, alongside integrations between specialized developer tools. Albrecht closes by inviting builders of those adjacent capabilities to integrate with Sculptor, anticipating that more of them would become accessible over the following year or two. The proposed development environment connects those capabilities across the workflow: discovering what to build, checking what changed, repairing failures, and examining how the software behaves after deployment.

![Slide lists post-deployment debugging, logging and tracing; automated QA; generating code from visual designs; integration with emerging dev tools; contextual search over everything; and improved base models.](https://ai.engineer/api/talk-essay-media/f6c03aee6c0ded3a28eb449c7a21049f57f4dfa345e06da6ba4ccf7e18b446ad)

“This is Just the Beginning”: post-deployment tools, automated QA, visual designs, integrations, search, and improved models.

## Resources

- [Sculptor: Catch and fix issues as you code](https://imbue.com/blog/sculptor) — The original research-preview announcement explains sandboxed agents, parallel repairs, custom checks and reviewable code changes.
- [Imbue's companion page for the talk](https://imbue.com/blog/how-to-build-robust-software-systems-with-ai-josh-at-the-ai-engineer-world-s-fair-2025) — Imbue's brief introduction to Josh Albrecht's World's Fair presentation about building robust software with Sculptor.
- [Sculptor's later redesign](https://imbue.com/blog/sculptor-announce) — A later announcement explains Sculptor's shift toward parallel coding agents, container isolation, saved sessions and interactive review.
- [Ruff FAQ](https://docs.astral.sh/ruff/faq/) — Explains Ruff's linting capabilities, installation and relationship to Pylint and Python type checkers.
- [Getting started with mypy](https://mypy.readthedocs.io/en/stable/getting_started.html) — Introduces Python static type checking, annotations, installation and stricter checking options.
- [Pylint documentation](https://pylint.readthedocs.io/en/stable/) — Documentation and diagnostic reference for checking Python code with Pylint.
- [Code Coverage Best Practices](https://testing.googleblog.com/2020/08/code-coverage-best-practices.html) — Google's testing guidance explains how coverage exposes testing gaps and why a high percentage does not establish correctness.

## Slides

### Beyond the Prototype: Using AI to Write High-Quality Code

Josh Albrecht introduces Imbue’s work on software agents and its product, Sculptor.

[0:17](https://www.youtube.com/watch?v=x_1EumTaXeE&t=17s) [Title slide naming Josh Albrecht, co-founder and CTO of Imbue, with a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/bc210465411b143791c9421285845ab264ae12813e9468f1dfe772d62a92ae17.webp)

Beyond the Prototype:
Using AI to Write
High-Quality Code

Josh Albrecht
Co-founder and CTO, Imbue
imbue.com

Connect with us!

1

### Focus on the Right Problem(s)

Coding agents offer many possible improvements, from performance on large context windows to cost, speed, and output parsing. Albrecht argues these are not the things to build.

[1:35](https://www.youtube.com/watch?v=x_1EumTaXeE&t=95s) [A large circle labeled “What you could build.”](https://assets.ai.engineer/sites/aiecode2025/sha256/f5dba61f1f717fca85333ba1b549f6dbf2bb46945acf064f72066085cc046351.webp)

Focus on the Right Problem(s)

What you could build

Interface text:
vellum
Microsoft
aws
[illegible] ai

### Focus on the Right Problem(s)

A small blue circle marks what you should build within the much larger set of possibilities. The speaker favors problems specific to your use case or domain over general coding-agent improvements.

[1:53](https://www.youtube.com/watch?v=x_1EumTaXeE&t=113s) [A large circle labeled “What you could build.”](https://assets.ai.engineer/sites/aiecode2025/sha256/114a8ebdf02e9f78bacfc46c81a12f1cdbdc14766a57915b9accc72a4ca690aa.webp)

Focus on the Right Problem(s)

What you could build

vellum

[1:58](https://www.youtube.com/watch?v=x_1EumTaXeE&t=118s) [A small blue circle labeled what you should build sits inside a large circle labeled what you could build.](https://assets.ai.engineer/sites/aiecode2025/sha256/f72c35c651d5c14747f5d9eeb9f79139b8224ec6ff526c9fe5a034f7b84e984b.webp)

Focus on the Right Problem(s)

What you could build
What you should build

Microsoft
aws
[illegible] ai

### The Problem We Focused On

Imbue aims to have another AI review a coding agent’s 59-line diff for problems such as race conditions or exposed API keys, giving users an alternative to checking every line or merging on hope.

[2:34](https://www.youtube.com/watch?v=x_1EumTaXeE&t=154s) [A code change for extracting Slack content and saving JSON is shown with 59 additions and the question, “What is wrong with this diff?”](https://assets.ai.engineer/sites/aiecode2025/sha256/8f1911528ca5fb1035d43a2b7feca2f24dbdbfd05c1a8ad0197972dc86eb39cf.webp)

The Problem We Focused On

Implement Slack content extraction and saving to JSON
May 18 · joshalbrecht/thoughtfull
+59 -0

What is wrong with this diff?

### What is High-Quality Software?

The speaker uses defect counts as a working measure of software quality, alongside how long fixes take and how many defects a technique catches.

[3:36](https://www.youtube.com/watch?v=x_1EumTaXeE&t=216s) [The slide states: Higher quality equals fewer defects.](https://assets.ai.engineer/sites/aiecode2025/sha256/94ed20255a443099b431b3f6652a7454b28cc04ab78fcd9c85544fb4b99d101f.webp)

What is High-Quality Software?

Higher quality = fewer defects

Interface text:
daily

### An Ounce of Prevention is Worth a Pound of Cure

The chart estimates defect costs rising from 1X at requirements to 30x–100x during deployment and maintenance. The speaker advocates immediate feedback when code changes.

[3:58](https://www.youtube.com/watch?v=x_1EumTaXeE&t=238s) [Bar chart showing defect costs increasing from requirements through deployment and maintenance.](https://assets.ai.engineer/sites/aiecode2025/sha256/f5bbf9f25478e76afc7b1de04581371992ec29ef6b1032e43dc8c6bb60e09ac7.webp)

An Ounce of Prevention is Worth a Pound of Cure

Cost of Defects

Requirements: 1X
Design/Architecture: 3X
Coding: 7X
Testing: 15X
Deployment/Maintenance: 30x ... 100x

less ← more
Development cycle

From https://www.functionize.com/blog/the-cost-of-finding-bugs-later-in-the-sdlc

Interface text:
daily

### Preventing Problems in AI-Generated Code

The speaker introduces four ways to prevent problems in AI-generated code, from researching existing solutions to encoding best practices in a strict style guide, before showing how they appear in Sculptor.

[4:29](https://www.youtube.com/watch?v=x_1EumTaXeE&t=269s) [Four practices: learn, plan, write specifications, and use a strict style guide.](https://assets.ai.engineer/sites/aiecode2025/sha256/64246917ae3019d0d54548c990d3bf21397f83671d9ac66d808fb88cee2ee153.webp)

Preventing Problems in AI-Generated Code

1. Learn - Research existing solutions and best practices
2. Plan - Create detailed plans before coding
3. Write specs - Document requirements and expectations
4. Have a strict style guide - Encode best practices and patterns

Interface text:
daily

### Learn What's Out There

Sculptor researches frontend libraries in response to a question, returning options such as React, Vue 3, and Svelte. The aim is to find existing solutions before reproducing work.

[4:48](https://www.youtube.com/watch?v=x_1EumTaXeE&t=288s) [Research interface showing a question about frontend libraries and a summary of options.](https://assets.ai.engineer/sites/aiecode2025/sha256/31cb164122f213e3417ab586fdf945768b0eece58231811d6f59c675b805cc9f.webp)

1. Learn What's Out There

what are the best libraries to use for frontend in 2025?

WebSearch: best frontend libraries frameworks 2025 React Vue Angular Svelte
WebSearch: "frontend frameworks 2025" React Vue Svelte Next.js state of js survey

Based on current trends and adoption in 2025, the top frontend libraries/frameworks are:

Top Tier:
• React - Still dominant with ecosystem maturity, Next.js integration, and React 19 features
• Vue 3 - Strong developer experience, composition API, and growing enterprise adoption
• Svelte/SvelteKit - Excellent performance, smaller bundle sizes, increasing popularity

Meta-frameworks:

Interface text:
Research Frontend Libraries for 2025
sculptor/research-frontend-libs-2025
Type a message...

### Start by Planning

The demo begins with a Scrabble solver request; the speaker describes using the system prompt to make the agent plan before writing code.

[5:20](https://www.youtube.com/watch?v=x_1EumTaXeE&t=320s) [A new-branch interface with a request to implement a Scrabble solver.](https://assets.ai.engineer/sites/aiecode2025/sha256/f7850fa13f9c9d752bcb34b00e05e4e1ae81529031cef0bc6afe3cf6642c0d0e.webp)

2. Start by Planning

Interface text:
Start a new branch
implement scrabble solver
ehsannoursalehi/scrabble
ehsan/working
Start Branch
Search branches...
Active  All
Develop Scrabble Word Optimization Engine
Updated 4 minutes ago – sculptor/build-scrabble-word-engine
Ready  +11  -0
Document Codebase Architecture And Functionality
Updated 5 hours ago – sculptor/add-codebase-documentation
Ready  +11  -0
Identify Performance Optimization Opportunities
Updated 6 hours ago – sculptor/explore-performance-improvements
Ready  +11  -0
Research Frontend Libraries for 2025
Updated 10 hours ago – sculptor/research-frontend-libs-2025
Ready  +11  -0
MP4 June 5 higher.mp4

### Start by Planning

A system prompt asks for 2–3 strategies and clarification before coding. The Scrabble agent returns board-validation questions; the user drafts a choice of strategy 2 while the diff shows no changes.

[5:25](https://www.youtube.com/watch?v=x_1EumTaXeE&t=325s) [A system-prompt dialog asks the agent to propose approaches and clarify requirements before coding.](https://assets.ai.engineer/sites/aiecode2025/sha256/026357e03278d5aabee240536d191b9b6a4c44aea6fea2b463190827552b7d1a.webp)

2. Start by Planning

Default System Prompt

Don't write any code until I sign off on the implementation / design strategy. For every request that I make, please give me 2-3 alternative strategies / architectures / designs, etc. and give me the tradeoffs between all of them. Then give me your recommendation. You should be extremely proactive about asking questions to resolve any ambiguity in the task. I would rather us go back and forth for a couple turns than you to make a wrong assumption.

Interface text:
Start a new branch
implement scrabble solver
MP4 June 5 higher.mp4
Cancel
Save
Updated 5 hours ago – sculptor/add-codebase-documentation
Identify Performance Optimization Opportunities
Updated 6 hours ago – sculptor/explore-performance-improvements
Ready +11 -0
Research Frontend Libraries for 2025
Updated 10 hours ago – sculptor/research-frontend-libs-2025
Ready +11 -0
daily

[5:42](https://www.youtube.com/watch?v=x_1EumTaXeE&t=342s) [A coding agent lists questions about a Scrabble board validator before implementation.](https://assets.ai.engineer/sites/aiecode2025/sha256/3a88e74ee5716f15d43f6a4b28969eb2800d1341435210fcca2869045239d874.webp)

2. Start by Planning

Questions to clarify requirements:
1. Board format: Should I assume all boards are exactly 15×15 with '.' representing empty spaces?
2. Center position: Is the center always position (7,7) in 0-indexed coordinates?
3. Word definition: Do words need to be at least 2 letters long? Should single letters count as words?
4. Connectivity: Should I use 4-way connectivity (only horizontal/vertical) or 8-way (including diagonals)?
5. Output format: Should the program output to stdout, return boolean values, or write results to a file?
6. Input handling: Should the program read all boards from boards.txt and validate each one, or validate one board at a time?

Would you like me to proceed with the functional approach, or would you prefer one of the other strategies? Also, please clarify any of the questions above that are important for the implementation.

great, lets go with strategy 2

Interface text:
Develop Scrabble Word Finding Algorithm
sculptor/create-scrabble-solver-engine
Review
Diff
Todo
Log
No changes since HEAD
MP4 June 5 higher.mp4
Graphite

### Start by Planning: System Prompt

After planning, the demo changes the agent’s system prompt to allow implementation while asking it to clarify ambiguity and write self-documenting code.

[5:52](https://www.youtube.com/watch?v=x_1EumTaXeE&t=352s) [A system prompt asks the coding agent to seek clarification and write self-documenting code.](https://assets.ai.engineer/sites/aiecode2025/sha256/aab7b591cce9fe7ae1b7571c344450882e1bd01642e98b3ec35b0b17b7402383.webp)

2. Start by Planning

Questions to clarify requirements:
1. Board format: Should I assume all boards are exactly 15×15 with '.' representing empty spaces?
2. Center position: Is the center always position (7,7) in 0-indexed coordinates?
3. Word definition: Do words need to be at least 2 letters long? Should single letters count as words?
4. Connectivity: Should I use 4-way connectivity (only horizontal/vertical) or 8-way (including

SYSTEM PROMPT
You should implement the task as described. But, if during the course of implementation, you discover any ambiguity (i.e. current unknown unknowns), then you should pause and ask me for clarification before you continue, since I would rather us spend more time clarifying the task than for you to make a wrong assumption.
You should write self documenting code without the need for unnecessary comments. Only include comments when they provide important context that explains *why* the code is doing something a certain way.

great, lets go with strategy 2

Interface text:
Develop Scrabble Word Finding Algorithm
sculptor/create-scrabble-solver-engine
Review
Diff
Todo
Log
No changes since HEAD
MP4 June 5 higher.mp4
Graphite

### Start by Planning: Implementation Checklist

After the user selects the functional approach, the agent creates an eight-step checklist for the Scrabble board validator. The checklist shows 0 of 8 tasks done as implementation begins.

[6:01](https://www.youtube.com/watch?v=x_1EumTaXeE&t=361s) [The coding agent begins the functional approach and displays an eight-item checklist.](https://assets.ai.engineer/sites/aiecode2025/sha256/baa17e17a2b565b07b4a50d11c8616cffbcdc0da0df59bb6c46b48bbe9ad71d3.webp)

2. Start by Planning

6. Input handling: Should the program read all boards from boards.txt and validate each one, or validate one board at a time?

Would you like me to proceed with the functional approach, or would you prefer one of the other strategies? Also, please clarify any of the questions above that are important for the implementation.

great, lets go with strategy 2

Perfect! I'll implement the functional approach. Let me create a todo list to track the implementation steps and then start coding.

Edited todo list   update 8 todos

Now let me start implementing the Scrabble board validator using the functional approach:

Thinking...

1. Load dictionary into a set for O(1) word lookups — high
2. Implement parse_board() to convert string to 2D grid — high
3. Implement check_center() to verify center tile has a letter — high
4. Implement find_words() to extract all horizontal and vertical words — high
5. Implement check_connectivity() using flood fill algorithm — high
6. Implement validate_board() main function that combines all checks — high
7. Add main execution logic to process all boards from boards.txt — med
8. Test with the provided example boards — med

0 of 8 Done

Interface text:
Develop Scrabble Word Finding Algorithm
sculptor/create-scrabble-solver-engine
Review
Diff
Todo
Log
MP4 June 5 higher.mp4
Type a message...
Graphite

### Write Specs and Docs

Specs and docs give coding agents project context. Sculptor helps maintain them by detecting when code and documentation drift apart and highlighting conflicting specifications.

[6:06](https://www.youtube.com/watch?v=x_1EumTaXeE&t=366s) [Slide explains that documentation gives coding agents context and can be checked for gaps and inconsistencies.](https://assets.ai.engineer/sites/aiecode2025/sha256/a6168fa9ad14243e4a26e26ce4981415f2161a152201b95288e174b50a7f05b6.webp)

3. Write Specs and Docs

• Critical for coding agents to have context
• AI makes maintaining docs easier
• Automated detection of:
  ◦ Out-of-sync documentation
  ◦ Missing specifications
  Inconsistent requirements

Interface text:
MP4 June 5 higher.mp4

### Have a Strict Style Guide

Sculptor suggests making ScrabbleBoard immutable to prevent race conditions, reflecting the team’s style guide for both coding agents and human teammates.

[7:11](https://www.youtube.com/watch?v=x_1EumTaXeE&t=431s) [Slide shows a coding agent’s suggestion to make a class immutable to prevent race conditions.](https://assets.ai.engineer/sites/aiecode2025/sha256/b806e915a83259a2ac8a034c75b1fd12ba7927173dd29206a31e65757c490f27.webp)

4. Have a Strict Style Guide (and enforce it)

[illegible]

🎯 Key Implementation Details
• Dictionary Loading: Case-insensitive with proper error handling
• Word Extraction: Handles both horizontal and vertical words efficiently
• Connectivity Algorithm: BFS traversal to check if all letters form one connected component
• Detailed Feedback: Shows exactly which validation rules failed and what words were found
• Robust Parsing: Handles various input formats and validates board dimensions

The implementation is production-ready and follows best practices for maintainability, testability, and extensibility! 🚀

Finished running pytest and imube_verify checks in 27 seconds. Generated 1 suggestion.

Suggestion: Make ScrabbleBoard immutable to prevent race conditions

Interface text:
Implement Core Scrabble Game Features
sculptor/add-scrabble-features
Copy
Type a message...
Graphite

### Detecting Problems in AI Generated Code

Linters perform static analysis, tests verify functionality, and an LLM reviews code. Together, these checks help catch AI-generated mistakes before production.

[8:02](https://www.youtube.com/watch?v=x_1EumTaXeE&t=482s) [Slide lists linters, tests, and LLM code review as ways to detect problems.](https://assets.ai.engineer/sites/aiecode2025/sha256/c534c2e169a98e4449490358886ac59c55c1943dfd38204c0782ef1cc22e9c4a.webp)

Detecting Problems in AI Generated Code

1. Run linters - Automated static analysis
2. Write (and run) tests - Verify functionality
3. Ask an LLM - Intelligent code review

### Run Linters

Linters detect classes of errors that AI can fix. Sculptor compares issues before and after a run to prevent new lint errors, even in a codebase with existing ones.

[8:30](https://www.youtube.com/watch?v=x_1EumTaXeE&t=510s) [Slide lists Python and JavaScript linting tools and says AI systems can easily fix the issues they identify.](https://assets.ai.engineer/sites/aiecode2025/sha256/e250ff65032aa2fb31a13425515a3d9ba4ec2406e29662c18585a0101ace10c6.webp)

1. Run Linters

Automated tools like these:

• ruff, mypy, pylint, pyre, black (Python)
• eslint, prettier (JavaScript)
• etc

...create issues that AI systems easily fix!

### Why write tests?

Tests protect behavior you care about when coding agents make changes. The speaker argues that agents also make writing tests easier.

[10:32](https://www.youtube.com/watch?v=x_1EumTaXeE&t=632s) [Slide asks why to write tests and quotes Software Engineering at Google on testing behavior you value.](https://assets.ai.engineer/sites/aiecode2025/sha256/d2942767e7218c5c4be0ed2cba7df39fa65b0b1d65735fa9cd38c7c8f9162f99.webp)

Why write tests?

"If you liked it, you shoulda put a test on it"

- Software Engineering at Google: Lessons Learned from Programming Over Time

### Ask an LLM

LLMs can check changes before a commit, question unclear goals, and review repository rules, specifications, and tests. Sculptor is also working on extensible checks for a codebase’s own best practices.

[14:56](https://www.youtube.com/watch?v=x_1EumTaXeE&t=896s) [Slide lists LLM checks for changes, goals, repository rules, specifications, and test coverage.](https://assets.ai.engineer/sites/aiecode2025/sha256/23a4016fae91cc5786644ca2ccfa998e40a6ab07708c3f2fcc463ce8f4bb4df6.webp)

3. Ask an LLM

LLMs can check for lots of things, including:
• issues with changes (before committing)
• if your goal is reasonable & unambiguous
• for issues in the current branch
• for violations of rules in your repo
• if details are missing from specs
• if specs are implemented correctly
• if implementations are well-tested

### Fixing Problems in AI-Generated Code

A clearly stated problem makes fixes easier. Coding agents can try multiple solutions in parallel sandboxes, then use a successful attempt—but only if you can tell when the problem is solved.

[15:53](https://www.youtube.com/watch?v=x_1EumTaXeE&t=953s) [A quotation and points about repeated attempts, parallel sandbox execution, and knowing when a problem is solved.](https://assets.ai.engineer/sites/aiecode2025/sha256/652b06171e40654ee116dad2d35f00d860537e66d1a9127a53024aed453f6445.webp)

Fixing Problems in AI-Generated Code

"A problem well stated is half-solved"

• Simple "try multiple times" strategies work well
• Parallel execution in sandboxes enables this
• Knowing when the problem is solved is crucial

### This is Just the Beginning

The speaker expects AI developer tools to expand beyond code generation: for example, automated QA could click through a website to check whether a user can complete a task. She also anticipates closer integration between tools.

[16:23](https://www.youtube.com/watch?v=x_1EumTaXeE&t=983s) [Potential future developer tools, including automated QA, visual design to code, contextual search, and improved models.](https://assets.ai.engineer/sites/aiecode2025/sha256/7ac33e3116c5b97a4424a44bce42fcbeef79a1ef6df9d80acc5736be916bac8a.webp)

This is Just the Beginning

• Post-deployment: debugging, logging, tracing, etc
• Automated QA
• Generating code from visual designs
• Integration with emerging dev tools
• Contextual search over everything
• Improved base models

Interface text:
Graphite

### Learn More About Imbue

Imbue invites viewers to try Sculptor, connect with the team, and explore open roles. A QR code offers another way to connect.

[17:35](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1055s) [A QR code and invitations to try Sculptor, connect with the team, and explore hiring.](https://assets.ai.engineer/sites/aiecode2025/sha256/669399438e7a02eb44ebbeb60b476850457d684ca8bfd33610f05dbc7ef3b8d5.webp)

Learn More About Imbue

Connect with us!

• Try Sculptor:
https://imbue.com
• Connect with our team if you have questions – follow us on X and sign up for our newsletter
• We’re hiring!

Interface text:
Graphite

## Full transcript

[0:00](https://www.youtube.com/watch?v=x_1EumTaXeE&t=0s) [upbeat music] It's great to be here.

[0:19](https://www.youtube.com/watch?v=x_1EumTaXeE&t=19s) So I'm Josh Albrecht. I'm the CTO of Imbue, and our focus is on making more robust, useful AI agents, in particular focusing on software agents right now. And the main product that we're working on today is called Sculptor.

[0:34](https://www.youtube.com/watch?v=x_1EumTaXeE&t=34s) So the purpose of Sculptor is to kind of help us with something that we've all experienced. You know, we've all tried these vibe coding tools and you, you know, tell it to go off and do something.

[0:45](https://www.youtube.com/watch?v=x_1EumTaXeE&t=45s) It goes off and creates a bunch of code for you, uh, and then, you know, voila, you're done, right? Well, not quite. Like at least today, there's a big gap between kind of the stuff that comes back, uh, and what you wanna ship to production, especially as you get away from the prototyping into a larger, more established

[1:00](https://www.youtube.com/watch?v=x_1EumTaXeE&t=60s) code bases. So today, I'm gonna go over some of the technical decisions that went into the design of Sculptor, uh, our experimental coding agent environment, uh, and kind of go through some of the context and motivations for the various ideas that we've explored and the features that we've implemented.

[1:18](https://www.youtube.com/watch?v=x_1EumTaXeE&t=78s) It's still a research preview, so these features may change before we actually release it. Uh, but I hope that, you know, whether you're an individual using these tools or you're someone who's developing the tools yourself, you'll find these, uh, kind of learnings from our experiments to be useful for yourselves.

[1:35](https://www.youtube.com/watch?v=x_1EumTaXeE&t=95s) So today, if you're thinking about how you can make coding agents better, then there's a million different things that you could build. You could build something that helps improve the performance on really large context windows.

[1:49](https://www.youtube.com/watch?v=x_1EumTaXeE&t=109s) You can make something to make it cheaper or faster. You could make something that does a better job of parsing the outputs. But I don't think that we really should be building any of these things.

[1:59](https://www.youtube.com/watch?v=x_1EumTaXeE&t=119s) I think that what we really want to be building is things that are much more specific to the use case or to like the problem domain or the thing that you are, like, really specialized in.

[2:10](https://www.youtube.com/watch?v=x_1EumTaXeE&t=130s) Most of the things that I just mentioned are gonna get solved over the next, call it, three to 12 to 24 months as models get better, coding agents get better, et cetera.

[2:20](https://www.youtube.com/watch?v=x_1EumTaXeE&t=140s) And so I think, you know, just like you wouldn't want to make your own database, I don't think we wanna be spending a lot of time working on the problems that are going to get solved.

[2:29](https://www.youtube.com/watch?v=x_1EumTaXeE&t=149s) Uh, instead, we wanna focus on the particular part of the problem that really matters for, for us, for our business. And so at Imbue, the problem that we're focusing on is basically this, like what is wrong with this diff?

[2:42](https://www.youtube.com/watch?v=x_1EumTaXeE&t=162s) You get a coding agent output and it tells you like, "Okay, I've added 59 new lines." Are those good? Like right now, you have an awkward choice between either looking at each of the lines yourself or just hitting merge and kind of hoping for the best.

[2:55](https://www.youtube.com/watch?v=x_1EumTaXeE&t=175s) Uh, and neither of those are a really great place to be, so we try to give you a third option. Uh, the goal is to help build user trust by allowing another AI system to come and take a look at this and understand like, "Hey, are there any race conditions?

[3:13](https://www.youtube.com/watch?v=x_1EumTaXeE&t=193s) Did you leave your API key in there?" Et cetera. So we wanna think about how do we help leverage AI tools, not just to generate the code, but to help us build trust in that code.

[3:24](https://www.youtube.com/watch?v=x_1EumTaXeE&t=204s) And kind of the way that we think about it is about, like identifying problems with the code. Because if there's no problems, then that's probably high quality code, and that's kind of the definition of high quality code.

[3:37](https://www.youtube.com/watch?v=x_1EumTaXeE&t=217s) If you think about it from like an academic perspective, the way that people normally measure software quality is by looking at the number of defects, and they look at like how long does it take to fix a particular defect, or how many defects are caught by this particular technique.

[3:53](https://www.youtube.com/watch?v=x_1EumTaXeE&t=233s) So this is sort of the definition that at least we're working o- from when we're thinking about making high quality software. And then if we think about, you know, the software development process, what you wanna be doing is getting to a place where you have identified these problems as early as possible.

[4:09](https://www.youtube.com/watch?v=x_1EumTaXeE&t=249s) So Sculptor does not work as like a pull request review tool, 'cause that's much, much later in the process. Rather, we want something that's synchronous and immediate and giving you immediate feedback.

[4:20](https://www.youtube.com/watch?v=x_1EumTaXeE&t=260s) As soon as you've generated that code, as soon as you've changed that line, you wanna know, like, is there something wrong with it? That's easier both for you to fix and also for the agent to fix.

[4:30](https://www.youtube.com/watch?v=x_1EumTaXeE&t=270s) So what are some ways that you can prevent problems in AI-generated code? We're gonna go through five different ways. Uh, the first is learning, planning-- Oh, sorry, only four different ways. [chuckles]

[4:42](https://www.youtube.com/watch?v=x_1EumTaXeE&t=282s) Learning, planning, writing specs, and having a really strict style guide, and we'll see how those manifest in Sculptor.

[4:51](https://www.youtube.com/watch?v=x_1EumTaXeE&t=291s) So the first thing you wanna do when you're using coding agents if you're trying to prevent problems is learn what's out there. We try to make this as easy as possible in Sculptor by letting you ask questions, have it do research, get answers about what are the technologies, et cetera, that exist, what are the ways that other

[5:07](https://www.youtube.com/watch?v=x_1EumTaXeE&t=307s) people have solved similar problems, so that you don't end up reproducing a bunch of work for what's already out there.

[5:15](https://www.youtube.com/watch?v=x_1EumTaXeE&t=315s) Next, we wanna think about how we can encourage people to start by planning. Here's a little example workflow where you can, you know, kick off the agent to go do something simple like, you know, implement this Scrabble solver and change the system prompt here to force the AI agent to first make a plan without writing any code

[5:33](https://www.youtube.com/watch?v=x_1EumTaXeE&t=333s) at all. Then you can wait a little while, it'll generate the plan, uh, and then you can go and change the system prompt again to say like, "Okay, now we can actually create some code."

[5:44](https://www.youtube.com/watch?v=x_1EumTaXeE&t=344s) So we make it really easy to kind of change these types of meta parameters of the coding agent itself. Of course, you can just tell the agent to do that, but by changing its system prompt, you sort of force it in a much stronger way to, uh, change its behavior.

[5:57](https://www.youtube.com/watch?v=x_1EumTaXeE&t=357s) And you can build up larger workflows by making sort of customized agents for always plan first, then always do the code, then always run the checks, et cetera.

[6:08](https://www.youtube.com/watch?v=x_1EumTaXeE&t=368s) Third, you wanna think about writing specs and docs as a kind of first class part of the workflow. One of the main reasons why at least I don't normally write lots of specs and docs in the past has been that it's kind of annoying to keep them all up to date, to spend all this time kind of

[6:24](https://www.youtube.com/watch?v=x_1EumTaXeE&t=384s) typing everything out if I already know what the code is supposed to be. But this is really important to do if you want the coding agents to actually have context on the project that you're trying to do because they don't have access to your email, your Slack, et cetera, necessarily, and even if they did, they might not

[6:40](https://www.youtube.com/watch?v=x_1EumTaXeE&t=400s) know exactly how to turn that into code.

[6:43](https://www.youtube.com/watch?v=x_1EumTaXeE&t=403s) So in Sculptor, uh, one of the ways that we try to make this easier is by helping detect if the code and the docs have become outdated. So it reduces the barrier to writing and maintaining documentation and docstrings because now you have a way of more automatically fixing the inconsistencies.

[7:03](https://www.youtube.com/watch?v=x_1EumTaXeE&t=423s) It can also highlight inconsistencies or parts of the specifications that conflict with each other, uh, making it easier to make sure that your system makes sense from the very beginning.

[7:12](https://www.youtube.com/watch?v=x_1EumTaXeE&t=432s) And finally, you wanna have a really strict style guide and try to enforce it. This is important even if you're just doing regular coding without AI agents, just with other human software engineers.

[7:22](https://www.youtube.com/watch?v=x_1EumTaXeE&t=442s) But one of the things that is special in Sculptor is that we make suggestions, which you can see towards the bottom here, uh, that help keep the AI system on a reasonable path.

[7:33](https://www.youtube.com/watch?v=x_1EumTaXeE&t=453s) So here it's highlighting that you could, you know, make this particular class immutable to prevent race conditions, which is something that comes from our style guide where we try to encourage both the coding agents and our teammates to write things in a more functional immutable style to prevent certain classes of errors.

[7:51](https://www.youtube.com/watch?v=x_1EumTaXeE&t=471s) We're also working on developing a style guide that's sort of custom-tailored to AI agents to make it even easier for them to avoid some of the most egregious mistakes that they normally make.

[8:03](https://www.youtube.com/watch?v=x_1EumTaXeE&t=483s) But no matter how many, uh, things you do to prevent the AI system from making mistakes in the first place, it's going to make some mistakes. And there are many things that we can do to prevent or to detect those problems and prevent them from getting into production.

[8:19](https://www.youtube.com/watch?v=x_1EumTaXeE&t=499s) So we'll go through three here. Uh, first, running linters, second, writing and running tests, third, asking an LLM, uh, and we'll dig into each and see how that manifests in Sculptor.

[8:31](https://www.youtube.com/watch?v=x_1EumTaXeE&t=511s) So for the first one, for running linters, there are many automated tools that are out there like Ruff or Mypy, Pylint, Pyre, et cetera, that you can use to automatically detect certain classes of errors.

[8:46](https://www.youtube.com/watch?v=x_1EumTaXeE&t=526s) In normal development, this is sort of obnoxious because you have to go fix all these like really small errors that don't necessarily cause problems. It's a lot of like churn and extra work.

[8:56](https://www.youtube.com/watch?v=x_1EumTaXeE&t=536s) But one of the great things about AI systems is that they're really good at fixing these. So one of the things that we've built into Sculptor is the ability for the system to very easily detect these types of issues and automatically fix them for you without you having to get involved.

[9:12](https://www.youtube.com/watch?v=x_1EumTaXeE&t=552s) Another thing that we've done is make it easy to use these tools in practice. A lot of tools end up like these, you know. How many people here, maybe a show of hands, how many people have a linter set up at all?

[9:28](https://www.youtube.com/watch?v=x_1EumTaXeE&t=568s) Okay. How many people have zero linting errors in their code base? Two. Great. I will hire you. Okay, cool. Uh, but [chuckles] you know, it's, uh, it's not, it's not easy, but one of the things that we've done in Sculptor is make it so that the AI system understands what issues were there before it started, and then what

[9:45](https://www.youtube.com/watch?v=x_1EumTaXeE&t=585s) issues were there after it ran. So at least you can prevent the AI system from creating more errors without you, even if it doesn't work in a perfectly clean code base.

[9:54](https://www.youtube.com/watch?v=x_1EumTaXeE&t=594s) Okay. Third, testing. So why should you write tests at all? I think I was pretty lazy as a developer for a long time and did not want to write tests because it took a, you know, a lot of effort.

[10:06](https://www.youtube.com/watch?v=x_1EumTaXeE&t=606s) You have to maintain them. I already wrote the code. It works. Okay. But one of the major objections to writing tests has kind of disappeared now that we have AI systems.

[10:16](https://www.youtube.com/watch?v=x_1EumTaXeE&t=616s) The ability to generate tests is now so easy that you might as well write tests, especially if you have correct code. You can tell the agent, "Hey, just write a bunch of tests, throw out the ones that don't pass, and just keep the rest."

[10:27](https://www.youtube.com/watch?v=x_1EumTaXeE&t=627s) So there's no real reason to not write tests at all. Uh, and B, at Go-- as they say at Google, "If you liked it, you should have put a test on it."

[10:36](https://www.youtube.com/watch?v=x_1EumTaXeE&t=636s) This becomes much more important with coding agents. Uh, the reason is that you don't want your coding agent to go change the behavior of your system in a way that you don't understand and don't expect and don't want to see happen.

[10:48](https://www.youtube.com/watch?v=x_1EumTaXeE&t=648s) So at Google, this matters a lot for their infrastructure 'cause they don't want their site to crash when someone changes something. But if you really care about the behavior of your system, you wanna make sure that it's fully tested.

[10:59](https://www.youtube.com/watch?v=x_1EumTaXeE&t=659s) So how do we actually write good tests? I'll go through a bunch of different, uh, components to this. So first, one of the things that you can do is write code in a functional style.

[11:10](https://www.youtube.com/watch?v=x_1EumTaXeE&t=670s) By this, I mean code that has no side effects. This makes it much, much easier to run LLMs and understand if the code is actually successful. You really don't want to be running a test that has access to, say, your live Gmail environment, where if you make a single mistake, you can delete all of your email.

[11:28](https://www.youtube.com/watch?v=x_1EumTaXeE&t=688s) You really want to isolate those types of side effects and be able to focus most of the code, uh, on the kind of functional transformations that matter for your program.

[11:38](https://www.youtube.com/watch?v=x_1EumTaXeE&t=698s) Second, you can try and write two different types of unit tests. Happy path unit tests are those that are ones that show you that your code is working. It's happy.

[11:47](https://www.youtube.com/watch?v=x_1EumTaXeE&t=707s) Hooray, it worked. Uh, you don't need that many of those. You just need a small number to show that things are working as you hope. The unhappy unit tests are the ones that help us find bugs, and here, LLMs can be really, really helpful.

[12:00](https://www.youtube.com/watch?v=x_1EumTaXeE&t=720s) So especially if you've written your code in a functional style, you can have the LLM generate hundreds or even thousands of potential inputs, see what happens to those inputs, and then ask the LLM, "Does that look weird?"

[12:12](https://www.youtube.com/watch?v=x_1EumTaXeE&t=732s) And often when it says yes, that will be a bug. And so now you have a perfect test case replicating a bug.

[12:18](https://www.youtube.com/watch?v=x_1EumTaXeE&t=738s) Third, after you've written your unit tests, it's maybe a good idea to throw them away in some cases. This is a little bit counterintuitive. In the past, it spent-- we took all this effort and spent all this time trying to write good unit tests, and so we feel some aversion to throwing them away.

[12:35](https://www.youtube.com/watch?v=x_1EumTaXeE&t=755s) But now that it's so easy to run LLMs and generate the test suite again from scratch, there's a reas- a good reason to not keep around too many unit tests of behavior that you don't care about too much.

[12:47](https://www.youtube.com/watch?v=x_1EumTaXeE&t=767s) You might also want to just refactor the ones that you generated into something that's slightly more maintainable. But when you do keep them around, it does kind of confuse the LLM when you come back and change this behavior, so it's something that's at least worth thinking about, whether you want to keep the tests that were originally generated,

[13:00](https://www.youtube.com/watch?v=x_1EumTaXeE&t=780s) clean them up, how many of them should you keep, et cetera.

[13:04](https://www.youtube.com/watch?v=x_1EumTaXeE&t=784s) Fourth, you should probably focus on integration tests, uh, as opposed to testing only the kind of code level functional, uh, behavior of your program. Integration tests are those that show you that your program actually works, like from the user's perspective.

[13:18](https://www.youtube.com/watch?v=x_1EumTaXeE&t=798s) Like when the user clicks on this thing, does this other thing happen?

[13:23](https://www.youtube.com/watch?v=x_1EumTaXeE&t=803s) AI systems can be extremely good at writing these, especially if you create nice test plans where you can write, okay, when the user clicks on the button to add the item to the shopping cart, then the item is in the shopping cart.

[13:34](https://www.youtube.com/watch?v=x_1EumTaXeE&t=814s) If you write that out and then you write the test, then you can write another test plan, like if the user clicks to remove the button, the thing from the shopping cart, then it is gone.

[13:43](https://www.youtube.com/watch?v=x_1EumTaXeE&t=823s) The systems can almost always get this right, and so it allows you to work at the level of meaning for your testing, which can be much more efficient. Uh, fifth, you wanna think about test coverage as a core part of your testing suite.

[13:57](https://www.youtube.com/watch?v=x_1EumTaXeE&t=837s) So if you're having Claude Code write things for you, then you don't care just about the tests working on their own, but you also care are there enough tests in the first place.

[14:08](https://www.youtube.com/watch?v=x_1EumTaXeE&t=848s) If you think back to the original screenshot where we get back our PR of, you know, how many lines have changed, if I tell you how many lines have changed, it's not that helpful.

[14:16](https://www.youtube.com/watch?v=x_1EumTaXeE&t=856s) If I tell you so many lines have changed and also there's a hundred percent test coverage, and also all the tests pass, and also a thing looked at the tests and thought they were reasonable, now you can probably click on that merge button without quite as much fear.

[14:30](https://www.youtube.com/watch?v=x_1EumTaXeE&t=870s) Uh, and sixth, uh, we try to make it easy to run tests in sandboxes and without secrets as much as possible. This, uh, makes it a lot easier to actually fix things and makes it a lot easier to make sure that you're not accidentally causing problems or making flaky tests.

[14:48](https://www.youtube.com/watch?v=x_1EumTaXeE&t=888s) The third thing that we can do to detect errors is ask an LLM. There are many different things that we can check for, including if there are issues before you commit with your current change, if the thing that you're trying to do even makes sense, if there are issues in the current branch you're working on, if there

[15:03](https://www.youtube.com/watch?v=x_1EumTaXeE&t=903s) are violations of rules in your style guide or in your architecture documents, if there are details that are missing from the specs, if the specs aren't implemented, if they're not well tested, or whatever other custom things that you want to check for.

[15:16](https://www.youtube.com/watch?v=x_1EumTaXeE&t=916s) One of the things that we're trying to enable in Sculptor is for people to extend the checks that we have so that they can add their own types of best practices into the code base, uh, and make sure that they are continually checked.

[15:29](https://www.youtube.com/watch?v=x_1EumTaXeE&t=929s) After you've found issues, then you have to fix them. Very little of this talk is about fixing the issues because it ends up being a lot easier for the systems to fix issues than you would expect.

[15:42](https://www.youtube.com/watch?v=x_1EumTaXeE&t=942s) I think this quote captures it relatively well, and that a problem well stated is half solved. What this means is that if you really understand what went wrong, then it's much easier to solve the problem.

[15:54](https://www.youtube.com/watch?v=x_1EumTaXeE&t=954s) This is especially true for coding agents because the really simple strategies work really well. So even just try multiple times, try a hundred times with a different agent, it actually ends up, like working out quite well.

[16:08](https://www.youtube.com/watch?v=x_1EumTaXeE&t=968s) And one of the things that enables this is having really good sandboxing. If you have agents that can run safely, then you can run an almost unlimited number, subject to cost constraints, uh, in parallel, and then if any one of them succeeds, then you can use that solution.

[16:24](https://www.youtube.com/watch?v=x_1EumTaXeE&t=984s) And this is really just the beginning. There are going to be so many more tools that are released over the next year or two, and many of the people in this room are working on those tools.

[16:34](https://www.youtube.com/watch?v=x_1EumTaXeE&t=994s) There will be things that are not just for writing code like we've been talking about, but for after deployment, for debugging, logging, tracing, profiling, et cetera. There are tools for doing automated quality assurance where you can have an AI system click around on your website and check if it can actually do the thing that you want the

[16:53](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1013s) user to do. There are tools for generating code from visual designs. There are tons of dev, dev tools coming out every week. You will have much better contextual search systems that are useful for both you and for the agent.

[17:05](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1025s) Uh, and of course, we'll get better AI-based models as well. If anyone is working on these other sorts of tools that, that are kind of adjacent to developer experience and helping you fix this like much smaller piece of the process, we would love to work together and find out a way to integrate that into Sculptor so that

[17:24](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1044s) people can take advantage of that. I think what we'll see over the next year or two is that most of these things will be accessible, uh, and it'll make the development experience just a lot easier once all these things are working together.

[17:37](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1057s) So that's pretty much all that I have for today. If you're interested, feel free to take a look at the QR code. Go to our website at imbue.com and sign up to try out Sculptor.

[17:46](https://www.youtube.com/watch?v=x_1EumTaXeE&t=1066s) And of course, if you're interested in working on things like this, we're always hiring, we're always happy to chat, so feel free to reach out. Thank you. [upbeat music]
