# Choose the project you can verify

Source: https://www.youtube.com/watch?v=y9JtCCZP2MQ

Machine-generated transcript; recognition errors remain. Source-recording timestamps. Boundary-crossing subtitle ends are clipped to this session.

[05:19:18](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19158s) My name is Jared. I'm the creator of Bun, and I work at Anthropic. Bun is the JavaScript runtime that powers cloud code.

[05:19:30](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19170s) And recently I merged a several million line of code PR to add an ahead-of-time compiler to Bun,

[05:19:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19179s) which is basically the idea is instead of paying the cost of a JIT at runtime,

[05:19:47](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19187s) you can compile all the code ahead of time.

[05:19:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19195s) And it's like a really big project.

[05:19:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19198s) Over the summer,

[05:20:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19200s) I rewrote Bun & Rust.

[05:20:05](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19205s) And this is a project on a larger scale.

[05:20:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19208s) This ahead-of-time compiler is like a larger project than that.

[05:20:13](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19213s) And I think one of the challenges right now with large projects is like,

[05:20:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19217s) how do you actually confidently ship millions of lines of code written by agents?

[05:20:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19225s) So that's what I'm going to talk about.

[05:20:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19227s) This was like the opening prompt after some discussion.

[05:20:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19231s) It was like, basically, we need to make Cloud Code use a lot less memory and faster.

[05:20:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19239s) And we can modify either WebKit or Bun.

[05:20:43](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19243s) Bun uses JavaScript core, which is WebKit, under the hood.

[05:20:49](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19249s) And it's been a little bit over a month.

[05:20:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19255s) this is this compiler has gone through five rewrites you can see the like the first couple

[05:21:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19263s) ones were like pretty short and then I like almost shipped the blue one and then I was like

[05:21:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19270s) this is building way too long like it's taking way too long to compile the code so rewrote it again

[05:21:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19277s) and this next one is shipping next week.

[05:21:24](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19284s) This is kind of the way I think about a project like this

[05:21:28](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19288s) is like you're really constrained

[05:21:30](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19290s) by how you test that the code works.

[05:21:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19294s) So that's like even more important

[05:21:36](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19296s) than what you're specifically,

[05:21:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19297s) like the approach you go with for how to build it

[05:21:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19299s) is entirely driven by how do you verify

[05:21:41](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19301s) that it actually is going to like work

[05:21:43](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19303s) and not be buggy and that you can trust it.

[05:21:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19308s) And so just a little more background on like,

[05:21:51](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19311s) like why this particular project,

[05:21:54](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19314s) why the ahead of time compiler,

[05:21:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19315s) and it's because for a typical JavaScript based application,

[05:22:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19320s) roughly half of like the CPU time

[05:22:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19323s) is just spent compiling code on demand.

[05:22:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19327s) And so a lot of the general like thing you've seen

[05:22:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19330s) in a lot of applications where people are rewriting

[05:22:13](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19333s) their code bases in a language like Rust,

[05:22:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19334s) or sometimes go.

[05:22:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19337s) It's like when coming from JavaScript,

[05:22:20](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19340s) it's because it's really the combination of the JIT

[05:22:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19345s) and that it's a single-threaded application.

[05:22:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19351s) So that's sort of the difference between a JIT and

[05:22:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19354s) is that you compile before,

[05:22:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19357s) whereas with a JIT you compile as you go along.

[05:22:41](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19361s) This is particularly important for something like Cloud Code when using the Cloud Code

[05:22:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19366s) like iOS app, where the CPU time is a lot if you have to have every session of Cloud

[05:22:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19375s) Code also compiling.

[05:23:01](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19381s) So we talked a lot about, should we rewrite Cloud Code in Rust?

[05:23:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19387s) because the Bun team, us, rewrote Bun and Rust.

[05:23:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19392s) So we're pretty good at rewrites.

[05:23:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19397s) And we decided not to.

[05:23:21](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19401s) And the reason why is because in Bun's case,

[05:23:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19405s) it's actually pretty easy to do.

[05:23:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19407s) Bun is an easy product to do a Rust rewrite

[05:23:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19409s) because the tests were written in a language

[05:23:32](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19412s) that's independent of the one it runs in.

[05:23:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19414s) So because all the tests were written in TypeScript,

[05:23:38](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19418s) we could run the tests on both the Zig version

[05:23:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19420s) and the Rust version, whereas for Cloud Code,

[05:23:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19426s) we, like Cloud Code's tests are written in TypeScript.

[05:23:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19428s) So we would have to port Cloud Code's test suite.

[05:23:53](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19433s) And as soon as you're like rewriting the tests,

[05:23:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19436s) then you can't really trust the tests anymore.

[05:24:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19440s) So that seemed like it'd be too risky.

[05:24:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19443s) So instead, what if we just make JavaScript faster so we don't have to rewrite it in Rust?

[05:24:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19452s) This was another prompt a little bit later.

[05:24:18](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19458s) And the result is that it actually did have JavaScript's memory, like the memory usage.

[05:24:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19467s) It uses about 4x less memory.

[05:24:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19471s) it starts about two times faster

[05:24:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19477s) and we're dogfooding this internally

[05:24:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19480s) and it'll be external soon

[05:24:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19485s) some more benchmarks

[05:24:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19486s) and then I'll also go into more detail

[05:24:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19488s) about how we actually did this

[05:24:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19490s) so like it's possible that

[05:24:54](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19494s) an interesting thing is like

[05:24:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19496s) like TypeScript 7 rewrote their code by hand

[05:25:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19500s) from JavaScript slash TypeScript to Go,

[05:25:05](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19505s) and the single-threaded version,

[05:25:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19510s) when compiled via Bun compile,

[05:25:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19512s) is now faster than the single-threaded TypeScript Go port.

[05:25:19](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19519s) And we will add threads also

[05:25:21](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19521s) to the ahead-of-time compiled version of Bun,

[05:25:23](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19523s) but we haven't done that yet.

[05:25:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19526s) so there is a PR.

[05:25:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19529s) And so a little bit about the actual ahead of time compiler.

[05:25:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19535s) TypeScript is not, types don't necessarily need to be correct,

[05:25:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19540s) so it treats types as predictions

[05:25:43](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19543s) instead of trusting them.

[05:25:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19548s) And this kind of problem works really well for models

[05:25:51](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19551s) because they're just really good at writing compilers.

[05:25:59](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19559s) So this is sort of like more about like how,

[05:26:05](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19565s) how like should we,

[05:26:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19566s) like how we were thinking about this decision.

[05:26:09](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19569s) And a big thing here is like,

[05:26:11](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19571s) there's just so many tests you can run when you're,

[05:26:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19574s) when it's like in ahead of time,

[05:26:16](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19576s) like for something like this,

[05:26:18](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19578s) because we don't need to write any tests pretty much ourselves.

[05:26:20](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19580s) Like we actually do write a lot of tests,

[05:26:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19582s) but we can largely trust that the,

[05:26:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19586s) tests that predate the project work. And this was the same thing with the Bunn Rust rewrite, where

[05:26:32](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19592s) we had Bunn's existing test suite we could run. And now with the compiler, we have Bunn's existing

[05:26:38](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19598s) test suite plus WebKids tests plus test262 plus CloudCodes tests. And then we also made it so that

[05:26:47](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19607s) That way, BUN itself, BUN's own test runner

[05:26:49](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19609s) could be run in the AOT mode.

[05:26:57](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19617s) And we have one breaking change,

[05:27:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19620s) or really it's two breaking changes

[05:27:01](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19621s) with the at-of-time compiler,

[05:27:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19623s) and that's pretty much why these particular tests

[05:27:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19626s) are failing.

[05:27:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19628s) But the two breaking changes are actually

[05:27:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19632s) mostly security things to prevent prototype pollution

[05:27:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19637s) tax, but they also end up benefiting for performance.

[05:27:23](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19643s) So what makes this particular kind of problem pretty good for agents is you can run all

[05:27:30](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19650s) three versions of an application, one with a JIT, one with ahead of time compiled, and

[05:27:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19654s) one also without the types.

[05:27:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19655s) And as long as the three of them behave the same, then you know that it works, at least

[05:27:41](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19661s) for that specific application.

[05:27:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19662s) but then you can take that kind of problem

[05:27:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19666s) and fuzz it a lot.

[05:27:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19670s) And just have clogged like churn out code generators

[05:27:54](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19674s) to like fuzz different patterns.

[05:27:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19678s) And you can catch a lot of bugs that way.

[05:28:02](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19682s) This is a very large project.

[05:28:04](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19684s) This is the biggest I think I've worked on

[05:28:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19688s) in terms of like code volume.

[05:28:11](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19691s) most of this is tests

[05:28:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19694s) and also because we rewrote it five times

[05:28:19](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19699s) when you're like doing this kind of project

[05:28:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19702s) how I think about it is like you have like

[05:28:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19705s) batches of like okay we need to write a lot of code at once

[05:28:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19709s) and then after that initial batch then it's okay

[05:28:32](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19712s) let's have let's like fix everything make sure it compiles make sure

[05:28:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19717s) it passes a bunch of tests and make sure it like really truly works and the actual logistics of

[05:28:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19725s) that are like when you're in the big batch kind of project stage then you don't want to run builds

[05:28:52](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19732s) you want to run one build because otherwise you're going to spend all your time compiling and like

[05:28:57](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19737s) waiting for build errors and it's going to like not be a good use of anyone's time

[05:29:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19743s) and then separately for like longer running like once you have once you've

[05:29:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19748s) written all the code you want to have it like have lots of individual agents.

[05:29:18](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19758s) This is like a little bit of the timeline of this project.

[05:29:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19766s) the first one was not very good

[05:29:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19771s) like cloud code was a gigabyte

[05:29:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19774s) which is terrible

[05:29:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19779s) and like this used as much as possible

[05:29:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19782s) from JavaScript core's existing code

[05:29:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19785s) so it was like basically the ceiling was

[05:29:47](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19787s) what is like only as good as like

[05:29:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19790s) what it could be from that

[05:29:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19795s) So then a little bit after this, we added a TypeScript type checker into Bunn itself.

[05:30:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19803s) These were prompts. And so we added BunCheck and then we're like, hey, Claude, please use BunCheck. And then after that, startup time got two times faster in Claude code and started using about 3x less memory.

[05:30:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19826s) But the code is slop.

[05:30:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19831s) And like, this is not good, it should never,

[05:30:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19835s) basically this meant that any object

[05:30:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19837s) that accessed a property called archive,

[05:30:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19840s) it would then assume that the bun.archive API

[05:30:43](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19843s) was potentially used.

[05:30:44](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19844s) And I looked at this and I was like,

[05:30:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19846s) we cannot ship this, this is not good.

[05:30:49](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19849s) So then,

[05:30:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19858s) This is like, yeah, and this is another thing too,

[05:31:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19860s) is like, it just like,

[05:31:04](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19864s) like when you're doing like a large planning,

[05:31:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19866s) like when you're starting to like have it

[05:31:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19868s) do a bunch of design docs and stuff,

[05:31:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19870s) it just loves writing markdown docs

[05:31:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19872s) and it doesn't make sense.

[05:31:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19874s) Cause who's gonna like 1500 markdown files

[05:31:18](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19878s) is just not reasonable.

[05:31:28](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19888s) So for this it was like, this was like I think the third compiler version, maybe the fourth,

[05:31:33](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19893s) no this is the third.

[05:31:38](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19898s) It, to like, just to kind of batch get it all out, 140 agents at the same time, write

[05:31:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19905s) all the code all at the same time, then get Cloud Code to run.

[05:31:54](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19914s) And then this is the fourth one.

[05:31:59](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19919s) And this one, the code was not slop, pretty good, with basically the same performance

[05:32:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19926s) as before.

[05:32:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19927s) However, the compilation time was way too slow.

[05:32:11](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19931s) because people coming from a JavaScript background

[05:32:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19934s) don't expect any compilation time.

[05:32:20](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19940s) So 50 seconds for...

[05:32:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19942s) Cloud Code is a pretty large code base,

[05:32:24](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19944s) but this meant that other projects

[05:32:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19947s) would take single-digit seconds,

[05:32:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19949s) and people don't want to wait that long.

[05:32:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19955s) So we're going to rewrite it again

[05:32:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19960s) because in older versions of this compiler

[05:32:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19962s) it was taking like 10 to 20 seconds.

[05:32:47](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19967s) And then Cloud does this thing where like

[05:32:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19970s) it'll try to like size up everything

[05:32:53](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19973s) to judge whether or not it's worth working on

[05:32:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19975s) but it's really easy to convince yourself

[05:32:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19978s) that like your measurements are good

[05:33:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19980s) when they're not good.

[05:33:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19988s) and part of it was also like,

[05:33:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19990s) why is this 900,000 lines of code?

[05:33:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19992s) This should be a lot smaller.

[05:33:19](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=19999s) So then we did it one more time

[05:33:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20002s) and the code was good

[05:33:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20005s) and the build time was under 20 seconds.

[05:33:36](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20016s) It's possible that we end up with a sixth time sometime later, but the fifth one is

[05:33:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20020s) like very shippable, and I've worked on this for enough time that it's like time to ship

[05:33:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20025s) it.

[05:33:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20028s) A lot of this was like, just like, hey, Claude, try this idea, and then see what happens, and

[05:33:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20035s) then it would like size up the idea.

[05:33:57](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20037s) and I think this is like still kind of not right.

[05:34:01](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20041s) Like I think a lot of this are like things

[05:34:04](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20044s) that we should try again and not let it,

[05:34:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20047s) like kind of what I said earlier about like sizing up

[05:34:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20052s) the impact of a change is like it's really easy

[05:34:16](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20056s) to size it up incorrectly and then talk yourself out of it.

[05:34:21](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20061s) So I think this probably needs to be revisited more.

[05:34:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20066s) My old approach, particularly with the Bunn-Russ rewrite,

[05:34:30](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20070s) was very much leaning on cloud code workflows to do things.

[05:34:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20075s) And pretty much now, with the most recent models,

[05:34:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20079s) I mostly just rely on sub-agents that are very long-lived,

[05:34:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20085s) except for that initial part where you're writing all the code all at once.

[05:34:51](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20091s) With the sub-agents, it works pretty well because they keep all the context

[05:34:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20096s) of the relevant code in their head,

[05:34:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20098s) or like in the context window.

[05:35:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20103s) And that's like less wasteful.

[05:35:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20106s) And like they can just keep going.

[05:35:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20114s) Here's a little more about like

[05:35:15](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20115s) how the different parts of the compiler works.

[05:35:19](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20119s) Generally it's like there's,

[05:35:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20122s) if we can confidently prove that

[05:35:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20126s) the value is, like the types are the same,

[05:35:28](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20128s) that they're monomorphic,

[05:35:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20129s) then it can become a field access.

[05:35:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20131s) Otherwise, it has to check every single time.

[05:35:36](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20136s) And it has like some inline caching,

[05:35:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20137s) but not like a lot.

[05:35:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20142s) As with a project of this size,

[05:35:44](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20144s) you really do need to,

[05:35:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20146s) it's kind of hard to follow sometimes,

[05:35:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20148s) like what everything's happening.

[05:35:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20150s) So I lean a lot on like just having an hourly update.

[05:35:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20156s) and where it would post a scorecard

[05:35:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20158s) comparing a bunch of metrics that we were prioritizing.

[05:36:04](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20164s) Usually though, when it was posting in Slack,

[05:36:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20166s) it would show like a previous and this version.

[05:36:09](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20169s) So that way it was easy to see if numbers went up or down,

[05:36:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20172s) but it's like not helpful in a slide

[05:36:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20174s) because the previous one is just the JIT.

[05:36:23](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20183s) The important thing is like,

[05:36:24](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20184s) none of these tests, for the most part, are things that are made up.

[05:36:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20191s) They're things that predated the project,

[05:36:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20194s) so that way we know that the agent couldn't have messed with it.

[05:36:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20200s) Fusili is like a JavaScript engine fuzzer that V8 uses.

[05:36:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20206s) And we were already using that in Bunn,

[05:36:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20208s) but we also extended it to test for TypeScript types as well.

[05:36:59](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20219s) and it's also important to check that the tests actually catch things

[05:37:03](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20223s) because it's really easy to fool yourself that it's finding bugs when it's not finding bugs

[05:37:11](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20231s) some more numbers on the ahead of time compiler it generally for most projects outperforms the JIT

[05:37:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20242s) not always.

[05:37:23](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20243s) You can see in the uglify.js case

[05:37:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20245s) and that's basically if it has to use

[05:37:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20247s) the interpreter a lot, so if there's a lot of

[05:37:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20249s) eval or new function

[05:37:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20251s) then it's not faster

[05:37:33](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20253s) but most applications

[05:37:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20255s) don't do that

[05:37:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20257s) and

[05:37:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20259s) it's like

[05:37:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20262s) it's as close as possible

[05:37:43](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20263s) as you can get to a

[05:37:44](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20264s) one-liner to make an application

[05:37:46](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20266s) run faster.

[05:37:49](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20269s) this is not 100% true

[05:37:50](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20270s) because as you can see

[05:37:52](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20272s) a GlifyJS and there are other cases as well

[05:37:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20275s) but it is

[05:37:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20276s) often true

[05:38:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20280s) so like yeah the other thing is object

[05:38:02](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20282s) prototype this is like

[05:38:04](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20284s) a rare edge case but basically

[05:38:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20287s) there's a lot of security

[05:38:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20288s) vulnerabilities in JavaScript

[05:38:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20290s) applications called like prototype pollution

[05:38:12](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20292s) and this usually works by assigning properties

[05:38:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20294s) onto object.prototype

[05:38:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20297s) and the compiler bans that.

[05:38:22](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20302s) So like for a large project like this,

[05:38:24](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20304s) what do you actually do all day?

[05:38:25](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20305s) These are like the top messages that I sent

[05:38:29](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20309s) saying continue a lot.

[05:38:34](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20314s) Also like my laptop kept dying.

[05:38:42](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20322s) And then like, but like more concretely,

[05:38:44](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20324s) what you actually do is like a bunch of leading questions like why are we doing

[05:38:49](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20329s) this can we do this also I end up saying like can you do this faster or like

[05:38:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20336s) what's taking so long because it kind of like talks itself into making up rules

[05:39:07](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20347s) and also it tends to not be it tends to like underestimate what it can do

[05:39:14](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20354s) so I often end up saying things like this to Claude

[05:39:17](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20357s) to get it to try harder.

[05:39:27](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20367s) And the result is that Jetstream 3

[05:39:31](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20371s) which is one of the main benchmarks

[05:39:33](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20373s) that JavaScript engines use

[05:39:37](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20377s) the bun outperforms both JavaScript cores JIT

[05:39:41](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20381s) and Node.js in like V8s JIT in most of them.

[05:39:52](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20392s) And the whole time it's been using Cloud Code

[05:39:56](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20396s) to build the Bun compile, to build Bun compile itself

[05:39:59](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20399s) and

[05:40:00](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20400s) then using the build of Claude Code that was compiled with Bun Compile.

[05:40:09](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20409s) So sort of how I think about this is like the testability thing is super important when you're working on a large project with Claude and to make sure that the tests actually test things.

[05:40:23](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20423s) and if you're not actively remeasuring

[05:40:26](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20426s) then it's going to get worse

[05:40:28](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20428s) because if there's so much stuff going on

[05:40:30](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20430s) it's really easy to lose track

[05:40:35](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20435s) and it's easier for large projects like this

[05:40:39](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20439s) that haven't shipped yet

[05:40:40](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20440s) to rewrite than it is to refactor

[05:40:45](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20445s) and if you don't act ambitiously

[05:40:48](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20448s) then the models will start having 0.1% improvements

[05:40:53](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20453s) instead of 10% improvements.

[05:40:55](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20455s) So you kind of have to constantly nudge it to like,

[05:40:58](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20458s) hey, you can do really cool things if you try harder.

[05:41:06](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20466s) You can try this now if you want

[05:41:08](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20468s) by running bun upgrade dash dash canary

[05:41:10](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20470s) if you have bun installed, and then just bun compile.

[05:41:20](https://www.youtube.com/watch?v=y9JtCCZP2MQ&t=20480s) Thanks, everyone.