# I deleted 90% of my harness (And It Got Better)

Source: https://www.youtube.com/watch?v=ZZ19zhYnDKY

Machine-generated transcript; recognition errors remain. Source-recording timestamps. Boundary-crossing subtitle ends are clipped to this session.

Opening audio is missing or near-silent. This edition starts with surviving speech and does not reconstruct the missing introduction.

[02:21:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8476s) and harness and the less code we need.

[02:21:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8478s) And if we look at um what we have been

[02:21:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8481s) doing at Google deepmind for the last 6

[02:21:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8482s) months. So at May we had Google IO IO

[02:21:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8485s) and today it's uh November 11 and we

[02:21:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8489s) have been shipping relentlessly.

[02:21:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8490s) Basically just earlier today we

[02:21:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8492s) announced our first decision model with

[02:21:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8495s) Gemini decision one. Yesterday we made

[02:21:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8498s) Gemini 4 carbon available and then if we

[02:21:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8500s) just look at the last six months we

[02:21:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8501s) basically ship now models every few

[02:21:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8503s) weeks. Um as already said Gemini Carbon

[02:21:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8506s) is now available for you. So if you are

[02:21:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8508s) a Google AI ultra subscriber, you can

[02:21:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8510s) use it in anti-gravity, you can use it

[02:21:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8511s) in the Gemini app, you can use it in AI

[02:21:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8513s) studio. Uh carbon is number one on the

[02:21:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8516s) cursor bench uh eval and also number one

[02:21:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8519s) on the perlexity benchmark and

[02:22:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8521s) internally at Google. We have been using

[02:22:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8523s) it already a lot and much more exciting

[02:22:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8526s) is that I'm happy to share that we

[02:22:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8528s) launched the Gemini decisions API

[02:22:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8530s) roughly one hour ago with our new Gemini

[02:22:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8533s) decision one model. So it's a similar

[02:22:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8535s) model to Chef. You can provide all of

[02:22:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8538s) the existing Gemini inputs, meaning it's

[02:22:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8540s) multimodal. So you can provide text,

[02:22:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8542s) audio, images, video, and also PDFs

[02:22:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8545s) natively, and then provide up to 1,000

[02:22:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8547s) questions in a single request. The

[02:22:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8549s) average end to-end latency, including

[02:22:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8551s) all of the network overhead, is between

[02:22:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8553s) 150 to 250 milliseconds, and it costs 10

[02:22:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8557s) cents per 1 million token. So how do

[02:22:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8560s) models get good at work? And that's a

[02:22:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8563s) rough overview on how post- training

[02:22:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8566s) works. So we have a task. We have our

[02:22:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8568s) model. The model has a certain set of

[02:22:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8570s) tools and then we have some kind of

[02:22:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8572s) compute environment. And at the end we

[02:22:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8574s) have some verifier. And the model tries

[02:22:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8575s) to basically solve that task to use

[02:22:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8577s) different tools. And then based on the

[02:22:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8579s) score from the verifier, we update the

[02:23:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8581s) model. But no one is telling the model

[02:23:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8583s) which tools to use. And basically it

[02:23:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8586s) only receives that reward feedback. So

[02:23:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8590s) if we look at two examples on how a

[02:23:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8592s) model can do a certain tasks with tools,

[02:23:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8594s) we have the path A where we have

[02:23:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8596s) dedicated tools for file system control.

[02:23:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8599s) So you all have probably used uh coding

[02:23:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8601s) agents and you have seen a read uh file

[02:23:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8603s) call or an edit file call. So that's one

[02:23:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8606s) way of doing it and the other way of

[02:23:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8608s) doing it would be just using a single

[02:23:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8610s) inline bash script for all of the

[02:23:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8611s) changes. So on the left side we roughly

[02:23:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8614s) need around 10 turns to make the same

[02:23:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8616s) changes to a coding um repository

[02:23:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8619s) compared to the two ch uh turns on the

[02:23:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8621s) right side just using bash and it also

[02:23:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8623s) reflects in the amount of tokens we

[02:23:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8626s) need. So the left path takes around 10x

[02:23:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8629s) more tokens than the life uh the right

[02:23:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8631s) path and it's doing the exact same thing

[02:23:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8634s) uh at the end for the model to achieve

[02:23:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8636s) that task. So in my experiment every

[02:23:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8639s) roughly 6 months I try to build a new

[02:24:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8642s) coding agent harness from scratch and I

[02:24:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8644s) always ask myself okay how much of the

[02:24:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8646s) harness do I really need to achieve the

[02:24:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8648s) same tasks and what is overhead to the

[02:24:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8651s) model and we no longer need. So if we

[02:24:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8653s) look at the coding agent harness which

[02:24:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8656s) um from 2025 we had a roughly around 10

[02:24:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8660s) to 15 to 20 tools. We had dedicated

[02:24:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8662s) tools for um file editing. We had

[02:24:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8666s) dedicated tools for planning, for to-do

[02:24:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8668s) lists and also we had a lot of dedicated

[02:24:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8671s) rules in our system prompt which told

[02:24:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8673s) the model what to do and what not to do

[02:24:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8676s) and that worked of course but since then

[02:24:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8678s) we made a lot of changes. So in my

[02:24:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8680s) experiments uh first one was done

[02:24:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8682s) roughly now two years ago uh in February

[02:24:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8686s) where we did not really had agents right

[02:24:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8687s) we had workflows we basically had

[02:24:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8689s) predefined steps rock applications back

[02:24:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8691s) then were very popular we had a planner

[02:24:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8693s) we had search we reflected on whatever

[02:24:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8695s) was found and then we generated an

[02:24:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8697s) answer then we started to see our first

[02:24:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8699s) agents we had many many different tools

[02:25:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8701s) for all sorts of fine cular actions um

[02:25:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8704s) and then came the coding agents where we

[02:25:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8706s) had those roughly 10 to 20 tools

[02:25:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8709s) And now in my last experiment I tried to

[02:25:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8711s) see okay what's going to happen if I

[02:25:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8713s) provide only one tool to my agent and

[02:25:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8716s) the outcome is was pretty exciting to

[02:25:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8719s) me. So this one tool agent works the

[02:25:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8721s) same way as every other agent except

[02:25:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8723s) that it only has one tool which is bash

[02:25:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8726s) and the model can use that tool for any

[02:25:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8729s) sorts of task and the bash tool is very

[02:25:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8731s) simple. So it runs in a se a se separate

[02:25:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8733s) clean subshell and it supports the

[02:25:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8736s) command argument. I also added a

[02:25:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8738s) description so that it's nice and easier

[02:25:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8740s) to display to the user what the model is

[02:25:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8742s) actually doing and then like some async

[02:25:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8743s) parameters. So basically when the model

[02:25:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8745s) runs a bash command uh the parameter

[02:25:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8747s) tells the model okay if it is uh the

[02:25:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8750s) command longer running than this um

[02:25:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8752s) milliseconds it's being um that the tool

[02:25:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8755s) call resolves and the model receives uh

[02:25:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8757s) a local file handle where it can look up

[02:26:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8760s) later okay if the process is still

[02:26:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8761s) running that's very interesting if you

[02:26:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8763s) do like all sorts of like uh web

[02:26:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8765s) development for example and the single

[02:26:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8768s) batch tool actually works for all of the

[02:26:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8769s) things we are using Asian so we can have

[02:26:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8771s) longunning work where we can start on

[02:26:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8773s) the foreground and then automatically

[02:26:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8775s) move it to the background. The model

[02:26:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8777s) receives some file handle and since it

[02:26:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8780s) has bash available, if we store it to a

[02:26:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8782s) file system, it can read the logs which

[02:26:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8784s) are being streamed. If the agent itself

[02:26:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8786s) is a CLI, it can call itself using bash.

[02:26:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8790s) So there's practically no need to create

[02:26:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8793s) sub aent tools for it because it can

[02:26:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8794s) invoke itself. And then of course we

[02:26:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8796s) don't need dedicated planning or memory

[02:26:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8799s) tools because we have our file system

[02:26:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8801s) available. We can provide some context

[02:26:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8803s) to the model. So the model can keep

[02:26:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8804s) track of its own work with the file

[02:26:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8806s) system and using the bash tools. In

[02:26:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8809s) addition of course to our tools we have

[02:26:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8811s) system prompts and similar to tools uh

[02:26:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8814s) system prompts evolved over time. Um

[02:26:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8817s) every dot here is roughly around 100

[02:26:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8819s) tokens and you can see that coding

[02:27:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8821s) agents mid2025

[02:27:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8823s) had a lot of context we needed to send

[02:27:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8826s) in every request. Of course, this also

[02:27:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8829s) improved and similar story here is like,

[02:27:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8831s) okay, what do I really need to provide

[02:27:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8833s) to my agent to make it successful and

[02:27:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8836s) work and what of all of the system

[02:27:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8838s) information and details we had in the

[02:27:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8840s) past are now basically part of the

[02:27:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8842s) model. So, how much of the harness helps

[02:27:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8846s) and how much is in the way and to find

[02:27:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8849s) that out I was running a lot of

[02:27:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8851s) different evaluations of a lot of

[02:27:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8852s) different agent harnesses with a lot of

[02:27:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8854s) different tools. And one example I like

[02:27:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8856s) to show here which was done with Gemini

[02:27:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8858s) 4 carbon from an eval task from deep sv

[02:27:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8862s) is like okay what happens if I provide

[02:27:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8864s) my existing coding agent tools with read

[02:27:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8867s) write search crap and my bash only tool

[02:27:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8871s) and I think you can pretty clearly see

[02:27:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8873s) that the top line is taking a lot of

[02:27:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8875s) more tokens a lot of more turns and a

[02:27:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8878s) lot of more times to achieve the same

[02:28:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8880s) task and what stood out really well is

[02:28:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8884s) that Okay, reading is done in parallel

[02:28:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8886s) in the beginning, but as soon as we

[02:28:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8888s) start going into making code changes, we

[02:28:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8890s) see that same behavior that the model is

[02:28:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8893s) making turn by turn a single edit, which

[02:28:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8896s) of course consumes a lot of time and a

[02:28:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8898s) lot of tokens. So how does this look at

[02:28:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8903s) a protoscope? So I have my own eval set

[02:28:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8906s) of around 350 tasks for coding knowledge

[02:28:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8909s) work uh and terminal commands all run

[02:28:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8913s) with Gemini 4 argan here and four

[02:28:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8916s) different agent harnesses. One is the

[02:28:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8918s) bash only harness and then okay with

[02:28:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8920s) bash we still have that issue on how can

[02:28:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8923s) I read images right if I'm I cannot cut

[02:28:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8926s) screenshot.png it would return a binary

[02:28:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8929s) and it will fill my context and the

[02:28:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8931s) model cannot see the image. I added uh a

[02:28:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8933s) view media tool which basically returns

[02:28:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8935s) the image in a way the model can see.

[02:28:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8938s) Then we have uh two different agent

[02:29:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8940s) harnesses with structure tools uh for

[02:29:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8942s) file system operations. And if we just

[02:29:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8945s) look at the score column, all of them

[02:29:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8948s) look roughly identical, right? So if we

[02:29:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8951s) would just evaluate our model in ancient

[02:29:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8954s) harness looking at the score, it would

[02:29:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8958s) basically not really matter which one of

[02:29:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8960s) those we would pick. Of course, there's

[02:29:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8961s) a slight variance in the scores, but I

[02:29:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8963s) only ran uh each task one time. So,

[02:29:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8967s) might be just some noise or maybe

[02:29:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8969s) something else. So, you would probably

[02:29:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8970s) need to run a few more samples per per

[02:29:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8972s) harness, but the score looks roughly the

[02:29:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8975s) same. So, how would we now know which

[02:29:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8978s) harness should I choose? If we just look

[02:29:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8981s) all at all of the evol being shown, it's

[02:29:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8984s) always mostly about score. But if we

[02:29:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8986s) look at the token column or even more at

[02:29:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8989s) a dollar per task, there's a picture

[02:29:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8991s) which is changing. So the token per task

[02:29:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8994s) is getting lower. Interestingly, if we

[02:29:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8996s) just use bash and then also the cash hit

[02:29:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8999s) rate increases. So if we just compare

[02:30:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9001s) like the bash media as the the baseline,

[02:30:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9004s) the tasks are almost 2x cheaper compared

[02:30:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9007s) to more complex more tools agent

[02:30:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9011s) harnesses. And we would not see it if we

[02:30:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9014s) just look at the score. And to show you

[02:30:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9018s) a few examples, I prepared some demos.

[02:30:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9020s) So here I'm um using the PI harness,

[02:30:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9023s) which makes it super simple to basically

[02:30:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9025s) define what tools you want to use. I

[02:30:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9028s) have the the same prompt for 3.5 flash,

[02:30:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9031s) which we released roughly 6 months ago,

[02:30:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9034s) uh for all tools. So like whatever pi

[02:30:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9036s) ships out of the box. Then we have it

[02:30:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9038s) for uh bash only and then we have it for

[02:30:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9041s) carbon and uh sorry for carbon with the

[02:30:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9045s) default tools and then with with um bash

[02:30:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9047s) only. This runs now since we only have

[02:30:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9050s) roughly around 20 minutes. I recorded

[02:30:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9054s) all of that and clicked that very nice

[02:30:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9056s) accelerate button here. And what we can

[02:30:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9058s) see on the right side is that the same

[02:31:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9061s) paradigma basically applies. when models

[02:31:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9064s) get better, they roughly need less

[02:31:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9067s) tools. So with 3.5 flash, we see with

[02:31:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9070s) the default tools uh compared to bash

[02:31:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9072s) only, we had the same amount of model

[02:31:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9074s) calls. So we had the same amount of tool

[02:31:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9075s) calls and we had roughly the same amount

[02:31:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9077s) of tokens. And if we look at the traces,

[02:31:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9080s) we can see for bash only, we have some

[02:31:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9083s) cut commands to read different files. We

[02:31:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9085s) have some inline scripts to make uh

[02:31:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9087s) changes and then like some git changes.

[02:31:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9090s) If we look at the default tools, we can

[02:31:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9092s) see roughly the same. We had uh read

[02:31:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9094s) file commands. We used the edit tool and

[02:31:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9096s) then had some git commands to make those

[02:31:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9098s) changes. If we now look at carbon with

[02:31:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9102s) the default tools, it already used way

[02:31:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9104s) less uh tool calls and model calls. So

[02:31:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9108s) it it's basically doing work in parallel

[02:31:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9111s) but also trying to reduce it. So we had

[02:31:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9112s) for example the still same for write

[02:31:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9116s) read calls and also the edit calls. But

[02:31:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9118s) what starts to get interesting if we are

[02:32:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9120s) going to look into bash and in bash we

[02:32:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9123s) had even less tool calls and model calls

[02:32:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9126s) and the interesting part here is that

[02:32:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9128s) the model writes a single Python script

[02:32:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9130s) to make all of that changes in one go

[02:32:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9133s) and no longer needs to call multiple

[02:32:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9135s) tools. So if we go further what does it

[02:32:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9139s) means in terms for for carbon here and

[02:32:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9141s) how bash pays off is that uh the better

[02:32:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9144s) the model gets or the more reasoning and

[02:32:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9147s) like capability it gets bash helps the

[02:32:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9149s) model to be able to chain more commands

[02:32:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9152s) in the single call for carbon it was

[02:32:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9155s) roughly 2.2 to 3.7 uh commands per bash

[02:32:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9159s) call and for flash it was only 1.3 to

[02:32:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9161s) 1.8 eight. So we are seeing up to 3x

[02:32:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9165s) less tokens uh by carbon to achieve the

[02:32:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9170s) same task compared to having the same

[02:32:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9172s) tools we need to use. So two more

[02:32:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9177s) examples to look at it as well is like

[02:32:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9179s) we have on the left side carbon with the

[02:33:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9181s) default tools. So we see our coding

[02:33:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9183s) agent tools with write with edit and we

[02:33:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9186s) see okay it does some parallel work in

[02:33:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9189s) turn two some parallel work in turn

[02:33:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9191s) three and four and then we have all of

[02:33:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9193s) the changes and on the right side where

[02:33:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9195s) it just has the bash tool it uses Python

[02:33:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9197s) to write an inline script to make all of

[02:33:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9200s) that changes at the same time with much

[02:33:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9203s) less token and the same counts for if we

[02:33:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9206s) compare Gemini 3.5 flesh to Gemini 4

[02:33:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9209s) carbon flesh with bash only is not

[02:33:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9212s) chaining those commands. So here it it's

[02:33:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9215s) not really it doesn't matter if I'm

[02:33:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9216s) having a a retool a write tool edit tool

[02:33:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9219s) and a bash tool or if I have just a bash

[02:33:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9221s) tool because 3.5 flesh used to use bash

[02:33:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9224s) for single turn command so it had a

[02:33:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9228s) single turn for reading it had a single

[02:33:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9230s) turn for writing but if we look at

[02:33:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9232s) carbon which in here in this example is

[02:33:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9234s) the exact same task it needs less weight

[02:33:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9237s) turns so the reading is not needed to

[02:33:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9239s) read all of the different files and uh

[02:34:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9241s) completely it just uses rip trap and

[02:34:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9243s) like some intelligent searching to find

[02:34:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9245s) where we need to make changes and then

[02:34:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9247s) it writes a single Python inline bash

[02:34:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9249s) command to make all of those edits in

[02:34:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9251s) one go saving us a lot of tokens. So

[02:34:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9255s) what does it bring us? Models will keep

[02:34:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9257s) eating the harness and I'm pretty sure

[02:34:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9260s) that 2027 will also be a lot more about

[02:34:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9263s) how many tokens do we need to solve a

[02:34:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9265s) task. Even for the few uh improvements

[02:34:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9268s) we made between Argan and Carbon is the

[02:34:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9270s) the better model needs less tokens per

[02:34:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9272s) task which of course is very exciting

[02:34:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9274s) for all of us because then we can use it

[02:34:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9276s) more and also it matters a lot what the

[02:34:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9279s) harness provides and we will might not

[02:34:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9281s) see it if we just look at the accuracy.

[02:34:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9284s) It's also very important that we start

[02:34:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9286s) measuring the token efficiency because

[02:34:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9289s) harnesses can look the same on accuracy

[02:34:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9291s) because our models are now very good. So

[02:34:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9293s) if your harness is more complex to use

[02:34:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9295s) the model will work around it will find

[02:34:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9297s) solutions around it but needs more turns

[02:35:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9300s) and more tokens to achieve the same

[02:35:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9302s) results and then also what comes

[02:35:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9304s) afterwards. So I will try to do the same

[02:35:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9307s) experiment 6 months from now and

[02:35:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9309s) currently the trend we see slowly um

[02:35:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9313s) rising at least on like social media and

[02:35:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9316s) like the industry is that we might go

[02:35:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9318s) back to some code mode tool that we

[02:35:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9321s) instead of writing a bash commands that

[02:35:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9323s) we write TypeScript or Python directly

[02:35:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9326s) that MCP will come back. I'm a big fan

[02:35:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9329s) of MCP and like they are making great

[02:35:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9331s) changes to make it more uh easy easily

[02:35:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9335s) usable especially uh interesting for all

[02:35:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9338s) of that code mode work is uh MCP servers

[02:35:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9340s) now return allow you to return

[02:35:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9342s) structured output so the model can write

[02:35:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9344s) better scripts to take more actions in

[02:35:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9346s) the single turn and it's also allows us

[02:35:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9348s) to do some better proxy around the

[02:35:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9350s) security because bash calls are very

[02:35:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9352s) hard to understand but if I have some

[02:35:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9355s) abstractive layer in typescript I can uh

[02:35:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9358s) watch all of the commands being um

[02:36:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9360s) executed and make sure those are secure

[02:36:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9363s) and for you build to delete your code

[02:36:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9367s) before the model does. give your model

[02:36:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9369s) uh to a computer, not a toolbox. And

[02:36:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9372s) when a new model ships, I definitely

[02:36:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9374s) recommend you um trying to think a

[02:36:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9377s) little bit outside of the box, be more

[02:36:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9379s) ambitious, try to really see, okay, what

[02:36:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9381s) if I remove my complete harness and just

[02:36:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9383s) give it one tool and then also please

[02:36:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9385s) scan the QR code. You will get free

[02:36:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9388s) credits. Uh we will send them out after

[02:36:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9390s) the AI engineering event to try out

[02:36:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9392s) Carbon and also to try out the new

[02:36:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9394s) Gemini decisions API. So if you have

[02:36:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9397s) some time today, I really can recommend

[02:36:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9399s) checking out the new decisions model

[02:36:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=9401s) carbon and thank you for your time.