# What will you build when AI is 10× faster? Source: https://www.youtube.com/watch?v=ZZ19zhYnDKY Machine-generated transcript; recognition errors remain. Source-recording timestamps. Boundary-crossing subtitle ends are clipped to this session. [00:11:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=691s) It's so good to be here. Thank you for [00:11:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=693s) being with me this morning. First [00:11:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=695s) session of the day. I see everyone here [00:11:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=697s) is a morning person. So I'm here to talk [00:11:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=699s) to you about what will you build when AI [00:11:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=702s) is 10 times faster. We're all builders [00:11:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=704s) here. So we're all thinking about what's [00:11:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=706s) next. [00:11:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=708s) Just a bit about me. Um my name is [00:11:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=710s) Natalie. I'm a co-founder of Gimlet [00:11:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=712s) Labs. At Gimlet, we're building an [00:11:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=714s) inference cloud that's designed to [00:11:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=715s) produce much, much faster tokens at high [00:11:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=718s) efficiency, leveraging different [00:12:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=720s) categories and types of chips. At [00:12:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=722s) Gimlet, I work on our kernel generation [00:12:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=724s) side of things across different hardware [00:12:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=726s) types. Um, another thing about me is I [00:12:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=730s) love agents. I use my coding agents for [00:12:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=732s) everything, but I really, really hate [00:12:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=734s) waiting. [00:12:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=737s) And one thing that given that kind of [00:12:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=739s) surprises me is when you're building an [00:12:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=740s) ultra fast inference cloud, people ask [00:12:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=742s) me this all the time actually. They're [00:12:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=744s) like, isn't inference already fast [00:12:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=746s) enough? [00:12:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=748s) And I'm like, what? [00:12:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=750s) But they have a point, right? Because I [00:12:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=752s) can do things and we can all do things [00:12:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=754s) with our agents so much faster than we [00:12:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=756s) could have done before. For example, uh [00:12:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=758s) this slide here, I actually made pretty [00:12:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=761s) using my agent and it took me with my [00:12:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=764s) agent much less time than it would take [00:12:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=766s) me to do that by myself. I'm not very [00:12:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=768s) good at design. And so I think that from [00:12:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=772s) an endto-end perspective, obviously [00:12:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=774s) we're getting more done, but in terms of [00:12:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=776s) is it fast enough, like that's still a [00:12:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=778s) question. So let's take a look at what [00:13:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=780s) happened when I used my agent to make [00:13:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=782s) that slide look pretty. You can see the [00:13:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=784s) ugly before slide on the right. [00:13:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=788s) So, we're going to press play here. [00:13:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=792s) And I said to my agent, can you improve [00:13:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=794s) the visual look of this slide? I love [00:13:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=796s) Figma. I use Figma for everything. So, [00:13:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=798s) of course, I'm going to use it with my [00:13:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=800s) agents. So, what we can see on the [00:13:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=802s) bottom here is the timeline of what [00:13:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=805s) happened. And it the text is a little [00:13:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=806s) small to see, but red is inference. Red [00:13:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=808s) is the time spent that the model is in [00:13:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=810s) inference. Green is the time it's spent [00:13:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=812s) making tool calls. and blue is the time [00:13:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=814s) that it's spent taking my input. And we [00:13:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=818s) can see that, you know, this this [00:13:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=819s) agent's doing good work. It's doing a [00:13:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=821s) lot of thinking, but I'm waiting here. [00:13:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=824s) I'm actually waiting so long I sped up [00:13:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=825s) this video 9x because I didn't want all [00:13:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=827s) of you to have to wait for the full turn [00:13:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=829s) to end. It ended up being about 2 [00:13:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=832s) minutes long. But we can see at the end [00:13:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=834s) of this after my one input, we have this [00:13:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=836s) nice pretty slide at the end. So, you [00:13:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=838s) know, on one hand, this is actually much [00:13:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=839s) faster than a human can do. But on the [00:14:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=842s) other hand, in those about two minutes, [00:14:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=843s) I have probably moved on to another [00:14:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=845s) task. I'm not really focused on this [00:14:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=847s) anymore. I've context switched. [00:14:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=851s) And when we look to, you know, the [00:14:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=853s) inevitable progression of inference [00:14:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=856s) getting faster and faster, I think it [00:14:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=857s) helps to look back in time and look back [00:14:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=860s) to the internet era. Um, I mean, we're [00:14:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=862s) still in the internet, but like you know [00:14:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=863s) what people canonically call the web [00:14:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=864s) era. And what we always see with a new [00:14:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=867s) shift is that with each new major wave, [00:14:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=870s) new capabilities are unlocked. And so we [00:14:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=872s) do things that weren't possible before [00:14:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=874s) that technology, like using an agent to [00:14:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=876s) edit my slide. So I think we should take [00:14:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=878s) a step back to 2000 in the early days of [00:14:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=880s) the internet. I think many people here [00:14:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=882s) weren't even alive, so it'll be a [00:14:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=883s) history lesson for you. And we're going [00:14:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=885s) to look at a little application called [00:14:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=888s) Squirrel Mail. Squirrel Mail is actually [00:14:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=890s) open source, so you can run it today. It [00:14:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=892s) was a web application for uh you know [00:14:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=893s) doing your email back then. [00:14:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=896s) So on the left what we have here is I'm [00:14:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=898s) running squirrel mail and I'm running it [00:15:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=901s) with simulated 2000 dialup speed and [00:15:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=905s) then on the right it'll run with 2026 [00:15:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=907s) speed. So the speed that we're used to [00:15:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=909s) today. So the both sides are going to [00:15:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=911s) take the same actions but um the left [00:15:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=914s) side I'm using it with that simulated [00:15:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=916s) lower uh speed internet [00:15:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=921s) and we can see that you know it's doing [00:15:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=923s) its job. Back in 2000 loading an image [00:15:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=927s) like this would have blown people's [00:15:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=928s) minds but our modern eyes are like wow [00:15:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=931s) that took forever to show up. [00:15:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=933s) And so we can see these lags. we can see [00:15:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=935s) a noticeable thing, but on the same [00:15:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=937s) time, someone might have looked at this [00:15:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=938s) back in 2000 and said, "This is fast [00:15:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=940s) enough. I don't actually need this to be [00:15:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=942s) faster." [00:15:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=946s) The thing is though, all of the major [00:15:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=948s) giants back in that era did find out it [00:15:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=951s) mattered. Pretty much every major [00:15:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=953s) company from Amazon to Yahoo to Google [00:15:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=956s) to Bing, all of or being at part of [00:15:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=959s) Microsoft like all of them found that [00:16:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=961s) even very small delays cost them users. [00:16:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=963s) It cost them people coming back. It cost [00:16:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=966s) them people doing as many clicks. It [00:16:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=968s) cost them literal money. Even a hundred [00:16:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=971s) milliseconds of delay added on to the [00:16:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=973s) typical experience had a demonstrable [00:16:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=976s) impact on the bottom line. And even when [00:16:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=979s) you got rid of that delay, those same [00:16:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=980s) users didn't come back to the platform [00:16:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=983s) as often as they did without that delay. [00:16:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=989s) And this makes sense because fast [00:16:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=991s) products keep us in flow state. They [00:16:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=993s) provide a stickier UX. About 100 [00:16:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=996s) milliseconds feels instant to us. One [00:16:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=998s) seconds, one second of weight, I still [00:16:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1001s) keep my train of thought. It's not a big [00:16:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1002s) deal. I don't really notice it. 10 [00:16:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1004s) seconds, we're going to start turning to [00:16:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1006s) something else from a psychology [00:16:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1008s) perspective. Maybe we pull out our [00:16:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1009s) phone, maybe switch to another agent. [00:16:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1011s) But today, from data from June, the [00:16:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1013s) median coding agent turn is 63 seconds. [00:16:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1019s) Each workload has different speed [00:17:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1021s) requirements. We don't need the same [00:17:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1023s) speed for everything. But over time, our [00:17:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1025s) expectations do rise in the fastest days [00:17:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1028s) of fastest internet in the squirrel mail [00:17:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1030s) days would feel incredibly slow to us [00:17:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1032s) and we can keep we can expect that trend [00:17:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1034s) to continue for inference speed as well. [00:17:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1038s) There's a problem though and the problem [00:17:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1039s) is everyone knows that we have massive [00:17:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1042s) compute scarcity. Today's infrastructure [00:17:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1044s) is under incredible pressure to deliver [00:17:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1046s) two things at the same time. More tokens [00:17:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1049s) and faster tokens. Unfortunately, these [00:17:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1052s) goals are fundamentally at odds because [00:17:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1055s) faster tokens require more energy to [00:17:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1057s) produce than slower tokens. So, with the [00:17:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1060s) same energy footprint, you can't make as [00:17:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1061s) many of them. [00:17:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1064s) Back before this was a session and not a [00:17:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1067s) keynote, thanks to upgrading this to a [00:17:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1069s) keynote everyone. Um, we uh we're going [00:17:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1072s) to play a game that I made uh called [00:17:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1074s) Token Mint Factory. And you can actually [00:17:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1077s) play this. I will not be offended if you [00:17:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1078s) play it right now even, but just so you [00:17:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1079s) know, it does not work on phones. It [00:18:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1081s) only works on your laptop. And basically [00:18:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1083s) in this game, you're running a token [00:18:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1084s) mint factory. You're printing literal [00:18:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1086s) tokens. And there's different modes. The [00:18:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1088s) modes are you can play to maximize how [00:18:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1091s) quickly you can put out one single [00:18:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1092s) token. Or you can maximize how many [00:18:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1095s) tokens you can put out in 60 seconds. Or [00:18:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1098s) you can play the SLA mode where you have [00:18:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1100s) to maintain a 20 second SLA on each [00:18:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1102s) token for it to count, but you're still [00:18:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1103s) trying to put out as many as you can. [00:18:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1105s) And as you play, you'll quickly see that [00:18:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1107s) the strategy is extremely different for [00:18:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1109s) doing fast tokens versus doing slow [00:18:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1111s) tokens. And neither one is actually [00:18:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1113s) better or worse from a decision [00:18:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1115s) perspective. It's just a matter of what [00:18:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1117s) the application needs and what makes [00:18:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1118s) sense for your factory. [00:18:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1121s) So I took real runs from people at my [00:18:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1124s) company that played this game and I made [00:18:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1126s) a paro frontier of these tokens in the [00:18:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1128s) game. And we can see there's a real life [00:18:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1131s) paro frontier of this game as well. If [00:18:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1134s) we have faster tokens, we don't have as [00:18:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1135s) many. And if we have more tokens, [00:18:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1137s) they're not as fast. But there is a way [00:19:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1140s) to get more up and to the right. And up [00:19:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1141s) and to the right is where we want to go. [00:19:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1143s) We want more faster tokens. I don't want [00:19:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1145s) to compromise. I want both. So the game [00:19:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1148s) has a concept of adding upgrades to your [00:19:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1150s) equipment. And what we can see is that [00:19:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1152s) compared to the no upgrade baseline, [00:19:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1154s) these technological innovations help [00:19:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1157s) drive us up into the right and get more [00:19:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1159s) tokens and faster tokens. [00:19:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1163s) So how do we upgrade our real life [00:19:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1165s) assembly lines so to speak to make [00:19:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1168s) making our actual tokens faster? Uh one [00:19:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1171s) thing that we do at Gimlet for this is [00:19:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1173s) we really believe that you have to use [00:19:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1175s) the right accelerator for the right job. [00:19:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1177s) And so we actually take models and we [00:19:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1180s) split them up into different pieces and [00:19:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1182s) we run each piece on a different [00:19:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1184s) category of accelerator. So we might run [00:19:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1186s) one piece of the workload on a GPU and [00:19:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1189s) another piece of the workload on [00:19:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1191s) something like an SRM ccentric [00:19:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1192s) accelerator. And we find that even with [00:19:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1194s) the network overhead, this delivers [00:19:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1196s) dramatically faster tokens. It's [00:19:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1197s) essentially like the same thing as the [00:19:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1199s) upgrade to the equipment in the token [00:20:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1201s) factory when you better match the [00:20:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1203s) accelerator to the job. Unfortunately, [00:20:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1205s) we have ended the time of free lunch [00:20:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1207s) transistor scaling, right? So now we [00:20:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1209s) have to look to other approaches to make [00:20:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1211s) things faster. [00:20:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1214s) And so with this, it's a little hard to [00:20:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1216s) read, but we can see the same real life [00:20:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1218s) paro frontier for a frontier model. If [00:20:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1221s) you look at, you know, simple [00:20:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1223s) homogeneous GPU inference, it's very [00:20:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1225s) high throughput. And then the latency [00:20:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1226s) drops off a little bit. You don't you [00:20:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1228s) can't get it as fast because the y-axis [00:20:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1230s) is how many tokens that I can make and [00:20:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1232s) the x-axis how fast these tokens are for [00:20:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1234s) each user. And we can see that depending [00:20:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1237s) on how you split the model up because [00:20:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1239s) there's different ways of doing it. You [00:20:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1241s) can get much faster tokens for the same [00:20:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1243s) power footprint. [00:20:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1245s) You can get many more tokens for the [00:20:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1248s) same speed or you can get more and [00:20:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1251s) faster tokens. So as an industry we need [00:20:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1254s) to continue pushing pushing pushing on [00:20:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1256s) this because otherwise our poor data [00:20:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1257s) centers are just going to keel over. [00:21:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1261s) So I just wanted to show a quick demo of [00:21:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1263s) this. So um using a coding agent build [00:21:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1266s) the same app using heterogeneous [00:21:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1268s) inference that leverages different chip [00:21:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1269s) types versus the traditional one. [00:21:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1274s) So this is going to be the game snake. [00:21:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1276s) And we can see on the left the [00:21:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1277s) heterogeneous fast one's done. I can see [00:21:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1280s) in my flow state I can keep adding [00:21:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1281s) things. I can keep playing but on the [00:21:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1283s) right side it's still going. And by the [00:21:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1285s) way I still think the left side's too [00:21:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1287s) slow. We need to make that even faster. [00:21:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1291s) And then we can add a competitor to the [00:21:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1293s) game and we can see even with the tool [00:21:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1295s) calls, even with all the things that [00:21:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1296s) it's doing, it's going to complete and [00:21:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1299s) then the other one will still be hanging [00:21:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1300s) because it's fundamentally about how can [00:21:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1302s) you drive more efficiency and better [00:21:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1304s) performance with the same resources. [00:21:49](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1309s) It's going to go on. I'm not going to [00:21:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1310s) make you guys wait for this. [00:21:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1313s) One thing that people often point out [00:21:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1314s) too is is inference the bottleneck? Our [00:21:57](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1317s) tool calls the bottleneck. And I think [00:21:59](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1319s) this is a great point. If we look back [00:22:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1321s) to the turn timeline from the time that [00:22:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1324s) I was spending updating my slide, we can [00:22:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1327s) see that 80% of it was inference and 18% [00:22:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1329s) of it was tool calls. If we simulate [00:22:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1332s) that same exact thing and have 10x [00:22:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1335s) faster inference, we can see that 67% of [00:22:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1338s) that will be tool calls at that point if [00:22:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1340s) we don't optimize the tool calls. That's [00:22:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1343s) why it's important that we have to [00:22:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1344s) constantly attack the bottlenecks of the [00:22:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1346s) application and get the lowhanging fruit [00:22:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1347s) where we can. Today, people are focused [00:22:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1349s) on inference speed and maybe we haven't [00:22:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1351s) optimized the tool call side and how [00:22:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1353s) tools are called as much because it's [00:22:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1355s) not the bottleneck. If I cut away the [00:22:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1357s) tool calls on the upper one, it wouldn't [00:22:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1358s) even really matter to the end [00:22:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1359s) application. But once we get much [00:22:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1361s) faster, we have to think about how to [00:22:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1363s) call these tools much more efficiently [00:22:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1365s) and much more performantly because [00:22:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1366s) otherwise they're going to be the reason [00:22:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1368s) that we're all waiting around. [00:22:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1373s) I think the good news is that optimizing [00:22:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1375s) these speeds and what we're all [00:22:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1376s) collectively doing as an industry is a [00:22:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1378s) verifiable task. If I know what my [00:23:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1380s) inputs and outputs should be, I can have [00:23:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1383s) agents do this very very full stack. And [00:23:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1386s) that's where we're increasingly moving [00:23:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1388s) to. We're having self-improvement loops [00:23:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1390s) with these agents to drive better [00:23:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1392s) efficiency on the fly as our inference [00:23:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1394s) is running. And at Gimlet, we're [00:23:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1396s) building recursive self-optimization for [00:23:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1398s) this that pulls from low-level telemetry [00:23:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1400s) from our data centers all the way up to [00:23:22](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1402s) the model layer and trying to use all of [00:23:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1404s) it to in to improve the overall [00:23:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1406s) efficiency of the workload because if [00:23:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1408s) you just optimize one small piece, it [00:23:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1410s) might not make that much of a dent. [00:23:35](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1415s) I want to show another demo because I [00:23:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1417s) think that cyber security is on [00:23:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1418s) everyone's mind right now. Um, we're all [00:23:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1421s) seeing like the huge incredible advances [00:23:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1423s) in AI driving awareness that there's all [00:23:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1426s) these vulnerabilities in our software [00:23:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1428s) and I think that this is an area where [00:23:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1430s) faster inference can really really help. [00:23:53](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1433s) So this is a dumb fake scenario but it's [00:23:55](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1435s) a real application and a real demo where [00:23:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1438s) imagine that you're running the dog club [00:24:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1440s) and that's a website for dog lovers to [00:24:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1442s) upload and vote on pictures of dogs. [00:24:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1445s) However, unfortunately, like we often [00:24:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1447s) see, one of your node dependencies has a [00:24:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1450s) vulnerability and this is a real [00:24:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1452s) vulnerability. This is really a node tar [00:24:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1455s) and you're running a security agent on [00:24:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1457s) your application and it's trying to [00:24:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1459s) monitor live traffic and see is there [00:24:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1461s) anything going wrong that I don't know [00:24:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1463s) about. This agent doesn't know about [00:24:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1464s) this vulnerability, but it does know its [00:24:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1466s) job is to protect your application. [00:24:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1471s) So, let's do a quick turn with normal [00:24:33](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1473s) user traffic. We're going to upload a [00:24:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1476s) dog biscuit. [00:24:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1479s) And on the upper side, we have the [00:24:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1481s) application logs. And on the bottom, we [00:24:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1483s) have the defender. And the defender is [00:24:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1484s) monitoring, but it's not in the critical [00:24:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1486s) path because it's not fast enough to [00:24:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1488s) actually respond to every user request [00:24:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1490s) live. It's monitoring things async. [00:24:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1494s) And we can see biscuit gets uploaded and [00:24:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1496s) we see that this is an okay, you know, [00:24:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1498s) this is an okay upload. The defender [00:25:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1500s) doesn't see any problem. [00:25:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1502s) But now what we're going to do is have [00:25:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1504s) this defender operating when there's a [00:25:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1505s) malicious upload. And this upload is [00:25:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1507s) exploiting the real vulnerability where [00:25:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1509s) in node tar there was a release where [00:25:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1511s) you could basically overwrite files that [00:25:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1513s) you weren't supposed to overwrite based [00:25:15](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1515s) on the destinations of what was being [00:25:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1517s) unzipped. [00:25:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1519s) So in this one we have Ranger, a fake [00:25:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1524s) bad uh malicious dog. Not really a dog. [00:25:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1527s) And what we're going to see is that the [00:25:30](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1530s) defender is going to figure out that [00:25:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1532s) this is a malicious thing. The agent is [00:25:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1534s) smart enough to detect that it's doing [00:25:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1536s) something that it shouldn't be doing, [00:25:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1537s) even though it hasn't seen this exact [00:25:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1538s) vulnerability before. But unfortunately, [00:25:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1541s) Cat's malicious upload has taken over. [00:25:44](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1544s) So your website's gone. It's too late. [00:25:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1547s) You didn't react in time. [00:25:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1552s) But if we retry this same exact [00:25:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1554s) malicious upload, but this time with an [00:25:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1556s) ultra fast defender, and again, this is [00:25:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1558s) a real a real demo fake use case. Um, we [00:26:01](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1561s) can see that with ultra fast inference, [00:26:03](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1563s) we can react to things and do things [00:26:05](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1565s) online much more quickly. And this [00:26:07](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1567s) should change the way that we think of [00:26:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1569s) how to use agents in the first place. [00:26:10](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1570s) We're all thinking about how to use them [00:26:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1572s) in an async way, but we should think [00:26:14](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1574s) about how to use them in online ways as [00:26:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1576s) well. So we can see on the bottom that [00:26:18](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1578s) the worker blocked their request and it [00:26:20](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1580s) was like what are you doing? This is [00:26:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1581s) obviously wrong. [00:26:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1586s) So faster internet took us from squirrel [00:26:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1589s) mail to ondemand highresolution [00:26:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1592s) streaming, right? Like if you just look [00:26:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1594s) from this to this, it's crazy. If you [00:26:36](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1596s) showed this to my like 5-year-old self [00:26:38](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1598s) or something, I would have been like my [00:26:39](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1599s) mind would have been blown. And this is [00:26:41](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1601s) just one of many things that I can [00:26:42](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1602s) select. So, like what does a 10,000 [00:26:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1605s) token per second world look like? We're [00:26:47](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1607s) at in the like low hundreds today. What [00:26:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1610s) does it look like when we get to a [00:26:51](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1611s) million tokens per second? Like what [00:26:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1612s) does that unlock for us? [00:26:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1616s) And I think that we should all ask [00:26:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1618s) ourselves this too. It's important in AI [00:27:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1620s) to skate to where the puck is moving, [00:27:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1622s) right? It's changing so fast. We need to [00:27:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1624s) be ready. We need to think boldly. It's [00:27:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1626s) not enough to just do what we've always [00:27:08](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1628s) been doing. I think we've all had the [00:27:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1629s) experience where we make an audacious [00:27:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1631s) request to our agent and after the [00:27:13](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1633s) latest model release, it just works. And [00:27:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1636s) it's going to be the same thing with [00:27:17](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1637s) latency. And so I think that we can [00:27:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1639s) think about ideas that are both coming [00:27:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1641s) soon when we hit a certain latency [00:27:23](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1643s) threshold, what we can do, but also the [00:27:25](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1645s) stuff that is real life sci-fi. Like if [00:27:27](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1647s) you look at something like Jarvis or [00:27:29](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1649s) something, there's no way today's [00:27:31](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1651s) inference could be fast enough to power [00:27:32](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1652s) that experience for Iron Man. But we [00:27:34](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1654s) will get there. So, I'm really excited [00:27:37](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1657s) about online use cases, online security, [00:27:40](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1660s) online things of like I want to just [00:27:43](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1663s) tell my agent what I want to build and [00:27:45](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1665s) it's building it right in front of me [00:27:46](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1666s) and I don't even have to type anything. [00:27:48](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1668s) I think we're going to have new patterns [00:27:50](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1670s) of interaction. You know, think tab [00:27:52](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1672s) complete for everything. How can we have [00:27:54](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1674s) the user side of things be as quick and [00:27:56](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1676s) immediate as possible because the [00:27:58](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1678s) inference side is also just as fast to [00:28:00](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1680s) accommodate it. And we're going to have [00:28:02](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1682s) so many more agents. We're going to have [00:28:04](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1684s) even maybe transformers driving robots, [00:28:06](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1686s) right? There's so much that we can build [00:28:09](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1689s) when this gets unlocked. And I think as [00:28:11](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1691s) builders on the frontier here, we should [00:28:12](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1692s) all be thinking about this, too. [00:28:16](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1696s) Thank you so much for being with me um [00:28:19](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1699s) this morning. Uh you can find me on X at [00:28:21](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1701s) NSino. Um we're hiring at Gimlet. Would [00:28:24](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1704s) love to talk to you all. And then if you [00:28:26](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1706s) want to play the token mint game, uh you [00:28:28](https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=1708s) can also find that QR here. Thank you so