1
07:26:09,360 --> 07:26:11,670
good to go here. Um, all right. Welcome

2
07:26:11,680 --> 07:26:14,310
everyone. Glad to be here. I'm Fabian,

3
07:26:14,320 --> 07:26:18,150
co-founder and CTO of of Lovable. And um

4
07:26:18,160 --> 07:26:20,150
we spent the last three years or so

5
07:26:20,160 --> 07:26:22,788
building a platform to let anyone create

6
07:26:22,798 --> 07:26:24,708
software and then operate that and

7
07:26:24,718 --> 07:26:28,868
deploy that software. Um and um the the

8
07:26:28,878 --> 07:26:32,548
topic of today is what it takes to set

9
07:26:32,558 --> 07:26:37,350
intelligence free. Um really um for us

10
07:26:37,360 --> 07:26:40,230
this takes us back to when we when we

11
07:26:40,240 --> 07:26:42,708
started three years ago. And um I have

12
07:26:42,718 --> 07:26:45,510
to say I have a bit of a complicated

13
07:26:45,520 --> 07:26:48,548
relationship with with agents. Um and I

14
07:26:48,558 --> 07:26:51,830
I'll tell you why. Um we almost failed

15
07:26:51,840 --> 07:26:55,190
as a company because we thought um back

16
07:26:55,200 --> 07:26:58,230
in 2023 that hey we these agents are the

17
07:26:58,240 --> 07:27:00,388
future. So let's build for them now. Um

18
07:27:00,398 --> 07:27:02,628
I think the we kind of turned out to be

19
07:27:02,638 --> 07:27:04,468
true that they were the future but we

20
07:27:04,478 --> 07:27:06,230
were definitely wrong on the the now

21
07:27:06,240 --> 07:27:08,548
piece in in 23. So this was back in kind

22
07:27:08,558 --> 07:27:13,750
of GPT4 days. Um and um at the at that

23
07:27:13,760 --> 07:27:15,350
time the intelligence was simply not

24
07:27:15,360 --> 07:27:17,110
there to have more of these kind of

25
07:27:17,120 --> 07:27:20,468
complex agentic systems. Um but now it

26
07:27:20,478 --> 07:27:23,510
is right now uh now the intelligence is

27
07:27:23,520 --> 07:27:25,270
actually at a place where where you can

28
07:27:25,280 --> 07:27:27,110
really build out more complicated

29
07:27:27,120 --> 07:27:29,910
systems around them. Um, and so I I'll

30
07:27:29,920 --> 07:27:32,948
give you kind of a a quick um background

31
07:27:32,958 --> 07:27:36,708
story here of of our journey um in in

32
07:27:36,718 --> 07:27:39,670
working with intelligence or or these

33
07:27:39,680 --> 07:27:42,310
LMS. And so

34
07:27:42,320 --> 07:27:45,190
um back in 2023, we built the first

35
07:27:45,200 --> 07:27:46,868
version of our product. We called it GP

36
07:27:46,878 --> 07:27:50,310
Engineer back then. Um and really the UI

37
07:27:50,320 --> 07:27:51,750
was very similar to what it is today

38
07:27:51,760 --> 07:27:53,030
where you can kind of have a chat and

39
07:27:53,040 --> 07:27:55,750
this preview. Um but the AI at that

40
07:27:55,760 --> 07:27:58,868
point was quite simple. We just had a

41
07:27:58,878 --> 07:28:01,110
prompt and a response, one LM call per

42
07:28:01,120 --> 07:28:03,670
user message. Uh, very simple. It it

43
07:28:03,680 --> 07:28:05,830
worked fine. Um, but you could kind of

44
07:28:05,840 --> 07:28:08,230
build one page apps and and simpler

45
07:28:08,240 --> 07:28:12,150
applications with this. Um, and then we

46
07:28:12,160 --> 07:28:14,388
thought, okay, this intelligence is

47
07:28:14,398 --> 07:28:16,150
going to keep on evolving. So, we have

48
07:28:16,160 --> 07:28:19,110
to be ready for uh for it as it evolves.

49
07:28:19,120 --> 07:28:20,708
And so, we thought, let's go out and

50
07:28:20,718 --> 07:28:23,510
build this agentic framework. And so we

51
07:28:23,520 --> 07:28:25,190
we went and we built this very

52
07:28:25,200 --> 07:28:27,750
sophisticated there is a beautiful

53
07:28:27,760 --> 07:28:30,788
framework for for agents to work. And um

54
07:28:30,798 --> 07:28:33,190
we had kind of this one main agent that

55
07:28:33,200 --> 07:28:34,868
would talk to the user and that agent

56
07:28:34,878 --> 07:28:37,270
would spawn other agents like an

57
07:28:37,280 --> 07:28:39,350
implementation agent here in the example

58
07:28:39,360 --> 07:28:41,670
and each agent could have a specialtity.

59
07:28:41,680 --> 07:28:43,830
This agent would kind of specialize in

60
07:28:43,840 --> 07:28:45,350
the first generation of an app and then

61
07:28:45,360 --> 07:28:46,548
you would have like a front-end agent

62
07:28:46,558 --> 07:28:48,468
and a backend agent and they would all

63
07:28:48,478 --> 07:28:50,150
talk to each other, send messages back

64
07:28:50,160 --> 07:28:52,708
and forth. and we thought, "Oh, this is

65
07:28:52,718 --> 07:28:54,070
beautiful." And there was papers around

66
07:28:54,080 --> 07:28:56,708
this and how well this could work. And

67
07:28:56,718 --> 07:28:58,948
in the end, it didn't work at all. We

68
07:28:58,958 --> 07:29:02,628
just made the product worse. Um, we um

69
07:29:02,638 --> 07:29:06,230
we definitely failed on this one. Um,

70
07:29:06,240 --> 07:29:08,708
but luckily before we ran out of money,

71
07:29:08,718 --> 07:29:10,230
we figured out like, hey, let's go back

72
07:29:10,240 --> 07:29:12,868
to a simpler system again. So uh in late

73
07:29:12,878 --> 07:29:15,910
24 we went back to a much simpler um

74
07:29:15,920 --> 07:29:17,830
system where it was a bit of a shame

75
07:29:17,840 --> 07:29:19,750
where we had like some some smaller LMS

76
07:29:19,760 --> 07:29:22,230
doing some knowledge retrieval and then

77
07:29:22,240 --> 07:29:24,150
really just one main LM kind of writing

78
07:29:24,160 --> 07:29:26,310
to the user and and writing the code in

79
07:29:26,320 --> 07:29:28,468
the same message. And this worked great.

80
07:29:28,478 --> 07:29:30,708
It was very responsive for for users in

81
07:29:30,718 --> 07:29:33,110
the UI and it was reliable and users

82
07:29:33,120 --> 07:29:34,868
were kind of in it and and iterating

83
07:29:34,878 --> 07:29:39,910
with uh with the model. Um, and so, um,

84
07:29:39,920 --> 07:29:43,590
let's close that down. Um, but now it's

85
07:29:43,600 --> 07:29:45,670
time again. We're back. We're so back.

86
07:29:45,680 --> 07:29:48,628
Agents this time, uh, they work. Um,

87
07:29:48,638 --> 07:29:50,310
and, uh, we're running them in

88
07:29:50,320 --> 07:29:51,910
production over a billion tokens per

89
07:29:51,920 --> 07:29:53,830
minute on on on these systems that I'm

90
07:29:53,840 --> 07:29:57,430
going to talk about. Um, but, um, first

91
07:29:57,440 --> 07:29:59,750
just looking at okay, what is actually

92
07:29:59,760 --> 07:30:03,270
loveable? So uh if you simplify the UI

93
07:30:03,280 --> 07:30:06,150
and this was same back in 2023 as it is

94
07:30:06,160 --> 07:30:08,548
today. Uh to the left we have a chat

95
07:30:08,558 --> 07:30:10,310
where humans come in and they kind of

96
07:30:10,320 --> 07:30:12,468
describe their their intent to to build

97
07:30:12,478 --> 07:30:14,310
or whatever they want to do. And to the

98
07:30:14,320 --> 07:30:16,948
right you have a preview uh which is the

99
07:30:16,958 --> 07:30:19,110
output or or the outcomes of of what

100
07:30:19,120 --> 07:30:23,590
you're building. Um and this back in 23

101
07:30:23,600 --> 07:30:25,590
you could do simpler things with right

102
07:30:25,600 --> 07:30:28,708
but now we've really seen working at

103
07:30:28,718 --> 07:30:30,388
this layer of abstraction you're able to

104
07:30:30,398 --> 07:30:32,948
build very complex things right and I

105
07:30:32,958 --> 07:30:34,948
think we've even seen uh a lot of

106
07:30:34,958 --> 07:30:37,110
tooling meant for for developers for

107
07:30:37,120 --> 07:30:39,430
example moving to this type of interface

108
07:30:39,440 --> 07:30:40,788
where you're not really looking at the

109
07:30:40,798 --> 07:30:42,548
code every day and so on and you're

110
07:30:42,558 --> 07:30:44,868
getting kind of abstracted and working

111
07:30:44,878 --> 07:30:47,750
on on be architecture uh or looking at

112
07:30:47,760 --> 07:30:49,510
some kind of output of of what you're

113
07:30:49,520 --> 07:30:52,708
doing. And um this is something that

114
07:30:52,718 --> 07:30:54,548
we've always believed in and it's been

115
07:30:54,558 --> 07:30:56,388
kind of a core thesis behind lovable

116
07:30:56,398 --> 07:30:58,548
that you won't have to care about the

117
07:30:58,558 --> 07:31:01,270
code in in the same way that you had had

118
07:31:01,280 --> 07:31:03,030
to do before. And I think today we're

119
07:31:03,040 --> 07:31:05,270
kind of in this almost a bit in between

120
07:31:05,280 --> 07:31:06,708
I think in the industry where you know

121
07:31:06,718 --> 07:31:08,150
some people might say you don't have to

122
07:31:08,160 --> 07:31:09,910
care about code at all and some people

123
07:31:09,920 --> 07:31:13,430
might have different opinions. Um um I

124
07:31:13,440 --> 07:31:15,270
of course like growing up coding and

125
07:31:15,280 --> 07:31:17,510
these things you get a bit attached u

126
07:31:17,520 --> 07:31:20,230
but in the end what matters is u is

127
07:31:20,240 --> 07:31:23,270
actually not the code um sorry if we

128
07:31:23,280 --> 07:31:25,430
remove that last part but uh what what

129
07:31:25,440 --> 07:31:29,510
matters is uh is the outcomes right um

130
07:31:29,520 --> 07:31:33,030
and so um the outcomes what is that well

131
07:31:33,040 --> 07:31:36,548
for for our users the outcomes is um uh

132
07:31:36,558 --> 07:31:39,110
creating a business and monetizing and

133
07:31:39,120 --> 07:31:41,270
creating value for for others in in in

134
07:31:41,280 --> 07:31:44,070
some type of way. And I'll kind of

135
07:31:44,080 --> 07:31:47,270
quickly go over um what we build as a

136
07:31:47,280 --> 07:31:48,788
platform to to enable this before I go

137
07:31:48,798 --> 07:31:50,948
in kind of on the uh on the technical

138
07:31:50,958 --> 07:31:53,830
side of this talk. So if you if you look

139
07:31:53,840 --> 07:31:56,868
at lovable the first thing that that you

140
07:31:56,878 --> 07:31:58,150
can say about the platform is that we

141
07:31:58,160 --> 07:32:00,868
have this agent and like any other agent

142
07:32:00,878 --> 07:32:02,948
it can do quite arbitrary things and

143
07:32:02,958 --> 07:32:05,110
it's quite specialized at at building

144
07:32:05,120 --> 07:32:08,548
software uh and it has access to a

145
07:32:08,558 --> 07:32:10,150
browser so it can verify what it's

146
07:32:10,160 --> 07:32:13,350
building and um it connects to to GitHub

147
07:32:13,360 --> 07:32:15,430
and you know it writes code that and you

148
07:32:15,440 --> 07:32:17,030
can do whatever you want with that code

149
07:32:17,040 --> 07:32:19,510
so on. So that's um kind of your basic

150
07:32:19,520 --> 07:32:22,230
uh coding coding agent. Um and the the

151
07:32:22,240 --> 07:32:23,910
other pieces of the platform is this

152
07:32:23,920 --> 07:32:25,750
kind of cloud and and business piece of

153
07:32:25,760 --> 07:32:28,948
the platform. And so um the cloud part

154
07:32:28,958 --> 07:32:31,190
of the platform is is about just giving

155
07:32:31,200 --> 07:32:33,590
kind of these u hosting primitives just

156
07:32:33,600 --> 07:32:35,990
seamlessly to to this agent. And so we

157
07:32:36,000 --> 07:32:38,070
we built out databases, authentication,

158
07:32:38,080 --> 07:32:40,150
storage, all the types of backend

159
07:32:40,160 --> 07:32:42,868
compute and security scanners and like

160
07:32:42,878 --> 07:32:44,230
backups and logs for all of these

161
07:32:44,240 --> 07:32:45,910
things. And all of it can be

162
07:32:45,920 --> 07:32:48,468
introspected by by the agent. Um, and

163
07:32:48,478 --> 07:32:50,230
this just makes it seamless to to build

164
07:32:50,240 --> 07:32:52,628
things. You don't have to kind of go to

165
07:32:52,638 --> 07:32:54,230
different services to set this together.

166
07:32:54,240 --> 07:32:55,910
We just kind of provide you uh these

167
07:32:55,920 --> 07:32:58,468
primitives. Um, so that's still around

168
07:32:58,478 --> 07:32:59,910
building software and hosting this,

169
07:32:59,920 --> 07:33:02,150
right? Um, but then the last piece is

170
07:33:02,160 --> 07:33:05,430
around these business capabilities where

171
07:33:05,440 --> 07:33:07,590
again the output of what our users are

172
07:33:07,600 --> 07:33:08,948
doing on lovable, we don't see that

173
07:33:08,958 --> 07:33:11,750
output as code. the output is is a

174
07:33:11,760 --> 07:33:15,350
product or or a business. Um, and if you

175
07:33:15,360 --> 07:33:17,590
want to succeed as a business, you need

176
07:33:17,600 --> 07:33:19,830
to do more things than just write code,

177
07:33:19,840 --> 07:33:22,230
right? And so we we we help our users

178
07:33:22,240 --> 07:33:23,990
with uh receiving payments, sending

179
07:33:24,000 --> 07:33:27,190
emails, optimizing SEO, um, doing ads on

180
07:33:27,200 --> 07:33:29,910
on Google and meta um, and kind of

181
07:33:29,920 --> 07:33:31,670
analyzing what is happening with the

182
07:33:31,680 --> 07:33:34,628
product through analytics and you can

183
07:33:34,638 --> 07:33:36,230
look at kind of conversions and and

184
07:33:36,240 --> 07:33:37,910
optimize these sort of things. and the

185
07:33:37,920 --> 07:33:40,150
agent is in the loop in all of this,

186
07:33:40,160 --> 07:33:43,430
right? So, that's kind of our uh

187
07:33:43,440 --> 07:33:46,070
capability offering, you could say. Um,

188
07:33:46,080 --> 07:33:47,990
and again, the core part of this is an

189
07:33:48,000 --> 07:33:49,750
agent that all of these capabilities are

190
07:33:49,760 --> 07:33:53,670
built for. Um, and at some point in

191
07:33:53,680 --> 07:33:55,670
level history, we used to limit our

192
07:33:55,680 --> 07:33:57,830
agents to work for only 30 minutes

193
07:33:57,840 --> 07:33:59,510
because we thought they weren't reliable

194
07:33:59,520 --> 07:34:01,670
to to work for a longer amount of time.

195
07:34:01,680 --> 07:34:04,468
Um but uh now we've changed this to

196
07:34:04,478 --> 07:34:07,350
really be uh without any any sort of

197
07:34:07,360 --> 07:34:09,270
bound in terms of the length an agent

198
07:34:09,280 --> 07:34:11,110
can work. And so you can imagine agents

199
07:34:11,120 --> 07:34:14,230
working for for 10 hours plus. Um and so

200
07:34:14,240 --> 07:34:16,070
I'll I'll go into some of the things

201
07:34:16,080 --> 07:34:18,548
that we've done um in our architecture

202
07:34:18,558 --> 07:34:20,868
to to enable this. And so the first

203
07:34:20,878 --> 07:34:23,510
piece is really around the agent runtime

204
07:34:23,520 --> 07:34:26,070
itself and how how we enable that to to

205
07:34:26,080 --> 07:34:27,830
run for a longer amount of time. And the

206
07:34:27,840 --> 07:34:31,270
second piece is the computer where each

207
07:34:31,280 --> 07:34:32,628
of these agents are actually doing their

208
07:34:32,638 --> 07:34:36,388
work running a dev server um and um

209
07:34:36,398 --> 07:34:38,230
actually verifying what it's doing. And

210
07:34:38,240 --> 07:34:39,990
so so the last piece then is the

211
07:34:40,000 --> 07:34:42,548
verification itself where if you want an

212
07:34:42,558 --> 07:34:44,070
agent to to work for a long amount of

213
07:34:44,080 --> 07:34:45,350
time, you need to make sure that you

214
07:34:45,360 --> 07:34:47,510
have these feedback loops in place so

215
07:34:47,520 --> 07:34:49,750
that it's not kind of going off uh in in

216
07:34:49,760 --> 07:34:53,670
in the wrong direction. Um and so if we

217
07:34:53,680 --> 07:34:56,070
start by looking at the agent runtime

218
07:34:56,080 --> 07:35:00,388
itself um you can imagine that um um you

219
07:35:00,398 --> 07:35:02,708
have um really three different sources

220
07:35:02,718 --> 07:35:06,070
that can um interact with an agent. So

221
07:35:06,080 --> 07:35:07,670
it can be yourself kind of prompting

222
07:35:07,680 --> 07:35:09,670
this or a human. It could be other

223
07:35:09,680 --> 07:35:11,670
agents and it could also be just a

224
07:35:11,680 --> 07:35:14,468
system on on a schedule waking up. And

225
07:35:14,478 --> 07:35:16,388
the way we chose to architect this is

226
07:35:16,398 --> 07:35:18,788
that for for each agent we have a

227
07:35:18,798 --> 07:35:21,350
separate inbox uh where any of these

228
07:35:21,360 --> 07:35:23,670
sources would kind of send in messages.

229
07:35:23,680 --> 07:35:25,750
And so you might have a message here

230
07:35:25,760 --> 07:35:27,990
like what's the status? Um and you can

231
07:35:28,000 --> 07:35:29,590
imagine you have this agent and this

232
07:35:29,600 --> 07:35:31,670
agent is kind of an always on agent that

233
07:35:31,680 --> 07:35:33,910
that might be working right and so um

234
07:35:33,920 --> 07:35:35,750
you can imagine it has done like three

235
07:35:35,760 --> 07:35:37,670
actions here a request a tool call and

236
07:35:37,680 --> 07:35:39,910
it got some some result and then at this

237
07:35:39,920 --> 07:35:41,510
point the agent is ready to actually

238
07:35:41,520 --> 07:35:45,750
receive uh messages. So um since anyone

239
07:35:45,760 --> 07:35:47,590
can drop a message into this inbox, the

240
07:35:47,600 --> 07:35:50,310
agent then can copy that itself into its

241
07:35:50,320 --> 07:35:51,910
own uh what we call a trajectory

242
07:35:51,920 --> 07:35:54,628
basically it's a task history. Um and

243
07:35:54,638 --> 07:35:56,310
this is the type of interaction that I

244
07:35:56,320 --> 07:35:57,670
think you're familiar with if if you're

245
07:35:57,680 --> 07:36:01,430
using agents overall. Um so that's kind

246
07:36:01,440 --> 07:36:03,430
of the the first design principle uh

247
07:36:03,440 --> 07:36:05,590
behind this agent runtime. I'll go

248
07:36:05,600 --> 07:36:09,350
through a few more here. And so um the

249
07:36:09,360 --> 07:36:12,150
the trajectory in which the the agent is

250
07:36:12,160 --> 07:36:13,670
working in which is kind of the history

251
07:36:13,680 --> 07:36:16,070
of what has happened so far um and

252
07:36:16,080 --> 07:36:18,868
really the basis for any sort of of

253
07:36:18,878 --> 07:36:21,830
memory for for for the agent right um

254
07:36:21,840 --> 07:36:24,948
and we designed this trajectory with uh

255
07:36:24,958 --> 07:36:27,830
really git as inspiration and so the

256
07:36:27,840 --> 07:36:29,750
trajectory what is it it's an append

257
07:36:29,760 --> 07:36:33,830
only log where each event uh points

258
07:36:33,840 --> 07:36:37,270
towards a parent the previous event Um

259
07:36:37,280 --> 07:36:41,750
and because of this you get um a nice um

260
07:36:41,760 --> 07:36:43,270
capability just like in git where you

261
07:36:43,280 --> 07:36:45,190
can fork things right. So you can have

262
07:36:45,200 --> 07:36:46,868
two events pointing to the same parent

263
07:36:46,878 --> 07:36:49,830
which essentially creates a fork. Um and

264
07:36:49,840 --> 07:36:51,510
because of that you can kind of spin up

265
07:36:51,520 --> 07:36:55,190
parallel agents at um um at any point of

266
07:36:55,200 --> 07:36:57,030
time in the history without having to

267
07:36:57,040 --> 07:36:59,270
duplicate all of the kind of the shared

268
07:36:59,280 --> 07:37:02,388
history. Um and um semantically it's a

269
07:37:02,398 --> 07:37:03,590
bit unclear what would happen if you

270
07:37:03,600 --> 07:37:05,590
merge this. So unlike it, we cannot

271
07:37:05,600 --> 07:37:07,750
merge agent trajectories, but we can

272
07:37:07,760 --> 07:37:09,670
kind of um spawn them out, right? And

273
07:37:09,680 --> 07:37:11,990
these agents can can talk to each other.

274
07:37:12,000 --> 07:37:14,948
Um the other thing that uh we made sure

275
07:37:14,958 --> 07:37:17,830
to design for is that this uh agent

276
07:37:17,840 --> 07:37:20,070
trajectory, it includes kind of the full

277
07:37:20,080 --> 07:37:22,150
history and everything that happens uh

278
07:37:22,160 --> 07:37:24,868
with the agent. And so every tool call

279
07:37:24,878 --> 07:37:26,388
um a bunch of other things happening on

280
07:37:26,398 --> 07:37:28,230
the platform if a tool call like failed

281
07:37:28,240 --> 07:37:30,548
or succeeded and a bunch of and what

282
07:37:30,558 --> 07:37:32,150
we've done is make sure that we have

283
07:37:32,160 --> 07:37:35,030
this appendon log and and from that we

284
07:37:35,040 --> 07:37:36,708
can actually build a prompt that we

285
07:37:36,718 --> 07:37:39,750
would feed into into the actual LM and

286
07:37:39,760 --> 07:37:41,430
um you might ask why why why would you

287
07:37:41,440 --> 07:37:44,310
want to do this? Well um one one reason

288
07:37:44,320 --> 07:37:46,868
is that um when you build a prompt you

289
07:37:46,878 --> 07:37:48,310
if you have a long history you probably

290
07:37:48,320 --> 07:37:49,910
want to do some sort of compaction. you

291
07:37:49,920 --> 07:37:52,310
want to summarize part of that history

292
07:37:52,320 --> 07:37:54,468
um in in the earlier steps for example.

293
07:37:54,478 --> 07:37:57,110
Um but this summarization would differ

294
07:37:57,120 --> 07:37:59,030
um for example if you use different

295
07:37:59,040 --> 07:38:00,708
models with different context lens you

296
07:38:00,718 --> 07:38:02,788
would probably think differently about

297
07:38:02,798 --> 07:38:05,190
when you should summarize and not. And

298
07:38:05,200 --> 07:38:07,670
so by separating the the prompt from the

299
07:38:07,680 --> 07:38:09,830
actual task history, we're able to

300
07:38:09,840 --> 07:38:11,590
render prompts for for any type of model

301
07:38:11,600 --> 07:38:15,030
or or any type of task um that that we

302
07:38:15,040 --> 07:38:16,548
can kind of find the best and optimal

303
07:38:16,558 --> 07:38:21,190
prompt for. Um, so to show this event

304
07:38:21,200 --> 07:38:23,350
history in in progress, we have this

305
07:38:23,360 --> 07:38:24,948
debug tool here. So here it's just

306
07:38:24,958 --> 07:38:27,590
recording of me dragging this uh slider

307
07:38:27,600 --> 07:38:29,590
in our UI. And you can see on the right

308
07:38:29,600 --> 07:38:32,388
that uh um the UI is kind of updating as

309
07:38:32,398 --> 07:38:34,070
I'm as I'm dragging this along and you

310
07:38:34,080 --> 07:38:36,788
have 156 events here and one each of

311
07:38:36,798 --> 07:38:39,270
those is an event in this event history.

312
07:38:39,280 --> 07:38:41,430
Uh and you can see how how the UI can

313
07:38:41,440 --> 07:38:43,590
render kind of any point in history. And

314
07:38:43,600 --> 07:38:46,150
this is just showcasing that at any

315
07:38:46,160 --> 07:38:49,190
point in time we have full control of um

316
07:38:49,200 --> 07:38:51,350
of say the UI or or anything else

317
07:38:51,360 --> 07:38:53,510
happening on the this platform thanks to

318
07:38:53,520 --> 07:38:55,510
um this history kind of detailing

319
07:38:55,520 --> 07:38:57,830
everything that has happened.

320
07:38:57,840 --> 07:39:00,868
Um the the last piece I'll go over here

321
07:39:00,878 --> 07:39:03,830
on the on the runtime itself is that um

322
07:39:03,840 --> 07:39:06,628
the process that is running these agents

323
07:39:06,638 --> 07:39:08,230
they're separate from from the actual

324
07:39:08,240 --> 07:39:11,270
task history. And so the process is uh

325
07:39:11,280 --> 07:39:13,430
almost entirely stateless in that

326
07:39:13,440 --> 07:39:15,750
between each each call each call each

327
07:39:15,760 --> 07:39:18,548
turn um it will actually just persist

328
07:39:18,558 --> 07:39:20,788
into the task history and so you can

329
07:39:20,798 --> 07:39:24,070
have a process come and go. Um and then

330
07:39:24,080 --> 07:39:25,590
the question is why why do we care about

331
07:39:25,600 --> 07:39:28,310
this? Um well

332
07:39:28,320 --> 07:39:31,110
uh at Lovable this system um we deploy

333
07:39:31,120 --> 07:39:33,990
to it um 700 times per day. So that's

334
07:39:34,000 --> 07:39:36,708
about once every two minutes. And you

335
07:39:36,718 --> 07:39:39,350
can imagine if you have um a agent

336
07:39:39,360 --> 07:39:43,030
working for for 10 hours um you want to

337
07:39:43,040 --> 07:39:44,548
not disrupt that agent when it's

338
07:39:44,558 --> 07:39:46,150
working, right? This this agent is doing

339
07:39:46,160 --> 07:39:48,628
something for for a customer. Um but at

340
07:39:48,638 --> 07:39:49,830
the same time we want to make sure that

341
07:39:49,840 --> 07:39:51,910
this agent is running on the latest

342
07:39:51,920 --> 07:39:53,910
deploy. and it might be a new feature or

343
07:39:53,920 --> 07:39:55,350
something like that, but it might also

344
07:39:55,360 --> 07:39:57,990
be like a security vulnerability in in

345
07:39:58,000 --> 07:39:59,750
the Linux kernel and it's like, okay, we

346
07:39:59,760 --> 07:40:02,708
need to update this ASAP. Um, and and so

347
07:40:02,718 --> 07:40:04,548
to do that, thanks to this separation

348
07:40:04,558 --> 07:40:06,548
between the actual history and and the

349
07:40:06,558 --> 07:40:09,270
processes, we can actually make sure

350
07:40:09,280 --> 07:40:11,830
that we have one one agent that is

351
07:40:11,840 --> 07:40:13,590
working on one task, but we can have

352
07:40:13,600 --> 07:40:15,590
different processes taking up and

353
07:40:15,600 --> 07:40:17,590
actually doing the compute behind that

354
07:40:17,600 --> 07:40:20,230
work. Um so so this task can keep on

355
07:40:20,240 --> 07:40:22,388
working while we have um different

356
07:40:22,398 --> 07:40:26,150
deploys um working on it be it for 10

357
07:40:26,160 --> 07:40:29,590
hours or or even longer right um so

358
07:40:29,600 --> 07:40:32,150
that's the agent runtime. The second

359
07:40:32,160 --> 07:40:34,948
piece of um of this platform right is

360
07:40:34,958 --> 07:40:36,948
the is the computer. So this is where

361
07:40:36,958 --> 07:40:39,110
the work is actually happening. Um and

362
07:40:39,120 --> 07:40:41,270
we've been through a lot of iterations

363
07:40:41,280 --> 07:40:43,190
and uh a lot of vendors kind of working

364
07:40:43,200 --> 07:40:45,990
with um um providing these sandboxes for

365
07:40:46,000 --> 07:40:48,310
for the computer for for the agent to

366
07:40:48,320 --> 07:40:50,310
work in. And we've landed in in this

367
07:40:50,320 --> 07:40:52,468
system that we've worked really hard now

368
07:40:52,478 --> 07:40:56,868
to develop which is um um checkpointing

369
07:40:56,878 --> 07:40:59,430
um a computer. And so you can imagine

370
07:40:59,440 --> 07:41:00,708
you come to level you start a new

371
07:41:00,718 --> 07:41:03,030
project. Uh we have this common what we

372
07:41:03,040 --> 07:41:06,070
call a golden checkpoint which contains

373
07:41:06,080 --> 07:41:08,310
the OS and the system services. It has

374
07:41:08,320 --> 07:41:10,548
common tools installed and a dev server

375
07:41:10,558 --> 07:41:12,468
that is ready to start. And this

376
07:41:12,478 --> 07:41:14,468
checkpoint, it it includes everything on

377
07:41:14,478 --> 07:41:16,468
disk uh memory and also running

378
07:41:16,478 --> 07:41:19,030
processes. And and so when you start a

379
07:41:19,040 --> 07:41:20,868
new project, you get all of that for

380
07:41:20,878 --> 07:41:22,948
free. And uh all of that is copied over

381
07:41:22,958 --> 07:41:26,868
in about 230 milliseconds. Um and so

382
07:41:26,878 --> 07:41:28,788
once you start a project, you you might

383
07:41:28,798 --> 07:41:30,628
have some specific code and and some

384
07:41:30,638 --> 07:41:33,910
dependencies within that project. Um and

385
07:41:33,920 --> 07:41:37,510
um that is also checkpointed as you go.

386
07:41:37,520 --> 07:41:40,468
And and so each time the the agent is

387
07:41:40,478 --> 07:41:43,270
running and it finishes a turn, we will

388
07:41:43,280 --> 07:41:45,110
actually save a checkpoint o of that

389
07:41:45,120 --> 07:41:47,350
computer and and from that we're able to

390
07:41:47,360 --> 07:41:50,548
again like very quickly subsecond uh uh

391
07:41:50,558 --> 07:41:52,310
spin that computer up whenever you need

392
07:41:52,320 --> 07:41:53,750
it. That means we can save on on

393
07:41:53,760 --> 07:41:55,750
resources by kind of spinning this down.

394
07:41:55,760 --> 07:41:58,948
If you leave for say 10 minutes or or so

395
07:41:58,958 --> 07:42:00,548
and come back, we can just spin it up

396
07:42:00,558 --> 07:42:01,750
extremely quickly. So you don't even

397
07:42:01,760 --> 07:42:04,788
notice this and even uh kind of requests

398
07:42:04,798 --> 07:42:07,190
um in transit will will not even be

399
07:42:07,200 --> 07:42:10,708
dropped in in in this setup. Um the

400
07:42:10,718 --> 07:42:12,948
other thing we can do thanks to uh this

401
07:42:12,958 --> 07:42:15,270
checkpointing is that just like the

402
07:42:15,280 --> 07:42:17,350
trajectory itself uh in the agent

403
07:42:17,360 --> 07:42:20,788
runtime we can also fork um the computer

404
07:42:20,798 --> 07:42:23,590
and when we do this um we have for

405
07:42:23,600 --> 07:42:26,628
example we have um a feature in in level

406
07:42:26,638 --> 07:42:28,070
that is called draft where you kind of

407
07:42:28,080 --> 07:42:29,590
work in different environments of what

408
07:42:29,600 --> 07:42:31,110
you're doing and and to support that we

409
07:42:31,120 --> 07:42:32,628
need to fork the computer that you're

410
07:42:32,638 --> 07:42:38,150
working in. Um and um here we have um um

411
07:42:38,160 --> 07:42:40,548
kind of the example of doing two at the

412
07:42:40,558 --> 07:42:42,628
same time. And um you can imagine doing

413
07:42:42,638 --> 07:42:46,150
hundreds of these, right? Um and so um

414
07:42:46,160 --> 07:42:48,548
we've we've had to do uh some

415
07:42:48,558 --> 07:42:50,548
optimizations to make this feasible to

416
07:42:50,558 --> 07:42:52,150
to actually run. So one of these

417
07:42:52,160 --> 07:42:54,628
optimizations is copy on write on the

418
07:42:54,638 --> 07:42:57,030
RAM itself. And so creating one of these

419
07:42:57,040 --> 07:42:59,510
forks of a computer is uh is basically

420
07:42:59,520 --> 07:43:01,670
free in terms of of the RAM that it's

421
07:43:01,680 --> 07:43:04,230
using. But um as soon as you start

422
07:43:04,240 --> 07:43:07,350
writing um um in in a process to to RAM,

423
07:43:07,360 --> 07:43:09,750
we will kind of copy that page over and

424
07:43:09,760 --> 07:43:11,670
only then will you start having to

425
07:43:11,680 --> 07:43:15,430
allocate new memory. Um and there's

426
07:43:15,440 --> 07:43:17,750
another optimization here also which is

427
07:43:17,760 --> 07:43:20,070
that um each of these sandboxes require

428
07:43:20,080 --> 07:43:22,548
a few hundreds of megabytes of memory.

429
07:43:22,558 --> 07:43:25,270
Um but and that would take a few seconds

430
07:43:25,280 --> 07:43:27,110
to stream in if you have it in like cold

431
07:43:27,120 --> 07:43:28,708
object storage and then it has to go to

432
07:43:28,718 --> 07:43:30,788
disk and then then into memory of this

433
07:43:30,798 --> 07:43:36,150
uh um this VM. But um thanks to uh um

434
07:43:36,160 --> 07:43:37,910
being able to stream the memory, we can

435
07:43:37,920 --> 07:43:41,190
actually boot up the the machine before

436
07:43:41,200 --> 07:43:44,788
all of the memory is is actually um

437
07:43:44,798 --> 07:43:47,430
ready on it. So the the memory the OS

438
07:43:47,440 --> 07:43:48,948
uses comes first and then like some

439
07:43:48,958 --> 07:43:50,708
processes and so on will will start to

440
07:43:50,718 --> 07:43:52,230
work as soon as the memory gets there.

441
07:43:52,240 --> 07:43:55,110
And this also shaves off some some extra

442
07:43:55,120 --> 07:43:57,350
startup time. Um, and then we started

443
07:43:57,360 --> 07:43:58,788
asking ourselves, how can we make this

444
07:43:58,798 --> 07:44:00,310
even more efficient and even more faster

445
07:44:00,320 --> 07:44:02,310
for our users? And so you might be

446
07:44:02,320 --> 07:44:04,070
familiar with V. It's this great dev

447
07:44:04,080 --> 07:44:06,230
server that most of the meta frameworks

448
07:44:06,240 --> 07:44:08,150
out there in the JavaScript ecosystem is

449
07:44:08,160 --> 07:44:10,708
using. Uh, and so we we took that and we

450
07:44:10,718 --> 07:44:13,430
forked it and we rewrote it in in Rust.

451
07:44:13,440 --> 07:44:15,910
And uh, that made our code starts two

452
07:44:15,920 --> 07:44:17,350
times faster and we went from getting

453
07:44:17,360 --> 07:44:19,350
this preview from 14 seconds to to 3

454
07:44:19,360 --> 07:44:21,750
seconds. And the main um kind of thing

455
07:44:21,760 --> 07:44:23,270
that we made faster here is that vit

456
07:44:23,280 --> 07:44:25,910
uses noode.js and we're able to like

457
07:44:25,920 --> 07:44:28,230
drop that dependency off itself and just

458
07:44:28,240 --> 07:44:30,868
have this rust binary. So um that made

459
07:44:30,878 --> 07:44:33,990
the memory footprint be 6.5 times less

460
07:44:34,000 --> 07:44:36,628
uh in these machines. Um and by taking

461
07:44:36,638 --> 07:44:38,548
control of the applications that we're

462
07:44:38,558 --> 07:44:39,910
running in these computers, we're also

463
07:44:39,920 --> 07:44:42,468
able to control kind of how we write to

464
07:44:42,478 --> 07:44:44,868
memory. And so when we control how we

465
07:44:44,878 --> 07:44:46,628
write to memory, we're even more

466
07:44:46,638 --> 07:44:48,310
optimizing. If you remember back to the

467
07:44:48,320 --> 07:44:49,910
kind of the copy and write solution when

468
07:44:49,920 --> 07:44:52,948
we're drafting different um um different

469
07:44:52,958 --> 07:44:55,430
computers and so we're able to uh run

470
07:44:55,440 --> 07:44:57,110
this extremely efficiently so that we

471
07:44:57,120 --> 07:44:59,830
can run um lots and lots of agents in in

472
07:44:59,840 --> 07:45:02,310
parallel and kind of have agents explore

473
07:45:02,320 --> 07:45:04,868
different solutions to solving a bug or

474
07:45:04,878 --> 07:45:06,628
improving the performance of of your

475
07:45:06,638 --> 07:45:10,388
applications and and so on. Um and so

476
07:45:10,398 --> 07:45:13,350
this has been fun. Um uh but the last

477
07:45:13,360 --> 07:45:15,830
piece now is is about verification. And

478
07:45:15,840 --> 07:45:17,270
of course this kind of computer and this

479
07:45:17,280 --> 07:45:19,350
dev server is a big piece of being able

480
07:45:19,360 --> 07:45:23,670
to verify what an agent is doing. Um and

481
07:45:23,680 --> 07:45:25,830
if you look at verification, you can

482
07:45:25,840 --> 07:45:27,510
think of these feedback loops, right?

483
07:45:27,520 --> 07:45:29,110
That the agent needs to get information.

484
07:45:29,120 --> 07:45:30,708
It took an action. Was this a good

485
07:45:30,718 --> 07:45:33,270
action or was it a bad action? And um

486
07:45:33,280 --> 07:45:35,030
you can think of it as this inner loop

487
07:45:35,040 --> 07:45:36,628
and this outer loop. So this inner loop

488
07:45:36,638 --> 07:45:38,948
is um during the turn of the agent. And

489
07:45:38,958 --> 07:45:40,628
you have kind of obvious things such as

490
07:45:40,638 --> 07:45:42,948
like the um is the build working? Do we

491
07:45:42,958 --> 07:45:44,548
have lint errors or type errors in the

492
07:45:44,558 --> 07:45:46,468
code that we're running? And then you

493
07:45:46,478 --> 07:45:48,230
can check for runtime errors in in the

494
07:45:48,240 --> 07:45:49,910
preview that you're running. You can

495
07:45:49,920 --> 07:45:51,670
also have within this computer a

496
07:45:51,680 --> 07:45:53,670
headless browser where the agent can go

497
07:45:53,680 --> 07:45:56,468
in and kind of explore um the the web

498
07:45:56,478 --> 07:45:58,468
app it's it's building or whatever it's

499
07:45:58,478 --> 07:46:00,708
building, right? Um and and a bunch of

500
07:46:00,718 --> 07:46:02,150
other things that we're using to help

501
07:46:02,160 --> 07:46:04,548
verify within the turn of an agent. But

502
07:46:04,558 --> 07:46:06,548
then you also have this outer loop which

503
07:46:06,558 --> 07:46:08,788
is kind of between each turn and the

504
07:46:08,798 --> 07:46:09,990
turn you can think of like when the

505
07:46:10,000 --> 07:46:12,788
agent is yielding to you as a user. Um

506
07:46:12,798 --> 07:46:15,350
and um between each turn you might get

507
07:46:15,360 --> 07:46:17,910
some info on on the product itself uh

508
07:46:17,920 --> 07:46:19,670
from from production from your users,

509
07:46:19,680 --> 07:46:21,030
right? And so you can look at how your

510
07:46:21,040 --> 07:46:23,990
visitors are behavior are behaving. Um

511
07:46:24,000 --> 07:46:26,310
what users are are telling you online.

512
07:46:26,320 --> 07:46:28,868
Uh you can do automatic AB tests and

513
07:46:28,878 --> 07:46:31,350
actually see change some copy and see is

514
07:46:31,360 --> 07:46:33,190
this improving conversion and so on. And

515
07:46:33,200 --> 07:46:34,468
then you can also take these business

516
07:46:34,478 --> 07:46:36,230
metrics and and remember this goes back

517
07:46:36,240 --> 07:46:37,750
to kind of the capabilities I showed

518
07:46:37,760 --> 07:46:39,750
you. By having control over these

519
07:46:39,760 --> 07:46:41,190
things, we can make sure that we can

520
07:46:41,200 --> 07:46:43,270
optimize how the agent is is getting

521
07:46:43,280 --> 07:46:45,110
this out of feedback loop. So, we can

522
07:46:45,120 --> 07:46:46,628
make sure that if you're running

523
07:46:46,638 --> 07:46:49,350
payments, we can actually track those

524
07:46:49,360 --> 07:46:51,190
transactions and conversions and make

525
07:46:51,200 --> 07:46:53,510
sure that um the agent can actually help

526
07:46:53,520 --> 07:46:56,388
you optimize uh not only the the code,

527
07:46:56,398 --> 07:46:59,270
right, but the the business strategy um

528
07:46:59,280 --> 07:47:03,510
around around your product. Um and so

529
07:47:03,520 --> 07:47:05,270
those are really the the three things

530
07:47:05,280 --> 07:47:08,230
that that we've been working for. And uh

531
07:47:08,240 --> 07:47:11,110
we'll continue to kind of build out um

532
07:47:11,120 --> 07:47:13,990
these uh these capabilities and this

533
07:47:14,000 --> 07:47:17,030
architecture around really orchestrating

534
07:47:17,040 --> 07:47:20,310
these LMS into into agents. And so um

535
07:47:20,320 --> 07:47:21,910
this agentic system that we're building,

536
07:47:21,920 --> 07:47:23,430
it's running billions of tokens per

537
07:47:23,440 --> 07:47:26,868
minute. Um and um these computers,

538
07:47:26,878 --> 07:47:28,628
right, we can spawn up with each agent

539
07:47:28,638 --> 07:47:30,388
and we can have multiple agents use one

540
07:47:30,398 --> 07:47:32,150
computer. And now we're kind of back to

541
07:47:32,160 --> 07:47:34,788
this uh quite beautiful setup where we

542
07:47:34,798 --> 07:47:36,150
have agents talking to agents and

543
07:47:36,160 --> 07:47:38,708
they're orchestrating and and this time

544
07:47:38,718 --> 07:47:41,270
it's it's actually working and it's

545
07:47:41,280 --> 07:47:43,030
working at scale thanks to some of these

546
07:47:43,040 --> 07:47:45,830
optimizations. Um and so if you're

547
07:47:45,840 --> 07:47:47,190
curious to try this out, you can head

548
07:47:47,200 --> 07:47:49,270
to.dev and you'll you'll see our our

549
07:47:49,280 --> 07:47:51,270
preview spinning up and and so on, but

550
07:47:51,280 --> 07:47:53,990
you'll get a very simple and um a nice

551
07:47:54,000 --> 07:47:55,350
interface and you won't have to care

552
07:47:55,360 --> 07:47:57,670
about all of this stuff. Thank you so

553
07:47:57,680 --> 07:48:00,230
much. It's been pleasure. Um, and uh,

554
07:48:00,240 --> 07:48:01,430
yeah, hope you hope you learned

555
07:48:01,440 --> 07:48:04,000
something. Thanks so much. [applause]