1
01:58:38,880 --> 01:58:40,950
and Vlad. We're excited to share our

2
01:58:40,960 --> 01:58:43,109
journey as well as our learnings around

3
01:58:43,119 --> 01:58:46,310
building an AI on call teammate at Meta

4
01:58:46,320 --> 01:58:48,790
that now runs real on call rotations

5
01:58:48,800 --> 01:58:51,589
every day and around the clock.

6
01:58:51,599 --> 01:58:53,510
We are part of the engineering team that

7
01:58:53,520 --> 01:58:55,430
builds the tooling platform and

8
01:58:55,440 --> 01:58:58,709
infrastructure for incident response on

9
01:58:58,719 --> 01:59:01,830
call experience and mitigations at meta.

10
01:59:01,840 --> 01:59:04,149
That space is evolving rapidly with the

11
01:59:04,159 --> 01:59:07,350
increase in AI capabilities.

12
01:59:07,360 --> 01:59:10,390
AI changed how fast we write code. About

13
01:59:10,400 --> 01:59:13,430
3/4 of new code is now written with AI.

14
01:59:13,440 --> 01:59:16,310
And more AI generated changes reach

15
01:59:16,320 --> 01:59:19,589
production every single day. And some of

16
01:59:19,599 --> 01:59:22,790
it can cause production failures just

17
01:59:22,800 --> 01:59:25,669
like any other code change can.

18
01:59:25,679 --> 01:59:28,550
But on call did not speed up with it.

19
01:59:28,560 --> 01:59:31,189
Highquality code can pass review and

20
01:59:31,199 --> 01:59:33,270
still break in production. and when it

21
01:59:33,280 --> 01:59:36,950
does, someone still gets paged at 2 a.m.

22
01:59:36,960 --> 01:59:38,950
The bottleneck moved from writing code

23
01:59:38,960 --> 01:59:42,550
to operating it in production. And on

24
01:59:42,560 --> 01:59:44,390
call is where that operational load

25
01:59:44,400 --> 01:59:47,990
lands. So we asked if AI is writing the

26
01:59:48,000 --> 01:59:53,830
code, can AI also help operate it?

27
01:59:53,840 --> 01:59:57,189
And we think it can. Now let's quickly

28
01:59:57,199 --> 01:59:59,430
run through one of those on call pages

29
01:59:59,440 --> 02:00:01,990
with a simplified example.

30
02:00:02,000 --> 02:00:05,910
It's 2:00 a.m. and 12 alerts are firing.

31
02:00:05,920 --> 02:00:07,589
Same night, it can go two different

32
02:00:07,599 --> 02:00:09,910
ways. On the left is how most of us do

33
02:00:09,920 --> 02:00:12,470
this today. And notice we already have

34
02:00:12,480 --> 02:00:15,270
good AI. First, you sort the alerts by

35
02:00:15,280 --> 02:00:17,990
yourself. Checkout, payments, disk,

36
02:00:18,000 --> 02:00:19,669
which one to pick first. Let's say you

37
02:00:19,679 --> 02:00:22,229
pick checkout. Then you ask your

38
02:00:22,239 --> 02:00:25,990
assistant why P99 latency is high.

39
02:00:26,000 --> 02:00:27,750
It's genuinely useful. It pulls the

40
02:00:27,760 --> 02:00:31,350
metrics and finds the deployment at 149.

41
02:00:31,360 --> 02:00:33,990
And when you ask, it reads what changed

42
02:00:34,000 --> 02:00:36,310
and checks the runbook.

43
02:00:36,320 --> 02:00:38,629
It figured out that the time config was

44
02:00:38,639 --> 02:00:41,750
changed to a lower value. And based on

45
02:00:41,760 --> 02:00:44,070
the runbook, it suggests to roll back,

46
02:00:44,080 --> 02:00:47,189
which is all very helpful. However, if

47
02:00:47,199 --> 02:00:49,189
you look at the shape of it, it waited

48
02:00:49,199 --> 02:00:51,270
to be asked.

49
02:00:51,280 --> 02:00:54,550
You picked every next step. You were the

50
02:00:54,560 --> 02:00:58,149
glue between the questions and once it

51
02:00:58,159 --> 02:01:00,950
answered, it went dormant.

52
02:01:00,960 --> 02:01:05,189
At 239, you know the cause, but the roll

53
02:01:05,199 --> 02:01:07,750
back, the changes, and the follow-ups

54
02:01:07,760 --> 02:01:10,070
are still yours.

55
02:01:10,080 --> 02:01:14,310
Now, on the right, the same 12 alerts,

56
02:01:14,320 --> 02:01:17,510
but an AI teammate started before you

57
02:01:17,520 --> 02:01:21,430
woke up. It had already ranked them.

58
02:01:21,440 --> 02:01:24,310
Checkouts first, payments merged in as

59
02:01:24,320 --> 02:01:26,790
the same cause, and the other 10 handled

60
02:01:26,800 --> 02:01:28,470
separately.

61
02:01:28,480 --> 02:01:33,510
It read the dashboards, difficed

62
02:01:33,520 --> 02:01:35,910
two past incidents.

63
02:01:35,920 --> 02:01:39,109
You get a hypothesis, a 30-cond way to

64
02:01:39,119 --> 02:01:42,709
verify it, and a fix waiting for your

65
02:01:42,719 --> 02:01:45,990
review. At 2011, you verified it and

66
02:01:46,000 --> 02:01:49,030
you're back to sleep. That gap is what

67
02:01:49,040 --> 02:01:53,109
this talk is about.

68
02:01:53,119 --> 02:01:56,390
So why were you the glue?

69
02:01:56,400 --> 02:01:58,790
Every team was bolting automation onto

70
02:01:58,800 --> 02:02:03,109
on call one silo at a time. An alert

71
02:02:03,119 --> 02:02:05,589
script for alerts, a triage bot for

72
02:02:05,599 --> 02:02:08,470
tickets, a deploy bot for deploys, a fix

73
02:02:08,480 --> 02:02:10,709
it agent for incidents.

74
02:02:10,719 --> 02:02:13,589
Each one solves its slice, keeps its own

75
02:02:13,599 --> 02:02:16,629
context, and forgets it. Now replaying

76
02:02:16,639 --> 02:02:20,390
your 2 AM page, the alert script saw P99

77
02:02:20,400 --> 02:02:23,589
latency above 2 seconds. The triage bot

78
02:02:23,599 --> 02:02:25,109
saw three tickets saying checkout is

79
02:02:25,119 --> 02:02:28,229
slow. The deploy bot knew about a config

80
02:02:28,239 --> 02:02:30,870
change at 149 and the incident history

81
02:02:30,880 --> 02:02:33,830
had two similar past incidents. Every

82
02:02:33,840 --> 02:02:35,750
clue existed,

83
02:02:35,760 --> 02:02:39,350
but nothing connected them. So the on

84
02:02:39,360 --> 02:02:42,870
call became the glue exactly

85
02:02:42,880 --> 02:02:44,950
as we saw on the left side of the

86
02:02:44,960 --> 02:02:46,790
previous slide.

87
02:02:46,800 --> 02:02:50,950
Our bet was one AI teammate per rotation

88
02:02:50,960 --> 02:02:52,870
instead.

89
02:02:52,880 --> 02:02:55,510
One loop across every signal, one team

90
02:02:55,520 --> 02:02:58,390
memory that learns every shift, its own

91
02:02:58,400 --> 02:03:01,270
identity so it acts as itself and a

92
02:03:01,280 --> 02:03:03,830
human gate so you approve instead of

93
02:03:03,840 --> 02:03:07,030
operate. But giving one agent all that

94
02:03:07,040 --> 02:03:09,910
context and reach raises the stakes.

95
02:03:09,920 --> 02:03:15,030
Which is why autonomy has to be earned.

96
02:03:15,040 --> 02:03:18,070
Under the hood, the teammate or the AI

97
02:03:18,080 --> 02:03:21,430
teammate is the simplest agent. There is

98
02:03:21,440 --> 02:03:24,790
model using tools in a loop acting on an

99
02:03:24,800 --> 02:03:27,589
environment and reading the feedback.

100
02:03:27,599 --> 02:03:29,450
Everything else in this talk is what we

101
02:03:29,460 --> 02:03:32,229
[music] build around that loop to make

102
02:03:32,239 --> 02:03:35,109
it safe to leave it running. And for

103
02:03:35,119 --> 02:03:37,430
that I will hand over to Swapnikai and

104
02:03:37,440 --> 02:03:40,229
Vlad next to get into the actual details

105
02:03:40,239 --> 02:03:43,109
of how it works.

106
02:03:43,119 --> 02:03:45,109
To achieve this we started with an

107
02:03:45,119 --> 02:03:47,350
autonomy ladder. Think about how you

108
02:03:47,360 --> 02:03:50,629
onboard a new engineer to on call. First

109
02:03:50,639 --> 02:03:53,430
the shadow. The investigators suggest

110
02:03:53,440 --> 02:03:56,390
but they don't touch production. Then

111
02:03:56,400 --> 02:03:59,030
the pair the right to fix and another

112
02:03:59,040 --> 02:04:01,669
engineer approves it. With time they

113
02:04:01,679 --> 02:04:04,310
gain knowledge. We give our agent

114
02:04:04,320 --> 02:04:07,109
exactly the same ladder. At level one,

115
02:04:07,119 --> 02:04:09,109
it investigates and tells you what it

116
02:04:09,119 --> 02:04:12,149
found. It changes nothing. At level two,

117
02:04:12,159 --> 02:04:14,470
it acts with approval. It drafts the

118
02:04:14,480 --> 02:04:16,550
fix, but nothing reaches production

119
02:04:16,560 --> 02:04:19,910
until a human approves. At level three,

120
02:04:19,920 --> 02:04:22,310
it clears lower criticality work on its

121
02:04:22,320 --> 02:04:24,709
own and only interrupts you for what

122
02:04:24,719 --> 02:04:27,589
matters. High criticality work stays

123
02:04:27,599 --> 02:04:29,510
with humans.

124
02:04:29,520 --> 02:04:32,790
So what earns the next run? We use a

125
02:04:32,800 --> 02:04:35,669
simple test. How costly is a mistake and

126
02:04:35,679 --> 02:04:38,550
how easy is it to catch? So an action

127
02:04:38,560 --> 02:04:41,510
climbs only when mistakes are cheap.

128
02:04:41,520 --> 02:04:45,270
They're caught by tests, CI or review.

129
02:04:45,280 --> 02:04:47,510
and the agent already has a track record

130
02:04:47,520 --> 02:04:50,709
on that exact action. That's why the

131
02:04:50,719 --> 02:04:53,350
agent can close a noisy alert on its own

132
02:04:53,360 --> 02:04:55,270
while a production mitigation still

133
02:04:55,280 --> 02:04:58,470
waits for a human.

134
02:04:58,480 --> 02:05:00,709
At level one, the investigation is

135
02:05:00,719 --> 02:05:02,950
layered and stops at the first confident

136
02:05:02,960 --> 02:05:05,669
answer. The agent investigated the issue

137
02:05:05,679 --> 02:05:08,390
using the team's memory and runbook. It

138
02:05:08,400 --> 02:05:10,709
also shared the root cause analysis, but

139
02:05:10,719 --> 02:05:13,750
human had to apply the fix.

140
02:05:13,760 --> 02:05:17,189
At level two, it writes the fix with the

141
02:05:17,199 --> 02:05:19,990
root cause and a test plan. It answers

142
02:05:20,000 --> 02:05:23,270
reviewer's comments and fixes CI, but

143
02:05:23,280 --> 02:05:25,350
nothing reaches production until a human

144
02:05:25,360 --> 02:05:26,950
approves.

145
02:05:26,960 --> 02:05:30,310
Most drafts don't land, and that's fine.

146
02:05:30,320 --> 02:05:33,669
Review is the filter. At both levels,

147
02:05:33,679 --> 02:05:36,550
the agent does the homework, but you

148
02:05:36,560 --> 02:05:39,350
still had to grade every single paper.

149
02:05:39,360 --> 02:05:42,390
The help got much better, but the tasks

150
02:05:42,400 --> 02:05:44,470
requiring human input didn't get

151
02:05:44,480 --> 02:05:50,229
smaller. Level three shrinks the pile.

152
02:05:50,239 --> 02:05:53,830
Look at this example. 23 items came in,

153
02:05:53,840 --> 02:05:57,350
but only three need you. On the right,

154
02:05:57,360 --> 02:05:59,910
the agent handled the long tail, noisy

155
02:05:59,920 --> 02:06:02,870
alerts, duplicates, routine fixes that

156
02:06:02,880 --> 02:06:05,910
went through normal review and CI. Two

157
02:06:05,920 --> 02:06:08,470
more are still being worked. You don't

158
02:06:08,480 --> 02:06:11,270
have to drive each one to completion. In

159
02:06:11,280 --> 02:06:13,830
the middle is your much smaller decision

160
02:06:13,840 --> 02:06:17,030
queue. Each item makes clear what it

161
02:06:17,040 --> 02:06:19,750
needs from you with a run this button

162
02:06:19,760 --> 02:06:22,470
that has a call to action. And remember

163
02:06:22,480 --> 02:06:25,109
our 2 a.m. check out page that still

164
02:06:25,119 --> 02:06:27,669
stays human owned. The agent brings the

165
02:06:27,679 --> 02:06:30,069
investigation. The human owns the

166
02:06:30,079 --> 02:06:32,709
critical decision. It's the new high

167
02:06:32,719 --> 02:06:35,189
ladder again. Trusted to handle routine

168
02:06:35,199 --> 02:06:37,109
work, ready with the homework when it

169
02:06:37,119 --> 02:06:40,310
needs help. Not just less work per item,

170
02:06:40,320 --> 02:06:45,189
but fewer items waiting on you.

171
02:06:45,199 --> 02:06:47,830
That board is where the work lives. But

172
02:06:47,840 --> 02:06:50,069
nobody stares at a board all the time,

173
02:06:50,079 --> 02:06:52,950
right? So the agent is also available

174
02:06:52,960 --> 02:06:55,350
across many surfaces like chat and

175
02:06:55,360 --> 02:06:58,790
native incident surfaces. At 9 every

176
02:06:58,800 --> 02:07:02,870
morning, it posts a digest. 23 came in,

177
02:07:02,880 --> 02:07:06,390
three need you, 18 handled. Each thing

178
02:07:06,400 --> 02:07:08,790
that needs you come with a one-click

179
02:07:08,800 --> 02:07:12,149
next step. So approving the quota bump

180
02:07:12,159 --> 02:07:15,430
takes one tap from your phone. The 18 it

181
02:07:15,440 --> 02:07:18,470
handled are summarized, not listed. And

182
02:07:18,480 --> 02:07:20,550
the two that are still in progress tell

183
02:07:20,560 --> 02:07:23,589
you what it's doing about them. And when

184
02:07:23,599 --> 02:07:25,910
you want to dig in, you just reply in

185
02:07:25,920 --> 02:07:28,629
the thread. As you see on the right

186
02:07:28,639 --> 02:07:30,870
here, you open the checkout incident and

187
02:07:30,880 --> 02:07:34,229
ask for the evidence. It already has it.

188
02:07:34,239 --> 02:07:37,669
Errors start at 8:41, 2 minutes after a

189
02:07:37,679 --> 02:07:39,430
config push cut the payment client

190
02:07:39,440 --> 02:07:41,830
timeout. The same pattern as two past

191
02:07:41,840 --> 02:07:44,790
incidents. You ask a follow-up, is it

192
02:07:44,800 --> 02:07:46,790
only EU? And it answers with a

193
02:07:46,800 --> 02:07:49,750
breakdown. It drafts the roll back. You

194
02:07:49,760 --> 02:07:52,229
approve it right there and errors start

195
02:07:52,239 --> 02:07:54,709
recovering. You never opened a dashboard

196
02:07:54,719 --> 02:07:57,189
and you never had to explain the context

197
02:07:57,199 --> 02:07:59,189
because it was already there. That's

198
02:07:59,199 --> 02:08:01,189
what a teammate feels like. A morning

199
02:08:01,199 --> 02:08:06,470
hand off a colleague you can ask.

200
02:08:06,480 --> 02:08:09,430
Now under the hood of this AI teammate,

201
02:08:09,440 --> 02:08:12,310
it's AI automations. The first

202
02:08:12,320 --> 02:08:15,430
investigates on every new signal auto

203
02:08:15,440 --> 02:08:18,229
triage classifies it, checks its

204
02:08:18,239 --> 02:08:20,069
criticality and kicks off the right

205
02:08:20,079 --> 02:08:23,189
agent. An alert goes to the incident

206
02:08:23,199 --> 02:08:25,910
investigator, a task to the team's own

207
02:08:25,920 --> 02:08:29,350
agent, a test failure to a coding agent,

208
02:08:29,360 --> 02:08:32,870
a look up to memory and code search.

209
02:08:32,880 --> 02:08:35,910
The second keeps the system healthy.

210
02:08:35,920 --> 02:08:39,189
Unattended automation fails quietly. So

211
02:08:39,199 --> 02:08:41,589
on a schedule, it looks for stuck or

212
02:08:41,599 --> 02:08:44,870
failed runs, retrace and repairs broken

213
02:08:44,880 --> 02:08:47,910
schedules, and it only pages a human if

214
02:08:47,920 --> 02:08:50,229
it can't fix it itself.

215
02:08:50,239 --> 02:08:53,270
Nothing stalls silently.

216
02:08:53,280 --> 02:08:56,069
The third hands off at every shift

217
02:08:56,079 --> 02:08:58,390
change and every morning at 9, it

218
02:08:58,400 --> 02:09:00,950
summarizes the shift. What needs you

219
02:09:00,960 --> 02:09:03,109
with the fix ready and what it handled

220
02:09:03,119 --> 02:09:06,149
on its own posted to the team chat. Not

221
02:09:06,159 --> 02:09:08,069
only in the morning, but every urgent

222
02:09:08,079 --> 02:09:10,390
issue that needs user attention gets

223
02:09:10,400 --> 02:09:13,030
notified in chat with findings and draft

224
02:09:13,040 --> 02:09:15,910
fixes. It also tags the next on call in

225
02:09:15,920 --> 02:09:18,390
the chat during handoff. The entire

226
02:09:18,400 --> 02:09:23,189
cycle still continues

227
02:09:23,199 --> 02:09:25,990
and because it belongs to the team, it

228
02:09:26,000 --> 02:09:28,390
runs the agents and the skills the team

229
02:09:28,400 --> 02:09:30,950
already trusts. That's the slot in the

230
02:09:30,960 --> 02:09:33,510
middle. Bring your own agent, add your

231
02:09:33,520 --> 02:09:36,310
own skills. If a team already has an

232
02:09:36,320 --> 02:09:38,550
agent that has the team's runbooks and

233
02:09:38,560 --> 02:09:40,550
the knowledge base, the automations

234
02:09:40,560 --> 02:09:43,350
leverage it. Alongside that, it has our

235
02:09:43,360 --> 02:09:45,830
own incident investigator and a deep

236
02:09:45,840 --> 02:09:48,310
dive coding session.

237
02:09:48,320 --> 02:09:51,750
Work comes in from alerts, tickets,

238
02:09:51,760 --> 02:09:55,109
incidents, team posts, and chat. The

239
02:09:55,119 --> 02:09:57,750
agent itself is generic. What makes it

240
02:09:57,760 --> 02:09:59,830
good at an on call rotation is the

241
02:09:59,840 --> 02:10:02,950
team's agents skills, memory, and

242
02:10:02,960 --> 02:10:05,830
playbook. And every run is logged, so

243
02:10:05,840 --> 02:10:07,990
anyone on the team can inspect the

244
02:10:08,000 --> 02:10:11,189
trajectory or resume the conversation.

245
02:10:11,199 --> 02:10:13,830
Now, I'm handing off to my teammate Vlad

246
02:10:13,840 --> 02:10:19,109
to talk more on the shared agent.

247
02:10:19,119 --> 02:10:21,750
An uncle agent can't belong to just a

248
02:10:21,760 --> 02:10:23,910
single person because the uncle itself

249
02:10:23,920 --> 02:10:26,790
doesn't. So it belongs to the team.

250
02:10:26,800 --> 02:10:29,270
There's one teammate per rotation on one

251
02:10:29,280 --> 02:10:31,990
board. Wherever's on call can ask it to

252
02:10:32,000 --> 02:10:33,589
correct it or pick up its work mid

253
02:10:33,599 --> 02:10:36,069
investigation. It carries the team's

254
02:10:36,079 --> 02:10:38,310
uncle expertise, the playbook, lord

255
02:10:38,320 --> 02:10:40,550
profiles and past decisions from one

256
02:10:40,560 --> 02:10:42,629
shift to the next one the way a good

257
02:10:42,639 --> 02:10:45,430
handoff should. Here's what that looks

258
02:10:45,440 --> 02:10:48,950
like. Monday at 2 am, Sam is on call and

259
02:10:48,960 --> 02:10:51,669
tells it remember if checkout time is

260
02:10:51,679 --> 02:10:53,910
out after a config push, roll back the

261
02:10:53,920 --> 02:10:57,910
config first. It saves it to the memory.

262
02:10:57,920 --> 02:11:00,149
Three days later, Jordan is on call hits

263
02:11:00,159 --> 02:11:03,350
the exact same alert and just asks for a

264
02:11:03,360 --> 02:11:06,870
fix. The agent answers with some fix,

265
02:11:06,880 --> 02:11:08,790
says where it came from, and has the

266
02:11:08,800 --> 02:11:11,510
roll back drafted. Teach it once, the

267
02:11:11,520 --> 02:11:14,149
next phone call gets it.

268
02:11:14,159 --> 02:11:17,589
And it access the team its own identity

269
02:11:17,599 --> 02:11:19,589
owned by the rotation with only the

270
02:11:19,599 --> 02:11:21,270
access the team is comfortable granting

271
02:11:21,280 --> 02:11:23,990
it.

272
02:11:24,000 --> 02:11:26,950
And it learns from every uncle shift.

273
02:11:26,960 --> 02:11:29,430
Handoffs, decisions, and corrections all

274
02:11:29,440 --> 02:11:32,310
flow into memory. So ship number 30 is

275
02:11:32,320 --> 02:11:34,310
better than the ship number one. The

276
02:11:34,320 --> 02:11:36,149
same way a new uncle gets better every

277
02:11:36,159 --> 02:11:40,069
week. But memory erodess over time. So

278
02:11:40,079 --> 02:11:43,669
ours has to earn its place. It retrieves

279
02:11:43,679 --> 02:11:46,470
semantically uses what it finds, scores

280
02:11:46,480 --> 02:11:49,510
where it actually helped and promotes or

281
02:11:49,520 --> 02:11:52,069
retires entries automatically.

282
02:11:52,079 --> 02:11:54,629
When someone says that was wrong, the

283
02:11:54,639 --> 02:11:56,629
item reopens and the correction becomes

284
02:11:56,639 --> 02:11:59,350
memory.

285
02:11:59,360 --> 02:12:01,350
Three things made this work beyond our

286
02:12:01,360 --> 02:12:02,950
team.

287
02:12:02,960 --> 02:12:05,030
First, a loop that never stops

288
02:12:05,040 --> 02:12:07,510
listening. Every piece of feedback or

289
02:12:07,520 --> 02:12:09,830
course correction reopens the item and

290
02:12:09,840 --> 02:12:12,790
ultimately gets absorbed into memory.

291
02:12:12,800 --> 02:12:15,030
Feedback is an input to the system, not

292
02:12:15,040 --> 02:12:18,709
a story read once a quarter. Second,

293
02:12:18,719 --> 02:12:22,149
customize instead of forking. Teams

294
02:12:22,159 --> 02:12:23,990
bring their own playbooks and skill

295
02:12:24,000 --> 02:12:26,069
workflows, and it's still the same

296
02:12:26,079 --> 02:12:28,550
teammate underneath.

297
02:12:28,560 --> 02:12:32,470
Third, what we call one rotation to end.

298
02:12:32,480 --> 02:12:34,950
Identity, config, and memory are per

299
02:12:34,960 --> 02:12:37,189
team. So, add a rotation becomes a

300
02:12:37,199 --> 02:12:39,510
setup, not a new project.

301
02:12:39,520 --> 02:12:41,750
The agent is generic while the expertise

302
02:12:41,760 --> 02:12:45,510
is owned by the team.

303
02:12:45,520 --> 02:12:47,510
So what has changed for the people on

304
02:12:47,520 --> 02:12:50,310
call? Well, first their job shifted from

305
02:12:50,320 --> 02:12:52,149
driving deep investigations to making

306
02:12:52,159 --> 02:12:54,390
key decisions. They start from a

307
02:12:54,400 --> 02:12:56,790
hypothesis with linked evidence, not a

308
02:12:56,800 --> 02:12:59,189
blank task or dashboard.

309
02:12:59,199 --> 02:13:02,149
Second, the load is getting lighter.

310
02:13:02,159 --> 02:13:04,390
Early signs of on call load reduction

311
02:13:04,400 --> 02:13:06,470
are promising for teams that run it

312
02:13:06,480 --> 02:13:08,870
automatic autonomously and we want to

313
02:13:08,880 --> 02:13:10,629
measure it rigorously before we put a

314
02:13:10,639 --> 02:13:12,629
number on it.

315
02:13:12,639 --> 02:13:14,870
And the third every change that agent

316
02:13:14,880 --> 02:13:18,470
writes still meets a human. Fixes land

317
02:13:18,480 --> 02:13:20,310
only through normal review and

318
02:13:20,320 --> 02:13:22,709
continuous integration. And most drafts

319
02:13:22,719 --> 02:13:25,589
don't land. That's the point. Review is

320
02:13:25,599 --> 02:13:27,350
the filter.

321
02:13:27,360 --> 02:13:29,669
Uncle stops being attacks and becomes a

322
02:13:29,679 --> 02:13:32,709
review cube. Here's

323
02:13:32,719 --> 02:13:35,830
what we're focusing on next. First,

324
02:13:35,840 --> 02:13:37,589
mitigations that get approved, not just

325
02:13:37,599 --> 02:13:40,149
drafted. We great every mitigation that

326
02:13:40,159 --> 02:13:42,709
agent proposes and feed the outcome back

327
02:13:42,719 --> 02:13:44,550
so it earns its next level instead of

328
02:13:44,560 --> 02:13:47,910
being handed it. Second, skills are

329
02:13:47,920 --> 02:13:50,069
written from the chat so a team can turn

330
02:13:50,079 --> 02:13:52,310
its runbooks into skills without writing

331
02:13:52,320 --> 02:13:55,669
any code. And third, measuring the time

332
02:13:55,679 --> 02:13:57,830
we give back properly with a rigorous

333
02:13:57,840 --> 02:13:59,910
study of uncall time saved rather than

334
02:13:59,920 --> 02:14:01,910
usage counts.

335
02:14:01,920 --> 02:14:04,950
And the next rung on the ladder is uh

336
02:14:04,960 --> 02:14:07,350
earning medium criticality autonomy with

337
02:14:07,360 --> 02:14:10,470
evidence. On that, thank you so much for

338
02:14:10,480 --> 02:14:12,470
listening to us and hopefully that was

339
02:14:12,480 --> 02:14:14,000
helpful. Bye.