How to Generate Mergeable Code with a Context Engine — Peter Werry, Unblocked

What we do at Unblocks is we build a

context engine. I just want to do a

quick sound check at the back to make

sure everyone can hear me fine. Can you

guys Yeah, we're good. Awesome. So at a

high level, a context engine delivers

organizational context to both your

human workers and now increasingly your

agents. Okay. So why why is that

important? Before we go too deep on the

mechanics of how a context engine works,

I just want to talk briefly about the

problem.

So, what we're going to do is we're

going to hop into our time machines and

we're going to travel back to the before

times uh before agents and uh discuss a

little bit about what we used to do as

humans uh before agents came into the

picture.

And so for years um you were the context

layer. You um had to go and do things

like this. you had to find things you

were looking for, trolled all over

different data sources, different

discussions taking place. Um, and then

through the codebase of course to try to

build up tribal knowledge and, uh,

throughout time as your code base

progressed, um, you'd be, you know,

fighting incidents and things like that.

And your organization over time builds

up battle scars um from all these all

these different things building code uh

documenting architecture and and dealing

with outages and things like that.

But now um we have a new problem because

uh as we introduce agents to the picture

um they suffer from all of these

challenges except for one thing. Agents

are like new employees.

they reset their knowledge every time

you start a new task. Okay? And so you

can think of an agent like an expert

software engineer um who's a new

employee on boarding for the first time.

Every time they have to rediscover your

code base, how your organization builds

tests um and how they deploy software

with each and every task.

Uh can I just uh put a put a show of

hands for everyone that's seen this

slide before by Vim?

So this is kind of like u this is a good

way to view where people are on what we

call like the AI maturity curve. Um

starting at the the far left uh this is

kind of representative of autocomplete

back in the GBT35 days. You know

remember co-pilot and things like that.

Um and then you know kind of move on to

using cursor. Um and then from there

you're you're think you're talking about

how you can start to solve the context

problem. So some people are building

organizational wikis. Just smile if if

this is kind of um bringing up memories

for you. Um and then you know all these

things are great except that uh how do

you give agents access to this and what

are the compounding problems that the

scaling problems as you move forward

well if you give MCP and skills to your

agents

um to teach them how to navigate and

build context and that's kind of where

uh people are today most people they're

at the sort of stage four to five level

okay

and uh they understand that context is

the bottleneck and they're trying to

build solutions to solve it for their

engineering teams. So looking ahead uh

to all the way to eight with software

factories. This is kind of where the

puck is going. I'm not sure if if folks

were at the keynote this morning, but um

it's it's all about like delivery of

context and unknown and unknowns. And

this becomes increasingly important as

people start thinking about full

automation of agents. they just can't

operate without organizational context.

They get lost.

So, you know, like that's the real

problem. Access to information doesn't

equal understanding. Um I I know that

folks are probably familiar with

claude.md

um and and uh and wiki layouts and all

these things. If you attach a wiki, it

still doesn't tell the agent where the

information is that it needs. It can

search for things in the wiki, but then

what happens is it'll suffer from

something that uh radiologists

uh call satisfaction of search. So, this

is a term in radiology

where you look at an X-ray and you're

trying to find a region um that might be

an indicator for cancer. Okay? And you

discover like one

indicator and if you stop there uh you

might miss other important indicators

that might you know lead to diagnosis of

even more uh issues. So this is what

happens with agents. They don't they

they find something that they they think

is correct and then they stop. Um the

the other thing about agents is that

they don't distill understanding.

They can look around, they can find

information, but they they don't

understand how all the pieces fit

together because without doing that leg

work ahead of time. Um, they don't

understand how, you know, your

dependencies interact with each other

and how your architecture and sort of

future planning is going to scope the

work that it does next. And so some some

people will then ask, well, what if we

just take the entire codebase and all of

our architecture documents and just slam

it into the context window. Um, and then

yes, maybe like your agents will reason

about everything all at once. And in

practice, that that of course doesn't

work. Um, not just because you've got

way more organizational context than can

fit into a context window, even one

that's a million tokens in size. Um, but

it it it causes the agent to get

distracted. When you're working on a

task, you want task specific flow. Um,

and so your agents will get distracted

easily if you give them things that

cause them to look this way in that way.

Um, and it'll just waste tokens and

time. So, in this morning's keynote, um,

Tariq from Claude Code mentioned unknown

unknowns. I just want to uh harp on that

phrase again. And it can be phrased a

different way, which is finding the

things that really matter.

And so this is what your agent can see

at the top of the iceberg. They can see

the code and they can operate on the

code. What they don't see are things

like the actual intent, the team

conventions, past decisions, things that

you've discussed in Slack, for example,

uh architecture rationale, and so on.

And that's why your agents need a

context engine to get real work done.

So, I'm going to now uh attempt a live

demo. And hopefully the demo gods are

kind. Um, so I want to pop back up

conceptually. Oops, I think I'm on the

wrong tab. We'll get to that one in a

sec.

So for now,

sorry about that. And here we are.

So I'm going to ask a question as if I'm

a, you know, I'm a human and I want to

get some information about my codebase.

And, you know, the human layer hasn't

gone away. We talk about agents and

their need for context, but um humans

are still asking questions about the

codebase and we need that level of

understanding because ultimately the

accountability stops with us. When you

hit merge on a PR, you need to

understand what it's doing um and you

need to understand how the architecture

works. So this question I asked here um

is about an internal component of our

system called the source mark engine and

you can see that it uh is able to

articulate it fairly well. um

understands the architecture. This this

diagram here is uh is generated. So it

this diagram doesn't exist. Um it just

figures it out based on the um the way

the code operates today and then some

proposals for future architecture.

And then uh what's really important is

that you show your work. This is a trust

building thing more than anything, but

it allows people to see if um if the

answer is maybe not entirely correct,

then you can in look into the uh the

knowledge base that you have and make

corrections.

Increasingly agents are doing this for

you.

So now um what I want to show you is

another place where humans spend their

time which is in Slack and this is where

a lot of the decisions get made of

course.

So I can do something like this.

And uh unblocked will sit and kind of

listen for things that are things that

can chime in on when it provides a high

degree of Oh, sorry. We went to the

wrong You guys can't see that. Thank

you, Claire.

Oh, come on down. Let's see if I can

bring it up. There we go.

Perfect. So I can ask questions like

this in unblocked and if it thinks it

can chime in on the answer then it will

chime in. Otherwise I can just

um address unblocked directly and ask

the same question

and when it thinks that it has an answer

to give then it will give an answer and

so we can get um

quite a bit of interesting content there

from unblocked. Thank you. Unblocked.

I'm going to switch up

and show you the the really interesting

thing which is the agents. Okay. So, um

in in that question, the source mark

engine, I'm not sure if people picked

up, but there was a little thing at the

bottom there that said, you know,

there's some optimization opportunities.

Um so what I did here is I went into

claw code and I asked it um without

using unblocked to um

uh generate uh a plan to optimize the

source mark calculator and it did that

and it happily went and you know

searched through the code and and tried

to figure out how the algorithm works

and so on. Um and it it reached a

conclusion that's great you know it does

a pretty good job um but you know it

maybe could do a little bit better. So,

I asked that question again uh using

unblock this time and it it really kind

of nails the the nuances because it

picks up on the the uh PRs that we um

where we discussed future possibilities

for improvement. um some Slack

conversations that we had and uh of

course you know notion and architecture

documents and it shows its work and this

is really important because um all of

these things here the sources come back

to Claude and then Claude knows exactly

where to jump to next if it needs to

elaborate on that context. And so I just

want to show you what the impact of that

is. So if I um Whoops.

Thank you. If I pull up usage here, you

can see that with unblocked, uh, the

total cost was, you know, subd dollar to

create the plan. Uh, took about a

minute. Ignore the wall clock time

because I've had this open for about an

hour. But, um, it's about a minute. And

then if I look at um the usage without

unblocked, you can see that it's about 2

minutes. And and and it costs more to

generate all that context. Now, the

reason that happens is because it has to

do more work. It has to look around. has

to discover things. Um, and this

compounds, not only does it have to do

more work to discover things, it doesn't

discover the right things. So, when you

get further down in your execution, it

may be operating on the wrong plan or

the wrong assumptions. And then you have

to go back and you have to loop over and

over again. So, the real value of a

context engine is not like the upfront

cost on these short tasks. It's the

compounding effect. Um the the other

Tariq from Sonar mentioned this in the

keynote this morning and it's true like

the loops compound and you have to be

like um uh efficient the entire way

through with your context. I'm just

going to jump back to

Safari and I'm going to point out um

some really interesting things. So we

also have a a code review agent.

And when we say um you know

organizational context, we're talking

about more than just the underlying

data. Uh we're talking about real

intelligence. So what unblock does is it

looks at um not like it looks at pull

request data and there are other data

sources for this and it generates a

series of best practices that help align

agents to your codebase. But we thought

that this would be really helpful to

surface for the review agent as well. So

what you can see here is um

it unblock chimed in and then Richie

here said, "Oh, that's cool. That's

something I would say." And that's

because that actually was something he

said. So it surfaced the uh the previous

comments. Richie's one of the senior

engineers and we use the sort of

seniority or expertise as a signal um to

boost uh comments that are important.

Okay.

So another uh interesting interaction by

Richie, he uh discovered that the number

of code review issues that were being

surfaced dropped uh precipitously

and he was debugging it with unblocked.

Um he got all the way to the bottom and

realized what roughly what the problem

was and then asked unblocked to fix it.

Now this this is something that we have

internally um you know that we're

experimenting with. Um, so unblocked uh

can run as an agent in the cloud. Um,

but what's really cool about this is

that it has all your organizational

context at its fingertips and the

results are are pretty magical. So it

can do things like generate this PR um,

and then what you'll see here is that

not only does it generate the fix, it

also is able to relate it to the all the

conversations that were happening. So

this PR was created because and you read

that context thing. It's mind-blowing.

After this PR, we switched to uh Claude

48 and it dropped a ton in issues

because of the behavior is quite a bit

different. So then it said Richie

directly correlated the drop. Now what's

this thing here? Let's click on it. It

is a Slack conversation. So, it found

the Slack conversation, correlated all

of that, you know, past history back

again, and then we ended up with a with

a final PR.

So, um, I'm going to I've got only a few

minutes left. I'm just going to close

this out really quickly. We have a uh a

couple of open- source projects that are

kind of interesting if people want to

play with them. One is the document

query engine. That was, uh, something

that I talked about on Monday in my

workshop. Um, I may uh talk about it

again tomorrow, but I just want to give

folks a sense of what this thing does.

Um, whoops.

If you want to play with it, it's open

source, so you can just download it and

have it go. It basically runs over your

um uh GitHub repository, ingests uh your

your historical pull requests, and then

uh synthesizes a schema based on the

documents that it can sample. Um and

then from there you can issue any kind

of queries that you like and get all

kinds of insights out of it through the

agent chat. You can ask all kinds of

questions. Um and then lastly the

engineering social graph. So this is the

thing that I was talking about earlier

that helps us pin down expertise and

team relationships. Um so what you can

see here is this sort of like the rough

breakdown of our team structure at

Unblocked. As you can see we're a fairly

small team. Um and so we've got these um

uh these clusters of people and how they

relate to each other indicates the kind

of um review relationships that they

have. So these are you know these lines

show like we review each other's code.

Um

we can then cluster that and generate

team labels for that or show the

coverage across your codebase. This is

really cool. you can see kind of where

the holes are, where you might be

lacking expert coverage. Um, and that's

exactly what we use within the context

engine itself.

All right,

one last thing we have uh for those that

want a taste of what a context engine

can do but don't want to sign up for

unblocked right away. Um you can use uh

something that we call the context

engine simulator which will basically

build up a context behind the scenes on

a per task basis and then use that

context uh to to drive the task. It'll

do it with context and without context

so that you can see what the differences

might be.

This is a QR code for that if you want

to just take a quick snap.

Awesome. And I'll just land

on a quote from one of our customers.

50% fewer tokens, faster triage, better

answers. And that's exactly what a

context engine can do.

One last shout out um before we end. My

colleague Brandon is giving a talk in

10 minutes uh at room 2020. um he's

going to speak to in a lot more detail

about some of the higher level things

that context engines can do. I'm going

to run over there right after this and I

think all of you should follow me.

Awesome. Oh, and don't forget to get a

coconut.

>> [music]