Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked

Read the talk

Your Agents Lack Context: How to Fix “You’re Absolutely Right!”

Brandon Waselnuk of Unblocked explains why capable coding agents still produce locally plausible, unmergeable work—and how identity-aware retrieval, conflict resolution, permissions, and structured queries can give them the organizational context they lack.

From a talk by Brandon Waselnuk

At a glance

Ideas worth remembering

  • Agent errors become more expensive as teams move from autocomplete toward parallel and background execution, because misunderstandings survive longer before a human catches them.

  • Static Markdown and MCP connections are useful, but maintenance, tool selection, and satisfaction-of-search failures keep them from supplying organizational understanding by themselves.

  • A context engine must combine cross-system retrieval with identity-aware relevance, conflict resolution, permission enforcement, and token-efficient delivery.

  • Semantic retrieval and structured querying solve different problems: RAG finds related material, while deterministic queries handle identities, relationships, statuses, and time ranges.

  • The reported 21-million-versus-10.8-million-token comparison illustrates the potential cost of repeated discovery, but it is a single speaker-reported task rather than a general benchmark.

The agent starts as a brilliant new hire

AI-generated code should feel like it came from someone who has worked on your team for years. That standard is demanding because experienced engineers have absorbed far more than source code. Rejected pull requests taught them local conventions. Meetings supplied product history. An overnight production incident connected a rollout procedure to its consequences. In effect, the engineer’s brain became a context engine through repeated exposure to how this company actually works. 1:13

Recording frame at 96 secondsOpen full source frame
Recording frame at 96 seconds

A new terminal session has none of that history. The model may be intelligent, but it does not know how this organization ships, which architectural decisions still apply, or what happened the last time a similar change reached production. If that missing context sends the agent in the wrong direction, every later action builds on a bad starting point. 1:51

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:12 · section reference included

Bad context gets more expensive as autonomy increases

Autocomplete kept the cost of a context failure small. A suggestion appeared, an engineer judged it using local knowledge, and either pressed Tab or rejected it. Moving toward agentic IDEs, parallel workers, and background agents removes that immediate filter. The agent must recognize when it has reached a wall, ask the right questions, and produce code that fits a revenue-bearing brownfield system—not merely a greenfield demo. 2:17

Recording frame at 142 secondsOpen full source frame
Recording frame at 142 seconds

What changes as the human moves farther out of the loop? The diagram follows the accumulating cost. A wrong autocomplete suggestion is discarded immediately; a longer agent run spends search tokens and rework on repeated corrections; parallel agents create more output to review; and a background agent needs machine-accessible context if it is expected to continue without asking a person. The relationship to notice is compounding: greater autonomy magnifies an early context error rather than merely repeating it. 3:17

The familiar failure is the correction loop: the agent announces that it finished, the engineer rejects the result, and several rounds of repair follow. Those rounds consume search tokens and human time. Parallel agents add a review tax because reviewers also need business logic and operating context to distinguish a plausible patch from a mergeable one. Fully unattended work raises the requirement again: the context source must remain queryable while the agent operates. 3:17

The compounding cost of missing context

Autocompletehuman-in-loop — A human immediately accepts or rejects one suggestion.
Agent correction loopsupervised-agent — Wrong assumptions generate repeated search and rework.
Parallel agentsscaled-agents — More generated work creates a review tax.
Background agentsautonomous — Unattended workers need context they can query themselves.
As human supervision decreases, an early misunderstanding survives longer and creates more search, rework, and review.
Read the diagram as text
  • Autocomplete. human-in-loop — A human immediately accepts or rejects one suggestion.
  • Agent correction loop. supervised-agent — Wrong assumptions generate repeated search and rework.
  • Parallel agents. scaled-agents — More generated work creates a review tax.
  • Background agents. autonomous — Unattended workers need context they can query themselves.
  • Autocomplete → Agent correction loop: Less immediate filtering.
  • Agent correction loop → Parallel agents: More work reaches review.
  • Parallel agents → Background agents: Human leaves the loop.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:17 · section reference included

Two useful approaches that plateau

Static project instructions can improve an agent, but they create what Waselnuk calls the curated context trap. A team writes Markdown files, gives agents a filesystem to search, and sees better results. Then the operational problems begin: the files must be distributed, they age like other documentation, and someone must decide which version of the organization’s practices deserves to be authoritative for everyone. 4:11

Recording frame at 360 secondsOpen full source frame
Recording frame at 360 seconds

MCP access solves a different problem: it gives an agent a path to another system. It does not ensure that the agent calls the tool, because tool and server descriptions influence selection. Even when the call happens, the agent may stop at the first plausible result. Waselnuk describes this as satisfaction of search: an architecture record says to use approach B, so the agent proceeds without finding last night’s Slack conversation saying the team has switched to A. 5:11

Access is therefore not understanding. A patch can compile while remaining operationally wrong because the agent cannot see what lies “below the waterline”: rollout procedures, feature-flag sequencing, incident history, and current decisions that have not reached an architecture document. The concrete consequence is code that passes a local technical threshold and still causes a production incident. 5:43

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:11 · section reference included

What a context engine must do before returning an answer

Consider the request, “I want to get auth stood up.” The words alone do not identify the right repositories, conventions, or experts. A context engine can begin with the requester: where they commit, which code they touch, and who reviews their work. That identity narrows the initial search, and the resulting relationships provide pivots into the rest of the organization’s information. 6:12

Recording frame at 462 secondsOpen full source frame
Recording frame at 462 seconds

The engine then has to perform several distinct jobs:

  • Unified system context: Search across the relevant systems instead of treating one repository or document store as the whole organization.
  • Targeted retrieval: Return a linked document quickly when the task is narrow, while allowing deeper research when discovery is necessary.
  • Conflict resolution: Detect that an old architecture diagram and a recent executive Slack thread disagree, then determine which evidence should guide the task rather than silently choosing the first result.
  • Personalized relevance: Use who is asking, where they work, and what they are working on to focus retrieval.
  • Token optimization: Compress the research into useful context instead of sending every discovered artifact to the model.
  • Permission enforcement: Preserve OAuth, SSO, and source permissions so information from a restricted project does not leak into another user’s answer. 6:42

Permission enforcement must happen during retrieval, not after an unrestricted answer has already been assembled. If the requester cannot access “secret project A,” material from that project must not enter the response. Human-facing answers and machine-to-machine responses also need different packaging: a person in Slack may benefit from explanation, while an agent needs a compact packet that avoids spending tokens on presentation. 6:42

What data movement turns scattered workplace records into task-specific context? The engine ingests engineering sources, including real-time incident-management signals, reasons across them using the six capabilities, and emits an answer shaped for the destination workflow. The diagram makes the critical middle layer visible: connecting sources directly to an agent skips identity, conflict, permission, and compression decisions—the very work that turns reachable information into usable context. 7:19

From organizational data to workflow-specific context

Organizational sourcesinput — Engineering data and real-time operational signals enter from workplace systems.
Identity and relevanceengine — The requester’s work, commits, and relationships focus the search.
Retrieval and conflict resolutionengine — The engine supports fast lookup or deeper research and reconciles contradictory evidence.
Permission filteringengine — The requester’s access limits what may enter the response.
Token-optimized contextengine — Research is compressed and formatted for the destination.
Human or agent workflowoutput — The answer reaches Slack, a coding agent, or another task-specific interface.
The engine does more than retrieve: it interprets, filters, compresses, and shapes evidence before delivery.
Read the diagram as text
  • Organizational sources. input — Engineering data and real-time operational signals enter from workplace systems.
  • Identity and relevance. engine — The requester’s work, commits, and relationships focus the search.
  • Retrieval and conflict resolution. engine — The engine supports fast lookup or deeper research and reconciles contradictory evidence.
  • Permission filtering. engine — The requester’s access limits what may enter the response.
  • Token-optimized context. engine — Research is compressed and formatted for the destination.
  • Human or agent workflow. output — The answer reaches Slack, a coding agent, or another task-specific interface.
  • Organizational sources → Identity and relevance: Ingest and focus.
  • Identity and relevance → Retrieval and conflict resolution: Narrow the territory.
  • Retrieval and conflict resolution → Permission filtering: Filter allowable evidence.
  • Permission filtering → Token-optimized context: Shape a safe response.
  • Token-optimized context → Human or agent workflow: Deliver in the required form.
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:12 · section reference included

The same prompt, with less repeated discovery

Unblocked ran the same prompt against the same model twice, once with context and once without. The run without the engine used about 21 million tokens; the context-assisted run used 10.8 million and finished about two hours sooner. Waselnuk attributes much of the difference to removing repeated exploratory greps at the start of agent sessions: the assisted run began “hydrated” with relevant organizational context instead of rediscovering it. 9:19

Recording frame at 574 secondsOpen full source frame
Recording frame at 574 seconds

This is a speaker-reported comparison of one sizable task; the recording does not supply enough task, harness, or evaluation detail to treat the numbers as a general benchmark. Its useful mechanism is narrower: if multiple sessions repeatedly search for the same architecture, ownership, and business rules, supplying that context up front can remove duplicated search work and leave more of the token budget for the requested change. 9:19

The reported outcome was roughly 50% fewer tokens, faster triage, and better answers because the model had business context. The token result is easiest to explain; the recording asserts the answer-quality improvement but does not provide an independent quality measure. 9:49

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:49 · section reference included

Three open-source components expose the underlying techniques

The first tool maps engineering relationships from GitHub activity. It identifies what a person commits, where they commit, and who reviews their work, then distills that activity into an experts graph. Team labels can optionally be added with an OpenAI or Anthropic API key, but the underlying relationship construction is deterministic. This graph gives the earlier auth request a concrete starting point: locate the requester, follow their collaboration and expertise links, and focus research on the parts of the organization where the request is likely to belong. 10:19

The second tool, the repo rules agent, discovers rules files across a repository, checks the extracted instructions and severities, and flags duplicates or other problems. The resulting index is queryable, so an agent can retrieve the rules relevant to its current work instead of loading every instruction file or searching several overlapping conventions independently. 10:49

The third tool demonstrates why semantic retrieval is only part of a context engine. Ask, “What are the open PRs that I worked on in the last week with authentication?” Similarity search alone cannot reliably apply the identity, status, time, and topic constraints. The workshop builds a relational query engine in six stacked pull requests: the agent discovers a schema, writes a query, and runs that query deterministically to retrieve structured records. RAG remains useful for finding semantically related documents; structured queries answer questions whose meaning depends on filters and relationships. 11:19

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:49 · section reference included

Context becomes shared business infrastructure

The same engine can support work outside code generation. Waselnuk describes customer-success teams using context to address tickets as they arrive and salespeople querying organizational information in the field. These workflows differ in their actions, but they need the same underlying abilities: retrieve current evidence, interpret it for the requester, and return it without violating permissions. 12:17

Recording frame at 751 secondsOpen full source frame
Recording frame at 751 seconds

Unblocked also built a readiness tool that asks questions about a team’s current use of AI and maps it onto the earlier adoption curve, then suggests ways to move toward more scalable agent use. That closes the loop with the opening warning: adding parallel or background agents before giving them organizational context multiplies correction and review costs. 12:48

The closing claim is blunt: the important gap is no longer intelligence but context. Within this talk’s scope, that means surrounding capable models with current organizational knowledge, relationships, permissions, conflict handling, and economical delivery. Better models can improve implementation, but they do not automatically inherit the years of rejected PRs, meetings, incidents, and local judgment that made an experienced teammate effective. 13:11

Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:17 · section reference included

Resources

From the talk

  • Builds a developer collaboration and expertise graph from GitHub pull-request and Git history. Useful for grounding personalized retrieval in reviewers, teams, code areas, and recent working relationships.

  • Discovers coding-instruction files, extracts and indexes their rules, merges near-duplicates, flags potential conflicts, and supports task-scoped queries instead of injecting every rule into every prompt.

  • A six-step workshop for turning natural-language questions into validated MongoDB aggregation pipelines over GitHub PR and issue data, including schema discovery, identity resolution, validation, and retry.

Transcript & slides

Loading transcript and slides…

Transcript download