# Build Your First Demand-Driven Context Base: Let AI Agents Tell You What They Need

Raj Navakoti

AI Engineer Europe 2026

[Talk page](https://ai.engineer/talks/_QAVExf_1uw-build-your-first-demand-driven-context-base) · [Video](https://www.youtube.com/watch?v=_QAVExf_1uw)

## Build a Context Base from the Questions Agents Cannot Answer

Raj Navakoti’s workshop follows missing institutional knowledge from a failed incident investigation to curated context, then tests the approach against maintenance, cost, and expert workload.

Familiarity with coding agents, retrieval-augmented generation, MCP connections, and Git pull requests will help you follow the implementation discussion.

### A capable agent still needs its notebook

[0:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=45s)

IKEA’s Delivery and Services domain spans more than 100 engineers and six product teams, according to staff software engineer Raj Navakoti.  

In *Memento*, a skilled protagonist repeatedly consults notebooks and tattoos to reconstruct what he was doing. Raj Navakoti describes this as a fifteen-minute memory limit and uses it to introduce a familiar agent problem: impressive capability does not supply the context needed to act. An agent may reason, calculate, and generate code well while lacking the institutional knowledge that makes its answer useful inside a particular company.

The tooling has advanced quickly: prompt engineering, retrieval-augmented generation, MCP connections, multi-agent systems, and deep agents. Raj illustrates that speed with a playful Replit example: a full-stack application in ten minutes, supposedly ready by the time the instant noodles are done. The million-dollar-app punch line captures the optimism surrounding prototypes. Enterprise delivery poses a harder question.

![Four-point timeline covering agent definitions, tools and memory, agent teams, and real workflows.](https://ai.engineer/api/talk-essay-media/1fee5b27700a5133041678da233b9ee6c84a015d2512e332f8d625194e0efc13)

AI to agent: from definition debates to execution in real workflows.

If agents can generate applications, review pull requests, and investigate incidents, why are Jira tickets and epics still sitting on the board? Those work items represent business delivery, not just generated output. Raj invokes McKinsey’s adoption-versus-impact gap, reporting that 88% of companies use AI but see only ‘6% of value creation’; the workshop does not establish how that value-creation figure is defined or measured. The useful question is what prevents a capable agent from completing an actual enterprise ticket.

Four sample tickets separate their requirements into three colors:

| Category | What the agent needs | How it becomes available |
|---|---|---|
| Green | General knowledge, such as API standards | Existing model knowledge |
| Orange | The organization’s prescribed way of working | Skills and instructions |
| Red | Internal terminology, rules, and domain facts | Institutional knowledge |

Completing a ticket requires all three. Strong performance on the green and orange requirements cannot compensate for a missing red requirement. Raj is optimistic enough about coding agents to speculate that the first AGI could be one; his concern is the knowledge surrounding their work.

### Retrieval does not repair its sources

[6:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=416s)

The conventional architecture connects model quality, an agent and its harness, a retrieval layer, and institutional sources such as Confluence, Jira, SharePoint, and GitHub. Retrieval is supposed to carry the missing knowledge into the agent’s working context. Raj mentions a 40% factual-accuracy figure for RAG or knowledge graphs over documented knowledge, but supplies no task, baseline, or definition that makes the percentage interpretable. The architectural dependency is clearer: retrieval needs useful knowledge to retrieve.

Raj describes enterprise setups with ten to twenty MCP servers and recognizes his own earlier approach in them: connect everything, then expect the agent to finish the tickets. Yet a connection returning output does not establish that its output is reliable or useful. He describes retrieved data as nondeterministic, unreliable, and insufficiently tested, and criticizes checking whether a tool returned something instead of evaluating whether it helped solve the task.

His own attempts left him supplying missing facts repeatedly. He recalls accuracy in the ten-to-thirty-percent range, as an informal account rather than a measured evaluation. The rest felt like data entry for the agent. Trying to write the entire domain context by hand only moved the burden: instead of filling gaps during each task, he exhausted himself trying to anticipate them all.

The source collection itself is the problem. Raj’s illustrative knowledge chart describes outdated, unreliable, duplicated, and tribal information. He gives 20% outdated, 20% unreliable, corrects the duplicated category to 10%, and assigns 40% to tribal knowledge; those spoken figures do not form a complete partition. The central category is knowledge that people possess but have never written down. Even a hundred MCP servers cannot retrieve that.

**The organization must turn its knowledge monolith into usable context blocks.** Raj compares that work to breaking a legacy application into microservices and moving from waterfall to agile. Model providers can improve models, agent builders can improve harnesses, and retrieval vendors can improve access—he describes a multibillion-dollar retrieval market—but none of those actors can supply an organization’s unwritten facts. He introduces [Demand-Driven Context](https://arxiv.org/abs/2603.14057), the March preprint accompanying the approach, as a method explored through dataset experiments.

### Let the task pull the missing knowledge

[12:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=779s)

Consider how a new employee learns a domain. After an initial orientation and some documentation links, the employee receives work. Nobody requires them to graduate from the entire company knowledge base first. The task makes particular gaps visible, and the employee asks colleagues for what they need. Demand-driven context applies that sequence to an agent: assign work, let it request missing information, and have it document the answers in reusable context blocks. The framework mediates between the source monolith, the agent, the human expert, and the curated knowledge.

![“What is DDC?” slide with connected boxes labeled Monolith, Agent, Human, and Context Blocks, annotated with agent pulls and human curates.](https://ai.engineer/api/talk-essay-media/9bcd62342836a6c3bf02f21fe09a605d91d274341fad2d8139605671fc25a211)

The pull-and-curate cycle connects agents, human experts and reusable context blocks.

One cycle has a concrete order:

1. Give the agent a task and access to the existing knowledge base.
2. Let it attempt the task and identify where the available context is insufficient.
3. Obtain a checklist of the facts required to proceed.
4. Have a domain expert supply those facts.
5. Use the answers to solve the task and curate reusable knowledge.
6. Repeat with another task, allowing this agent and other agents to reuse what was learned.

The durable output is more than a successful conversation: the next session starts with knowledge that the previous session lacked.

The analogy to test-driven development explains the scope of each addition. A failing test identifies missing behavior; the developer adds enough code to make it pass. Here, a failed work attempt identifies missing context; the expert supplies the necessary facts, and the agent preserves them. Repeated tasks gradually build a useful context base without demanding an exhaustive rewrite of institutional documentation upfront.

### An incident exposes what was never written down

[16:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=995s)

The terminal demonstration assembles this behavior from skills, rules, agents, hooks, and somewhere to store knowledge. Raj uses Copilot at work and Claude Code for the workshop; the method does not depend on either product. Separate panes show the workflow, the existing knowledge collection, and live additions. Flat files represent material that would otherwise live in Confluence, Slack, and GitHub.

The assigned task is incident root-cause analysis. Its context deliberately combines documented facts, outdated material, and information that was never recorded. The agent first searches the existing collection. That is still retrieval; demand-driven context adds what happens afterward. When the retrieved material does not answer the question, the agent must expose the missing information and continue the investigation with an expert.

Raj reports an initial confidence score near 1.5. The agent points to terminology and business logic it does not understand, turning a vague documentation problem into specific requests. This is the value of a task as a probe: asking someone to document everything in their head is unbounded, while asking for the facts needed to investigate this incident is answerable. Raj supplies a prepared, high-level answer. The live run also asks an unexpected clarification about a fictional name, which had not appeared during rehearsal.

New entities then appear in the live knowledge view. Raj clarifies that the filesystem is a demonstration choice: existing MCP or RAG connections can still access material in Confluence or Slack. He describes roughly 56 initial entities, six undocumented entities surfaced by the problem, and another five or six entities curated after his answer. These approximate counts illustrate discovery and persistence, rather than a requirement that each incident produce a fixed number of records.

Across the fourteen-incident demonstration, Raj reports confidence rising from 1.5 to 4.4 as expert answers are collected and documented. No scoring rubric or calibration is supplied, so this trajectory shows the demo’s reported confidence, not independently measured incident-resolution accuracy. The intended change in responsibility is nevertheless concrete: the agent becomes a knowledge manager as well as a consumer. After repeatedly supplying answers by hand, Raj found the interaction too laborious. Archived Jira items, incidents, and support tickets offer a way to automate the discovery work.

### Turn historical incidents into a documentation backlog

[27:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1622s)

The batch demonstration uses a preset platform-operations scenario with twenty recent incidents. Markdown or JSON files contain the incident descriptions and comments; other files represent the knowledge base. The scanner asks how well that knowledge supports each incident: what is usable, what is questionable or outdated, and what is missing. Its pipeline generates probes, runs them against the available knowledge, and analyzes the resulting gaps. The displayed interface shows probe generation and testing complete while gap analysis is still running.

![Scanning your domain interface with completed generating and running probes panels, category scores, and an analyzing gaps panel marked running.](https://ai.engineer/api/talk-essay-media/5b183e805d90c6a412121d9e76971a44adf45c9064a9bbf8d1753c11952a11d9)

Probe generation and testing are complete while gap analysis continues.

A notification service that fails to send customer SMS messages makes the mechanism concrete. The agent can understand SMS in general while knowing nothing about this company’s notification service. Searching for that service may reveal no documentation at all. Alternatively, it may find an old or incomplete Confluence page and ask whether the page is still trustworthy. A year-old page is a reason to investigate its status, not proof that its content is wrong.

The scanner then moves from individual incidents to a consolidated backlog:

1. **Extract demand:** identify the information required by each incident.
2. **Consolidate gaps:** group findings around systems, APIs, business processes, and other domain entities.
3. **Classify evidence:** distinguish clean, stale, incomplete, entirely missing, and tribal information.
4. **Prioritize repair:** assign critical, high, or medium priority and create Kanban work items.

Frequency matters. In Raj’s example, a missing notification-service description that recurs across twenty incidents should be addressed early because it blocks repeated work. Repaired knowledge is saved in the context lake, and the scanner provides a place to inspect how the knowledge base changes as the backlog is addressed.

### Give shared knowledge a review process and a map

[33:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2002s)

Once agents curate information, where should it live? Raj prefers a GitHub repository. Multiple teams, domain experts, and agents may contribute to the same knowledge base, so conflicts and review become normal requirements. Pull requests and reviews provide an existing workflow for proposing changes and resolving disagreements. Curated material can subsequently be published to Confluence or Slack. Direct Confluence integration is also possible; the repository is his implementation preference, not part of the definition of demand-driven context.

A **meta-model** supplies the navigation structure. It describes how business processes relate to systems, how systems relate to APIs, and where business and technical terms belong. A directory structure can represent those entity types, while the records express their relationships. Instead of confronting an undifferentiated pile of files, the agent gets a map of the domain.

That map also helps with change impact. When asked to fix a system, an agent can look for the business processes that depend on it and the APIs that might need to change. Raj strongly recommends this structure, while explicitly treating it as an add-on: a task can reveal missing facts without a meta-model, but relationships make the resulting knowledge easier to navigate and reuse.

### Curate the context that repeated work actually needs

[36:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2186s)

From his experience and feedback from others, Raj identifies two benefits: surfacing knowledge that was never documented, and assigning the management of that knowledge to agents. Historical work items give documentation repair a concrete basis, instead of producing an endless Miro board of speculative missing pages. Experts still supply facts, but the agent can discover requests, organize answers, and preserve them.

The approach has less to offer a small team whose documentation is already good. It is also early, and the manual cycle is best treated as an experiment rather than a sustainable way to operate. Raj’s preferred path is automation once the team understands the loop and its outputs.

Architecturally, curated context sits between the knowledge monolith and the retrieval layer. Raj compares it to a cache: keep frequently useful information close at hand, with links back to deeper sources for unusual cases. His 80/20 rule is a prioritization heuristic—the most useful portion of the documentation may serve most recurring needs—not a measured cache-hit rate. The agent should revisit the wider source collection when the curated blocks do not contain enough information.

### Start with one work item

[40:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2415s)

The workshop offers three entry points: try the preset scanner, inspect the [DDC framework repository](https://github.com/ea-toolkit/ddc) and starter guide, or give an existing agent a single work item and ask it to assess the supporting knowledge. Raj invites participants to remix the approach and contribute. The hosted exercise has a speaker-funded, first-come budget; it is a workshop demonstration rather than a promise of ongoing hosted availability.

The smallest exercise needs no new retrieval infrastructure. Use the agent’s existing MCP connections, provide one Jira ticket or incident, and ask which parts of the required context are usable, doubtful, or absent. This Markdown prompt expresses the exercise for a notification-service incident:

```markdown
# Work item
Investigate why the notification service is not sending customer SMS messages.

# Knowledge assessment
Inspect the connected knowledge sources for the information needed
 to investigate this incident.

For each required fact, report:
- What information the investigation needs.
- Which source, if any, supplies it.
- Whether the information is clean, potentially stale, incomplete,
  or missing from the available sources.
- Which question a domain expert must answer before you can proceed.

Return a prioritized checklist of the gaps.
Do not fill missing institutional facts with assumptions.
```

Inspect the missing-information findings first. They turn a general request to improve documentation into specific questions an expert can answer.

### Keep the scope small enough for experts to answer

[43:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2619s)

The first audience challenge asks whether the approach has worked at scale. Raj says it has not. He narrowed his experiments from enterprise scope to a domain and then to a small team’s tickets, incidents, and Confluence pages. Smaller scope made the work faster and more useful. A broader domain can require several experts because no individual knows enough to answer all the questions.

That limitation is operational, not just technical. An audience member compares relentless agent follow-ups to a denial-of-service attack on the team. Historical tickets may also omit the details the scanner needs. Raj agrees with both concerns and acknowledges that obtaining dedicated expert time from leadership is difficult. His expectation that organizations will increasingly fund context repair as people become agent managers is a prediction about future work, not evidence that the staffing problem has been solved.

Adding source code raises another problem: **which source has authority when code and documentation disagree?** Raj reports good experiences with code alone and prose alone, but mixed results when combining them. An agent can infer behavior from a GitHub repository that conflicts with Confluence. He proposes explicit ranking rules—prefer the code evidence, consult Confluence when that evidence is absent—but says this combined-source behavior is still being worked on.

The next question asks whether a failure should improve the agent’s skill as well as its knowledge base. Raj’s implemented skills are static; evolving them is an untried extension. His immediate priority is moving context repair before live operations. He initially repaired context while an agent was attempting a current task, but the interruptions demanded too much time and patience. He suggests that a narrowly scoped team with a strong domain expert could improve documentation substantially over a short effort; this remains an estimate, later qualified during the discussion.

### Whole-domain context and evolving memory

[50:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3008s)

An audience member asks about the latency and context cost of reading several documents from a repository. Raj reports an average consolidated domain size of around 96K tokens in his experiments. For those domains, he found putting the whole context into the window more useful than his graph-RAG experiments, and suggests additional retrieval mechanisms when the collection approaches a million tokens. He supplies no measured head-to-head evaluation. The [million-token announcement](https://claude.com/blog/1m-context-ga) he invokes was available before the workshop; Claude Code inclusion specified **Opus 4.6** on Max, Team, and Enterprise plans, rather than every Claude Code configuration.

A question about the preprint then turns to domain knowledge versus strategy knowledge. Raj tentatively contrasts demand-driven context with [Agentic Context Engineering](https://arxiv.org/abs/2510.04618), or **ACE**, recalling improvements derived from conversation history and operational interaction. His intended distinction is the emphasis on institutional domain knowledge, including facts that must be elicited from experts. ACE itself also includes curation and both offline and online context optimization, so it should not be reduced to conversational memory alone.

### Freshness, permissions, and possible extensions

[53:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3194s)

Remote documents need more than their text. Raj’s Confluence extraction includes creation and update dates and creator metadata. A date threshold can flag a page for review, but a human decides whether it is actually stale: an old document may still be important and correct. Curated records carry dates and states such as `stale`, `active`, or `clean`, allowing the agent to avoid material already marked stale and look for newer evidence.

Access control belongs to the implementation. Raj relies on GitHub’s read, write, and merge permissions for the repository-based version. The workshop scanner uses presets because he does not want participants uploading proprietary information. A hosted product would need its own access design; the demand-driven method does not itself guarantee that permissions from upstream sources are preserved in curated copies.

An audience member asks whether agents could discover gaps in internal tools as well as documentation. The example is a platform CLI that abstracts Kubernetes: a team with many resources should not need a separate call for each one when a bulk operation would serve the task better. Raj initially distinguishes the method from the tools used to implement it, comparing an agile approach with choices such as Scrum, Kanban, or Lean. After the question is clarified, he accepts tool and business-process gap discovery as possible extensions. Missing bulk operations and performance improvements are suggestions from the discussion, not capabilities demonstrated by the scanner.

### A filled gap can become a wrong answer

[58:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3509s)

The strongest maintenance challenge follows one fact through time: today the answer is `B`, tomorrow `B1`, and later `C`. If all three survive in different documents, the knowledge base must recognize their relationship and stop serving superseded answers. Raj says the scanner flags duplicates and treats a single human-updated document as the latest source of truth. The audience pushes further: finding one changed word across several large documents is a search and synchronization problem, with recurring costs.

**Freshness metadata does not establish correctness.** A recent document can contain a wrong fact. A context block that helped solve one ticket can therefore produce the wrong answer on the next ticket even though the system considers the gap filled. Raj acknowledges the problem and observes that a person following the same incorrect documentation could make the same mistake. That explains the shared dependency on source quality, but it does not provide an automatic way to detect the error before use. The audience’s remaining question is how often to rescan and what that maintenance will cost.

A separate periodic update process is proposed, followed by a claim that it would cost less than repeated onboarding meetings. Another participant challenges that comparison. Raj reports that none of his tested domains exceeded 100K tokens. He does not think daily scans are necessarily required and tentatively suggests that the workshop scanner’s usage could be inexpensive, but no measured cost analysis is presented. An airline-sized organization with hundreds or thousands of repositories, system documents, and third-party sources would face a different problem. The discussion settles on scope and change rate as the relevant conditions, rather than an enterprise-wide cost conclusion.

Evaluation remains a separate question from report detail. After trying the finance preset, an audience member asks how to know the recommendations are sufficient or actually improve the result. Raj explains that the underlying output is much more detailed than the interface: he reports roughly 100–150 lines of Markdown findings per ticket. Those reports can help someone inspect the reasoning, but the answer does not supply an outcome-validation method for deciding that the knowledge base is now good enough.

### Discovery is the beginning of repair

[1:04:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3881s)

The final exchange revisits whether a finite collection of Jira tickets can eliminate the tribal-knowledge gap in a few weeks. Raj makes a consequential correction: the short, team-scoped effort is about discovering gaps, not guaranteeing that all of them have been filled. His proposed sequence is to run one or more initial scans, understand the state of the knowledge base, repair it before operations, and continue agent-and-skill cycles afterward. **Finding the work is different from completing the repair.**

An audience member then offers another source of answers: the knowledge-transfer conversations that already happen when someone joins the organization. Zoom or Teams onboarding calls, chats, and problem-solving discussions can contain the facts that never reached Confluence. Why ask experts to repeat them if transcripts already exist? Raj initially points out that not every useful exchange is recorded and that indiscriminate meeting ingestion can add a large amount of irrelevant text. The participant proposes compressing those transcripts once and making the result part of the knowledge base.

Raj accepts that option. The right source depends on where the institution’s knowledge actually lives: conversations, written documentation, or both. If onboarding and problem-solving transcripts contain the needed facts, use them and compress them into useful context. Expert questioning can then concentrate on what those sources still cannot answer.

## Resources

- [Demand-Driven Context](https://arxiv.org/abs/2603.14057) — The methodology preprint explains failure-driven knowledge curation, an entity meta-model, and a worked retail incident-management example.
- [DDC framework and starter materials](https://github.com/ea-toolkit/ddc) — Methodology, starter-guide links, domain templates, example knowledge bases, and demonstration materials for trying demand-driven curation.
- [Context-gap scanner source](https://github.com/ea-toolkit/ddc/tree/main/tools/context-gap-scanner) — The context-gap scanner directory within the DDC project.
- [Agentic Context Engineering](https://arxiv.org/abs/2510.04618) — ACE develops evolving context playbooks through generation, reflection, and curation, supporting offline optimization and online adaptation.
- [Claude’s March 2026 million-token context announcement](https://claude.com/blog/1m-context-ga) — Anthropic’s announcement documents model and subscription conditions for million-token context availability before the workshop.

## Slides

### AI Engineer Europe

[0:00](https://www.youtube.com/watch?v=_QAVExf_1uw&t=0s) [AI Engineer Europe logo in white on black.](https://assets.ai.engineer/sites/aiecode2025/sha256/aab59c7da120f8504a96c9b8622254406bbff88706e2388a912d6dcc744c1118.webp)

AI Engineer
EUROPE

### Presenting sponsor

[0:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=5s) [Google DeepMind presenting sponsor card.](https://assets.ai.engineer/sites/aiecode2025/sha256/22e8cb940ef752894fb0f47ce4cfbb7a04b212fd252bdf1347f3f5879fa393f3.webp)

PRESENTING SPONSOR

Google DeepMind

### Platinum sponsors

[0:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=10s) [Braintrust, WorkOS and OpenAI platinum sponsor logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/1369b141e5b3963a2ad15a058b8724231a167e51f5b78624d85cae9fb8634aba.webp)

PLATINUM SPONSORS

Braintrust
WorkOS
OpenAI

### Raj Navakoti

Raj Navakoti is a staff software engineer in IKEA’s Delivery and Services domain. His interests include AI architecture, neuroscience and linguistics.

[0:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=48s) [Raj Navakoti introduces his role at IKEA, interests and public profiles, with a portrait and QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/6792056ab82ca357895caa238a50d896461e71b223e52cc88fa05be647a80934.webp)

RAJ NAVAKOTI

Staff Software Engineer
@Ingka Digital, IKEA

AI Architecture · Neuroscience · Linguistics

WEB — rajnavakoti.dev
LI — linkedin.com/in/rajnavakoti
GH — github.com/rajnavakoti

Interface text:
←  2 / 34  →
[N] [×]

### Before We Start

Raj Navakoti takes an audience pulse check on London visits, engineering backgrounds, AI agent use, and agent extensions.

[1:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=79s) [Four audience questions about visiting London, engineering, AI agents, and agent extensions.](https://assets.ai.engineer/sites/aiecode2025/sha256/1dd1394127822033a4ddce7a234fc859c3acd7665eeaeed7b990f70c79b926fe.webp)

BEFORE WE START

Q1 — Who's first time in London? (welcome, the city is confusing — just like your knowledge base)
Q2 — Who here is an engineer? (vibe coding and prototyping counts)
Q3 — Who's actively working with AI agents? (Claude, Copilot, Cursor, custom — anything)
Q4 — Who uses agent extensions? (skills, rules, MCP servers, RAG pipelines)

Interface text:
←  3 / 34  →
[N] [×]

### What You'll See in the Next 1+ Hour

The talk moves from agents and extensions to context gaps, then hands-on demos of demand-driven context. The speaker plans to spend more time on the problem before moving into the demos.

[1:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=119s) [Agenda covering the situation, problem, and resolution for agent context, alongside an illustration of Hamlet holding a skull.](https://assets.ai.engineer/sites/aiecode2025/sha256/b6b1aa487ac7b445466f635dba9b72ccfb461617e8726d137e601c624283fd51.webp)

WHAT YOU'LL SEE IN THE NEXT 1+ HOUR?

“A story of an agent discovering and curating his own context.”

SITUATION
> AI to agents evolution
> What agents can and can't do
> The extensions landscape

PROBLEM
> Why epics aren't moving
> The retrieval illusion
> The 40% tribal knowledge gap

RESOLUTION
> Demand-Driven Context
> Live demos: manual + scanner
> Context lake and meta model

HAMLET: Alas, poor Yorick!

Interface text:
←  4 / 34  →
[N] [×]

### The Situation

Act 1 introduces where we are and how we got here.

[2:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=162s) [Act 1 divider: The Situation — where we are and how we got here.](https://assets.ai.engineer/sites/aiecode2025/sha256/cd4a7b61c6230fa9ea50d0861dabe21072fe9d2937a498d2632f6bf06d91a32f.webp)

ACT 1
THE SITUATION
Where we are — and how we got here

Interface text:
5 / 34

### The New Kid — Meet Leo

The slide rates AI 8–10/10 on general skills but 0/10 in your domain, pairing that gap with Memento’s short-term memory problem.

[2:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=167s) [A chart rates AI capabilities highly but domain knowledge at zero, alongside scenes from Memento about short-term memory.](https://assets.ai.engineer/sites/aiecode2025/sha256/ba3aeee112f3beb94259cf66489b9977065912336610bb71c042e43a9587f852.webp)

THE NEW KID — MEET LEO

this is the gap
this workshop is about
↓

Reasoning: 9/10
Computation: 10/10
General Knowledge: 8/10
Code Generation: 9/10
YOUR Domain: 0/10

- That's the thing. I have this condition.
- A condition?

- It's my memory.
- Amnesia?

No, no. It's different from that.
I have no short-term memory.

If we talk for too long,
I'll forget how we started.

Interface text:
←  6 / 34  →
[N]  [×]

### AI to Agent — The Evolution

The timeline traces agents from debates over definitions through tools and orchestration to execution in real workflows. The speaker describes the progression from prompt engineering and RAG to MCP and multi-agent systems.

[3:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=237s) [A four-stage timeline moves from debates over agent definitions through tools and agent teams to execution in real workflows.](https://assets.ai.engineer/sites/aiecode2025/sha256/ca9e3aa60d60e91e2ce437bd888628fbf7291ccad35f02a68e0f9023050c7bd7.webp)

AI TO AGENT — THE EVOLUTION

Everyone had a different definition. Heated debates everywhere.
“Wait... what IS an agent exactly?”

Tools, memory, rules, MCP. The extension era begins.
“So agents and workflows are the same thing?”

Agent teams, orchestration, delegation. The system starts to scale.
“Can someone PLEASE define agentic?”

From chat to execution: agents become part of real workflows.
“Ohhh... the model was only one layer.”

// NOTES
“To put this in perspective: in early 2024, Devin solved 14% of real GitHub issues independently. By 2026, agent teams score over 80% on the same benchmark. AI agent capability has been doubling every 7 months. Replit's Agent built a full production app — database, auth, real-time UI, deployed — in under 10 minutes from plain English. The capability leap is undeniable.”

Interface text:
←  7 / 34  →  [N]  [×]

### The Problem

Act 2 turns from AI agent capabilities to what is blocking enterprise AI.

[4:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=269s) [Act 2 divider: The Problem — what is blocking enterprise AI.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c77bcdc8d7c61a2e106c3261924173bc02764f7d517a9afc7c45f649fb02d5e.webp)

ACT 2
THE PROBLEM
What's actually blocking enterprise AI

// NOTES
Now we get to it. This is the meat. If they only remember one thing from the workshop, it should be from the next 15 minutes.

Interface text:
←  8 / 34  →
[N] [×]

### A Queen's Dilemma

AI agents can write code, review changes and handle incidents. The speaker asks why those capabilities aren't moving Jira epics and delivering business results.

[4:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=274s) [A question asks why Jira epics are not moving despite capable AI agents, beside lists of code, operations, business and knowledge tasks.](https://assets.ai.engineer/sites/aiecode2025/sha256/2ce9fc4a0e06aeff2b3c36e0528f2396745b74663f57f22e0d9513f7b4d26e44.webp)

A QUEEN'S DILEMMA

"If AI agents are that smart, why are my Jira epics not moving?"

CODE & OPS
> Writing code from specs
> Refactoring and reviewing
> Generating tests
> Monitoring and auto-resolving incidents

BUSINESS & KNOWLEDGE
> Processing documents end-to-end
> Handling complex support tickets 24/7
> Summarizing large document sets
> Detecting anomalies and fraud

// NOTES
"MIT found 95% of enterprise AI pilots failing — not because models are weak, but integration. McKinsey: 88% use AI, only 6% capture real value. Gartner: 40% of agentic projects won't reach production. Agents can do all of this."

Interface text:
←  9 / 34  →
[N]  [×]

### Code != Development

Four sample Jira tickets mix what agents know, teachable rules, and institutional knowledge. The speaker argues that agents struggle with the last category, even though completing a ticket requires all three.

[5:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=331s) [Four sample work tickets use colored markers to distinguish agent knowledge, teachable rules and institutional knowledge.](https://assets.ai.engineer/sites/aiecode2025/sha256/29f9feb09a2709c22b3362ebe63ed6bbeb89e44e5ecfe364b24ef7c8c928d7b5.webp)

CODE != DEVELOPMENT

Legend:
Green: Agent knows this
Gold: Teachable via skill or rule
Red: Institutional knowledge — scattered, outdated, or undocumented

DLVR-2891 — STORY
Publish API contracts to partner portal
[Green] All v4 contracts published to portal
[Gold] OpenAPI spec follows REST standards
[Red] Partner access levels mapped per tier
[Red] Deprecation notice sent for v2 endpoints

PLAT-0312 — TASK
Migrate capacity service to new platform
[Green] Zero-downtime deployment strategy
[Gold] Infra-as-code follows platform conventions
[Red] Capacity template validation rules preserved
[Red] Custom market overrides for 3 markets carried over

SRE-0099 — INCIDENT
Settlement batches arriving 6 hours late
[Green] Root cause identified and documented
[Gold] Monitoring alert added for batch SLA
[Red] Silent retry mechanism in reconciliation documented
[Red] Upstream KYC provider throttle limit confirmed

CUST-0415 — BUG
Dual-eligible members getting wrong copay
[Green] Copay correct for Medicare+Medicaid
[Gold] Unit tests cover all 4 COB methods
[Red] State-specific regulation applied
[Red] Priority order matches retired analyst config

Interface text:
← 10 / 34 →
[N] [×]

### The “Kind Of” Solution

A retrieval layer connects enterprise knowledge sources to an agent and its harness, which produce code, designs, decisions and artifacts. The slide notes that this works for documented knowledge.

[6:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=418s) [Diagram connecting enterprise knowledge sources through a retrieval layer to an agent and its outputs.](https://assets.ai.engineer/sites/aiecode2025/sha256/6db3701472ade0b1365f2a3991695682159e5109826204d24c9ab2dcfa88625d.webp)

THE "KIND OF" SOLUTION

enterprise knowledge base →

CONTEXT
• Confluence
• Jira
• SharePoint
• GitHub
• Slack

RETRIEVAL
• RAG
• Graph RAG
• MCP
• API

retrieval layer

agent & harness

AGENT
• Skills
• Hooks
• Rules
• Specs
• Instructions

ROI
Code, designs, decisions, artifacts.
Works for documented knowledge.

Interface text:
← 11 / 34 →
[N] [×]

### What Everyone Is Doing About It

Dumping context drowned the agent in noise; RAG missed unwritten knowledge; a knowledge graph went stale in 3 weeks; and manual curation led to burnout at 15%.

[7:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=471s) [Four approaches to enterprise knowledge—dumping everything, RAG, knowledge graphs, and manual curation—are shown alongside a historical photograph of typists.](https://assets.ai.engineer/sites/aiecode2025/sha256/779e44c7ef57cd7059c7160487b948ccb3ca8e6a5d4dde4d2ea47c147292e23e.webp)

WHAT EVERYONE IS DOING ABOUT IT

"First RAG, then MCP, now it's a data entry job."

ATTEMPT 1: DUMP EVERYTHING
> 2M tokens of irrelevance
> Agent drowned in noise

ATTEMPT 2: RAG
> Chunked embedded retrieved
> RAG can't retrieve what was never written down

ATTEMPT 3: KNOWLEDGE GRAPH
> Entities and relationships
> Went stale in 3 weeks — nobody maintains it

ATTEMPT 4: MANUAL CURATION
> Blank page — started documenting
> Burned out at 15% — I was guessing what to document

Interface text:
←  12 / 34  →
[N]  [×]

### The Real Problem

The speaker sketches enterprise knowledge as a monolith: 40% is tribal and undocumented, while other slices are outdated, unreliable or duplicated. He proposes breaking it into context blocks useful to agents.

[10:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=608s) [A pie chart categorizes enterprise knowledge as clean, outdated, unreliable, duplicated, and tribal, beside an image labeled Monolith.](https://assets.ai.engineer/sites/aiecode2025/sha256/dddf658bf3bc28ed907ef84929d1d068e875323860c79ebb7f1f167d8eb0ea77.webp)

THE REAL PROBLEM

"Your enterprise context is a monolith."

(clean) some useful info
(outdated) last_updated_1997.pdf
(unreliable) final_real_final.doc
(duplicated) copy_of_copy.xlsx
(tribal) I know a guy who knows a guy and he knows how it works

MONOLITH

Interface text:
← 13 / 34 →
[N] [×]

### The Paper

Raj and Saideep Navakoti’s March 2026 preprint proposes using agent failures to identify which domain knowledge to curate.

[12:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=753s) [Slide introducing the Demand-Driven Context paper by Raj Navakoti and Saideep Navakoti, with an arXiv page screenshot.](https://assets.ai.engineer/sites/aiecode2025/sha256/f30058774ca6341dc7d34bba64d39770a7ecfd8893b4e9a4f452350bc18335f2.webp)

THE PAPER

“Demand-Driven Context: A Methodology for Building Enterprise Knowledge Bases Through Agent Failure”

Raj Navakoti and Saideep Navakoti
arxiv — March 2026

Embedded arXiv page:
Cornell University
Learn about arXiv becoming an independent nonprofit.
We gratefully acknowledge support from [illegible]

arXiv > cs > arXiv:2603.14057
Computer Science > Artificial Intelligence
[Submitted on 14 Mar 2026]

Demand-Driven Context: A Methodology for Building Enterprise Knowledge Bases Through Agent Failure
Raj Navakoti, Saideep Navakoti

Large language model agents demonstrate expert-level reasoning, yet consistently fail on enterprise-specific tasks due to missing domain knowledge -- terminology, operational procedures, system interdependencies, and institutional decisions that exist largely as tribal knowledge. Current approaches fall into two categories: top-down knowledge engineering, which documents domain knowledge before agents use it, and bottom-up automation, where agents learn from task experience. Both have fundamental limitations: top-down efforts produce bloated, untested knowledge bases; bottom-up approaches cannot acquire knowledge that exists only in human heads. We present Demand-Driven Context (DDC), a problem-first methodology that uses agent failure as the primary signal for what domain knowledge to curate. Inspired by Test-Driven Development, DDC inverts knowledge engineering: instead of curating knowledge and hoping it is useful, DDC gives agents real problems, lets them demand the context they need, and curates only the minimum knowledge required to succeed. We describe the methodology, its entity meta-model, and a convergence hypothesis suggesting that 20–30 problem cycles produce a knowledge base sufficient for a given domain role. We demonstrate DDC through a worked example in retail order fulfillment, where nine cycles targeting an SRE incident management agent produce a reusable knowledge base of 46 entities. Finally, we propose a scaling architecture for enterprise adoption with semi-automated curation and human governance.

Comments: 18 pages, 5 figures, 1 algorithm. Preprint
Subjects: Artificial Intelligence (cs.AI)
ACM classes: [illegible]
Cite as: arXiv:2603.14057 [cs.AI]
(or arXiv:2603.14057v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2603.14057

Submission history
From: Raj Navakoti [view email]
[v1] Sat, 14 Mar 2026 17:58:56 UTC (21 KB)

Interface text:
15 / 34
[N] [×]

### The Positioning — The Complete Picture

DDC extracts, consolidates and curates scattered context into agent-ready blocks. Retrieval connects those blocks to agents, with domain-expert reasoning and business delivery as the intended outcome.

[38:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2337s) [A pipeline moves from context sources through DDC, curated blocks, retrieval and an agent to business delivery.](https://assets.ai.engineer/sites/aiecode2025/sha256/6c01dce04c061d2dad27738652e040d0198e34b3594efada776b9e82d9342d22.webp)

THE POSITIONING — THE COMPLETE PICTURE

CONTEXT
• Confluence
• Jira
• SharePoint
• GitHub
• Slack

messy, tribal, outdated

the missing piece
DDC
• Extract
• Consolidate
• Curate

CURATED BLOCKS
structured, agent-ready

RETRIEVAL
• RAG
• Graph RAG
• MCP
• API

AGENT
• Skills
• Hooks
• Rules
• Specs

ROI
Domain-expert reasoning. Not just code — business delivery.

Interface text:
30 / 34
[N]
[×]

### What You Take Home

The DDC cycle starts with an agent failing at a real problem, then uses a demand checklist to guide curation. A scanner, template repo and guide help participants try the approach.

[40:13](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2413s) [Workshop takeaways covering the framework, process, and tools for demand-driven knowledge.](https://assets.ai.engineer/sites/aiecode2025/sha256/0376f15ea808cd632a3da613f3bc310a606d0332061477e27863531373f3e64c.webp)

WHAT YOU TAKE HOME

"Don't push knowledge. Let agents pull. 30 real problems beat 6 months of documentation."

THE FRAMEWORK
> DDC methodology — TDD for knowledge
> Entity meta model — 13 types with relationships
> Context lake — Git-based knowledge repo

THE PROCESS
> Give agent a problem → let it fail
> Demand checklist reveals gaps
> Curate minimum → agent succeeds
> Knowledge compounds across cycles

THE TOOLS
> Context Gap Scanner — scan your domain
> Template repo — entity formats and starter structure
> GETTING-STARTED guide

Interface text:
← 31 / 34 →
[N] [×]

### Your Turn (Optional)

For 15 minutes, participants can scan a domain, make a demand checklist for a real problem, or run a DDC cycle on a problem card. The presenter will walk around and help.

[41:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2475s) [Three optional exercises: use the scanner, plan a mental DDC cycle, or work through a problem card.](https://assets.ai.engineer/sites/aiecode2025/sha256/c6f379b1fcdbceb517945c4595c8d85f02688553b807c2bc9422b468f2a7026a.webp)

YOUR TURN (OPTIONAL)

OPTION A: USE THE SCANNER
> Open the URL on your laptop or phone
> Type a paragraph describing your domain
> See your AI-Readiness Score

OPTION B: MENTAL DDC CYCLE
> Think of a real problem from your domain
> What would an agent need to know to solve it?
> Write down the demand checklist

OPTION C: PROBLEM CARDS
> Pick a card from the table
> Enterprise problems with hidden domain knowledge
> Run a DDC cycle on it

"15 minutes. I'll walk around and help."

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Wed 8 Apr 16:08
Build Your First Demand-Driv…
Context Gap Scanner — See…
Context Gap Scanner — See…
Sample.drawio (1).svg
GitHub - ea-toolkit/ddc: Dem…
rajnavakoti-dev.web.app/presentations/ai-engineer-workshop/?slide=32
Incognito
RAJ_N | SYS.ONLINE
HOME  WRITINGS  PUBLICATIONS  PRESENTATIONS  [DK]
32 / 34

### Context Gap Scanner

The speaker invites attendees to try the preset Context Gap Scanner. The visible result scores readiness at 58/100 and identifies gaps in tribal knowledge and processes.

[41:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2497s) [Scanner landing page with a domain scan button and a partial readiness score.](https://assets.ai.engineer/sites/aiecode2025/sha256/bf2477da7b926ab9eec38983bb1ad09db58d46c21bddc1783d9830ec613ba374.webp)

CONTEXT GAP SCANNER

See where your
AI agents will fail.

Your AI agent doesn't know your domain. Find out exactly what's missing — before it matters.

SCAN A DOMAIN

AI-READINESS SCORE
58 /100

PARTIAL — Agents can handle basics but will miss edge cases

This domain shows moderate AI readiness with strong terminology knowledge but critical gaps in tribal knowledge and processes. The most concerning pattern is the concentration of operational knowledge in specific individuals (only 2 people know split-brain resolution) and undocumented procedures that create single points of failure. While

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Wed 8 Apr 16:09
Build Your First Demand-Driv…
Context Gap Scanner — See…
Context Gap Scanner — See…
Sample.drawio (1).svg
GitHub - ea-toolkit/ddc: Dem…
context-gap-scanner.web.app
Incognito

### Scanner probe results

The preset scanner scores domain terminology at 80% and tribal knowledge at 35%. An incident diagnosis scores 4/5, while the probe identifies missing details needed to investigate and change the connection pool.

[41:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2507s) [Readiness category scores and an incident diagnosis probe with identified knowledge gaps.](https://assets.ai.engineer/sites/aiecode2025/sha256/3013461a864f4ccc16bf0f824b6028942e1d95a639e6610b3fed80130f5eaadf.webp)

Business Processes 40%
Domain Terminology 80%
Data Models & Schemas 65%
System Integrations 70%
Tribal Knowledge 35%

Probe Results

01 Incident Diagnosis 4/5

QUESTION
You're on-call and receiving alerts that payment-engine p99 latency has spiked to 8 seconds during EU morning peak traffic. Orders are timing out and the order-gateway circuit breaker is triggering. What's the most likely root cause based on recent patterns, and what's your immediate remediation step?

AGENT ATTEMPT
Based on the recent incident pattern, the most likely root cause is connection pool exhaustion on ledger-db, which previously caused a similar p99 spike to 12s during EU morning peak (INC-2891). The immediate remediation step would be to increase the connection pool size on ledger-db from its current setting, as this fixed the identical issue in February.

KNOWLEDGE GAPS (6)
Current connection pool size on ledger-db (only know it was increased from 20 to 50 in February)
How to actually increase the connection pool configuration
Location of connection pool settings in the codebase
Whether there are database monitoring dashboards to confirm pool exhaustion

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Wed 8 Apr 16:09
Build Your First Demand-Driv…
Context Gap Scanner — See…
Context Gap Scanner — See…
Sample.drawio (1).svg
GitHub - ea-toolkit/ddc: Dem…
context-gap-scanner.web.app
Incognito

### Demand-Driven Context framework

The repository describes a cycle in which an agent attempts a problem, a human fills the gaps it identifies, and the agent curates the information. Repeating the cycle builds a knowledge base.

[41:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2515s) [Repository README describes the problem, the framework, and its knowledge curation cycle.](https://assets.ai.engineer/sites/aiecode2025/sha256/8c37dd68f3370d71ca1ba5c1aaf3d47213e6c3fab3bb7ca77ad9a23bff630d5f.webp)

Demand-Driven Context (DDC) Framework

DDC is TDD for knowledge bases — failing agents drive curation, not failing tests.

A methodology for building enterprise AI agent knowledge bases through problem-driven curation.

Stop trying to document everything. Give your agent a concrete problem. Let it tell you what's missing. Repeat 30 times.

The Problem

Enterprise AI agents excel at reasoning but lack the domain-specific context to make accurate decisions. Current approaches fail because they try to anticipate what an agent needs:

• Context window stuffing — too large to compress, agent can't prioritize
• RAG over wikis — retrieves documents, not understanding
• Top-down curation — takes months, half is never used, stale before you finish
• Skills.md — great for patterns, insufficient for deep domain knowledge

The Solution: DDC

DDC flips the approach: instead of telling agents what to know, let problems reveal what to curate.

Problem -> Agent Attempts (fails) -> Information Checklist
    -> Human Fills Gaps -> Agent Curates -> Agent Succeeds
                                             |
                         Repeat 20-30x --+-- Knowledge Base Emerges

Think TDD for knowledge bases: failing tests drive code, failing agents drive curation.

What's In This Repo

Directory | Contents

TypeScript 50.9%
JavaScript 18.0%
Python 16.7%
CSS 7.4%
Shell 3.6%
HTML 3.4%

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Wed 8 Apr 16:09
Build Your First Demand-Driv…
Context Gap Scanner — See…
Context Gap Scanner — See…
Sample.drawio (1).svg
GitHub - ea-toolkit/ddc: Dem…
github.com/ea-toolkit/ddc
Incognito
README
MIT license

### Framework repository

The public DDC repository brings together examples, templates and tooling. The speaker also points to a starter guide for trying the framework.

[42:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2523s) [Repository overview showing project directories, files, and a brief description.](https://assets.ai.engineer/sites/aiecode2025/sha256/012f3517440ba69ab2b72340d0c4877295d72a1cc3e581363d28a2327f8c30ab.webp)

ea-toolkit / ddc Public

main
34 Branches
0 Tags
Go to file
Code

rajnavakoti Merge pull request #48 from ea-toolkit/chore/scanner-firebase-migrati... b95d406 · 2 weeks ago
197 Commits

.claude — chore: reframe domain as synthetic, remove all fingerprint... — last month
.github — feat: add claude code extensions for ddc workflows — last month
ddc-cycle-log — add: Initial DDC framework — methodology, tooling, synth... — 2 months ago
demo — feat(meta): update to 13-entity ddc meta-model — last month
examples/healthcare-claims — feat(meta): update to 13-entity ddc meta-model — last month
meta — chore: reframe domain as synthetic, remove all fingerprint... — last month
retailco — chore: reframe domain as synthetic, remove all fingerprint... — last month
templates — feat: add retailco domain kb and update template structure — last month
tooling — add: Initial DDC framework — methodology, tooling, synth... — 2 months ago
tools/context-gap-scanner — chore: move scanner to tools/, migrate Netlify to Firebase — 2 weeks ago
.gitignore — chore: move scanner to tools/, migrate Netlify to Firebase — 2 weeks ago
CLAUDE.md — feat: add claude code extensions for ddc workflows — last month
GETTING-STARTED.md — chore: reframe domain as synthetic, remove all fingerprint... — last month
LICENSE — add: Initial DDC framework — methodology, tooling, synth... — 2 months ago
METHODOLOGY.md — chore: reframe domain as synthetic, remove all fingerprint... — last month
README.md — chore: reframe domain as synthetic, remove all fingerprint... — last month

About
Demand-Driven Context — TDD for knowledge bases. A methodology for building enterprise AI agent knowledge bases through problem-driven curation.

Readme
MIT license
Activity
Custom properties
0 stars
0 watching
1 fork
Report repository

Releases
No releases published

Packages
No packages published

Contributors 2
rajnavakoti Raj
claude Claude

Interface text:
Chrome File Edit View History Bookmarks Profiles Tab Window Help
Wed 8 Apr 16:09
Build Your First Demand-Driv...
Context Gap Scanner — See... (two tabs)
Sample.drawio (1).svg
GitHub - ea-toolkit/ddc: Dem...
github.com/ea-toolkit/ddc
Incognito
Platform Solutions Resources Open Source Enterprise Pricing
Search or jump to... Sign in Sign up
Notifications Fork 1 Star 0
Code Issues Pull requests Actions Projects Security and quality Insights

### Repository contents and quick start

The repository includes the DDC methodology, templates, tooling and an example knowledge base. Its quick start walks readers through exploring the example, starting a domain and running a first DDC cycle.

[42:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2530s) [README table of repository contents followed by steps for trying the framework.](https://assets.ai.engineer/sites/aiecode2025/sha256/3307093151cdc60cf4dd7b41b2f8ec6f9c51aafbdd93306c89d745bc628c4825.webp)

Think TDD for knowledge bases: failing tests drive code, failing agents drive curation.

What's In This Repo
Directory | Contents
METHODOLOGY.md | The DDC framework — how it works
WHY.md | The hypothesis and motivation
CLAUDE.md | AI agent guidance for DDC knowledge bases
meta/ | Entity and relationship type definitions
templates/ | Starter kit — domain skeleton, entity templates, cycle log template
tooling/ | Web UI for exploring knowledge bases (FastAPI + React)
examples/healthcare-claims/ | Synthetic domain with ~40 entities and 5 DDC cycle logs
demo/ | Empty knowledge base + script for trying DDC yourself
ddc-cycle-log/ | Cycle log format spec and analysis scripts

Try DDC in 30 Minutes
Have Claude Code (or any AI coding agent) installed? Follow the Getting Started guide to:
1. Browse the example healthcare claims knowledge base
2. Copy the template and start your own domain
3. Run your first DDC cycle on a concrete problem

See demo/DEMO-SCRIPT.md for a live demo walkthrough.

Quick Start
1. Explore the Example Domain
cd tooling
DDC_KNOWLEDGE_BASE_PATH=../examples/healthcare-claims/domain-knowledge ./start.sh

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Wed 8 Apr 16:09
Build Your First Demand-Driv...  Context Gap Scanner — See...  Context Gap Scanner — See...  Sample.drawio (1).svg  GitHub - ea-toolkit/ddc: Dem...
github.com/ea-toolkit/ddc
Incognito
README  MIT license

### Questions

In Q&A, the speaker says organizations need a central solution and can choose their own methods and apps for implementing the approach.

[56:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3388s) [A Questions slide projected beside the presenter.](https://assets.ai.engineer/sites/aiecode2025/sha256/e41439078d968a2f99939b565f402272d19fd34a9d06603a275074e97c6871df.webp)

QUESTIONS

## Full transcript

[0:00](https://www.youtube.com/watch?v=_QAVExf_1uw&t=0s) [upbeat music] Uh, thank you.

[0:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=16s) Maybe let, we can get started. Uh, first of all, thank you so much for coming for the workshop, especially ones who didn't get the seat. Uh,

[0:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=26s) I, I promise you, I'll do my best to make it entertaining, especially for you who are sitting. [laughs] Uh, thank you so much.

[0:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=33s) We are the volunteers, right?

[0:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=34s) Yeah. [laughing] Uh, actually it, it, it makes sense, so now I know like why the tickets got sold out, right? Uh, which workshop actually sold out the tickets. [laughing]

[0:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=45s) So, uh, s- let's start with, uh, my introduction. So I'm Raj. Uh, I work as a staff software engineer, uh, at IKEA. Uh, I work for a domain called Delivery and Services.

[0:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=57s) Uh, basically we are like almost, um, more than 100 engineers and six product teams altogether. It's like a mini company within the company itself. Uh, I'm very interested with architecture, neuroscience and linguistics, and now AI.

[1:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=71s) So if anyone has some cool projects, because everybody is building cool projects these days, please find me after this meeting.

[1:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=80s) So quick pulse check, uh, with the, uh, audience. Uh, who is visiting London for the first time?

[1:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=88s) Okay, cool. Welcome to London. Uh, who is here, uh, from the engineering background?

[1:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=96s) Also with vibe coding and, uh, prototyping.

[1:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=98s) Mm-hmm.

[1:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=98s) No all of... Okay. So, uh, who actively uses agents like Copilot or... Okay, this is going to be tough for me then.

[1:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=107s) Yeah. [laughs]

[1:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=108s) So extensions? Okay, so everybody's pro. Okay, fine.

[1:52](https://www.youtube.com/watch?v=_QAVExf_1uw&t=112s) Not so much as I wanted to.

[1:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=113s) Okay. So much tension now. Uh, so you're gonna sit here in this hot room for more than an hour. So first I will, uh, give a bit of introduction of what I'm gonna present today.

[2:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=127s) Uh, it's basically on agent and the context management. Uh, I, I divide it into three parts. Uh, one is the situation which all of you already know, so I'll keep it tight and short, like five minutes.

[2:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=138s) Uh, then I will talk about the problem. This is where I'll spend a bit more time on the slides because I think like, uh, nobody is actually seriously looking into the problem, uh, and I want to bring it up.

[2:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=152s) Then less slides and more into some kind of a hands-on, uh, how the, uh, actual demand-driven context actually works. All good?

[2:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=165s) Yes.

[2:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=165s) Okay. So let's start with the first one.

[2:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=169s) How many have seen this movie? Memento.

[2:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=173s) Oh, yeah.

[2:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=173s) Okay. Okay. Okay, cool. So I'll, I'll give the gist of the movie. So this guy is very skilled, uh, very talented. The only problem he has is, uh, he can't hold memory more than 15 minutes.

[3:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=185s) So every 15 minutes he has to take his notebook, uh, watch his [REDACTED:physical_attribute] that he put it on his one and figure out, "Okay, what I was doing before the 15 minutes?"

[3:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=194s) And he does it again and again. If you relate to the AI and AI agents things and all, it actually fits exactly how the movie is and how the agents are right now.

[3:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=203s) If you go and watch this movie, you don't need to y- watch YouTube or blogs to understand agents and MCP. This movie actually tells you ab- about everything, literally.

[3:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=212s) Uh, and as this guy has a memory problem, uh, in the same way, the AI, uh, that we got introduced couple of years ago, uh, is very good with reasoning, computation, uh, code generation.

[3:44](https://www.youtube.com/watch?v=_QAVExf_1uw&t=224s) It's, it's benchmark as like, uh, above the par. The only problem is the institutional knowledge, right? The domain knowledge that you have, that, that's the only thing we have to be, uh, a, a bit problematic.

[3:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=235s) So from AI to agent, if you look at the eva- uh, evolution, uh, it exploded. So it started from prompt engineering, first of all. Uh, then there was RAGs, uh, now MCPs, then multi-agent.

[4:09](https://www.youtube.com/watch?v=_QAVExf_1uw&t=249s) Now it is deep agents. Uh, I recently found out like, uh, using Replit, actually, you can build a full stack app in 10 minutes. That means the... By the time you make the instant noodles, you already have a million-dollar, uh, app already working on your laptop.

[4:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=264s) So we, we got it to this point, uh, like it's extraordinarily good.

[4:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=271s) Now, that's AI and agents. So let's talk about enterprise AI.

[4:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=276s) Okay. Uh, I don't know how, how many have you have this question, but most of the enterprises I see the question is, okay, AI is pretty smart. It's doing, uh, code generation, full stack apps, reviewing your PRs, uh, doing incident management, all those things, uh, right?

[4:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=293s) So if AI was doing that much, why is the Jira tickets or epics are not moving on the dashboard, right? Why do I don't see the delivery actually? So everybody is speaking about look at like three minutes, everything is ready.

[5:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=305s) Yeah, okay, fine. Why are my Jira epics are not moving? Because that defines the business delivery and that defines the return of investment, right? Uh, and as you see, like, uh, it's, it's from the McKinsey this year, like 88% of all companies use AI, but they only see like, uh, 6% of value creation.

[5:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=333s) Okay. So I think this is the problem, uh, that we have. Uh, I have four Jira tickets, different ones, uh, sample ones, and you can see the green ones that I have, uh, marked is basically which LLMs are already trained on, like API standards or like, uh, uh, things very, uh, they already know it's a general knowledge,

[5:54](https://www.youtube.com/watch?v=_QAVExf_1uw&t=354s) right? That's fine. Those tasks from the ticket it can pick up and it can do it. Now, there are second part orange ones, which we have to teach them actually.

[6:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=362s) So you have... You know this, but do this in this way. So all this kind of an orange color things will fit into, you know, your agent extension like skills or like, uh, [audio cuts out]

[6:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=376s) But red ones, that's what the institutional knowledge is, which sits within the company and within the people. So unless if it picks a task, uh, if it picks a ticket, it has to fulfill all of them.

[6:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=391s) It is so good at with, uh, green ones and orange ones, but it struggles with the red one, with the institutional knowledge. And, uh, what I believe is, uh, right now the coding agents are getting so, so better.

[6:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=405s) I, I feel like if there is an AGI coming, the first AGI will be a coding agent for sure. Uh,

[6:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=416s) so to fix the, uh, giving the institutional knowledge to, uh, the agents, so we have an industry solution already. So this is basically your return of investment on AI pipeline will look like, right?

[7:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=430s) So you have a LLM model quality, uh, you have agents, and you have a agent harness, and your, uh, your institutional knowledge sits under Confluence, Jira, SharePoint, GitHub, all those things.

[7:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=442s) And basically, retrieval layer is what industry is telling us will fix that issue. So you build that retrieval layer, and then it will fetch all those things and give it to an agent, and the agent should be able to do it, right?

[7:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=453s) So basically, uh, so forty percent, uh, actu- factual accuracy can be achieved through RAG or like knowledge graphs, actually, uh, but with a documented, uh, knowledge base.

[7:52](https://www.youtube.com/watch?v=_QAVExf_1uw&t=472s) Now, basically, if you build a retrieval layer, it has to work, right? Now, let me ask you a question. How many have you built a retrieval layer things like RAGs and MCPs?

[8:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=482s) Okay, cool. All of them. Okay. Now the question is, how many did you build? How many M-MCPs did you build? How many have you built more than twenty MCP servers?

[8:13](https://www.youtube.com/watch?v=_QAVExf_1uw&t=493s) Okay. Okay. So nobody beats my record then. Uh, so what I see is, uh, mostly in the enterprise organizations and all, people are building like ten to fifteen or like twenty MCP servers or like RAGs, uh, knowledge graphs on top of their institutional knowledge and to the agent, right?

[8:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=513s) So the assumption is if we, if I can build all these MCP servers and give that agent, I don't need to work anything like it will do. But the thing is, uh, when you plugging this MCP servers, basically all this data coming out is mostly undeterministic, it's unreliable, and it's untested, right?

[8:54](https://www.youtube.com/watch?v=_QAVExf_1uw&t=534s) So especially in engineering, nobody does evals. Actually, it's, it's more like a data, uh, machine learning concept, but we don't do eval. So for me, if you pl-- Sorry.

[9:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=544s) Uh, uh, uh, if you plug, uh, an MCP or like a RAG and all, we see whether the output is coming or not, rather than is it really valuable actually?

[9:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=555s) Is it really solving the problem or not? Uh, that's the main problem that I see. B- I'm not saying pointing other people because I was that person. I was like, "Okay, let me build all MCP servers, plug in my institutional knowledge.

[9:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=567s) I'm gonna prove that point that agents can s- semi-autonomously can continue and f- uh, fill those Jira tickets and finish it," right? But every time when I build those MCP servers, ten percent, twenty percent, thirty percent of time it was accurate, but the rest of the time I was doing the data entry job for them, actually.

[9:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=586s) So I was filling the gaps, asking the questions. So basically, I'm doing more work than actually doing less work. Uh, so I think this is the main problem, and I, I actually was in this fourth stage where I literally started to write the domain context with handwritten, actually.

[10:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=604s) So okay, let me write everything and prove the point, but I got really exhausted, uh, of doing it.

[10:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=611s) Okay. So how-- I don't know how many can you relate, uh, with this pie chart. Uh, but most of the enterprise, uh, the institutional knowledge is kind of something like this.

[10:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=623s) So twenty percent if you see it's outdated, twenty percent it's unreliable, uh, twenty percent i-- uh, ten percent is al- always duplicated with different places. And the major problem is forty percent of the knowledge is always, uh, tribal knowledge, which means people know how things work.

[10:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=641s) So it's, it's never documented, actually. So in this situation of an enterprise and you build like a hundred MCP servers and plug into that monolith, it doesn't matter how many you build, it won't work because basically your whole institutional knowledge is a, is a monolith.

[10:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=658s) Uh, I think like, because you're all from an engineering background, so you already know, uh, the transformation of monolith legacy system to microservices, right? So in the same way, unless we break down that monolith knowledge base into some kind of a context blocks which are useful for agents, then only we can actually make it useful for them,

[11:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=679s) uh, for the agents, and actually make them semi-autonomously can actually do the tasks.

[11:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=684s) So that said, uh, uh, we are going to talk in this workshop mostly on that monolith, how to break it, uh, what is the approach to break it, and how it is useful, uh, when once we break it.

[11:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=696s) And this is a job we need to do because, uh, uh, the LLM providers will, uh, focus on the, uh, LLM, uh, model quality. The agents will focus on the harness things, and there is a big re- uh, uh, retrieval market of nine billion.

[11:52](https://www.youtube.com/watch?v=_QAVExf_1uw&t=712s) They're focusing on retrieval. But nobody is going to come to your company and fix your knowledge base. You have to fix it yourself, right? So how can we do it?

[11:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=719s) Uh, okay. So the demand-driven context, uh, is what the other solution, uh, I was trying to propose, right? So basically, if I have to give an abstract of it, what it is, uh, like we have mic-- uh, monolith Uh, services, and we have this process of breaking them into microservices.

[12:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=740s) We have waterfall model, which we transform into agile. In the same way, when you have a monolith of institutional knowledge, how do you transform into a context blocks using an approach?

[12:30](https://www.youtube.com/watch?v=_QAVExf_1uw&t=750s) So this is an approach of how we can do it.

[12:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=754s) Uh, before starting, uh, just not an idea. So, uh, we already tried with some datasets and tried to prove this approach works. And, uh, in the March we have published a preprint, uh, uh, in arXiv.

[12:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=767s) So if anyone interested in reading a academic papers, uh, you can find it, uh, with the Demand-Driven Context, or like I can also give you a link after the, uh, workshop.

[12:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=779s) Okay. So how does it work actually? Uh,

[13:06](https://www.youtube.com/watch?v=_QAVExf_1uw&t=786s) when we are giving institutional knowledge to agents, basically what we are trying to do is we are trying to do a push strategy, right? So we build everything and we push it to, uh, to it.

[13:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=796s) So in, in this approach, it's more pull approach, uh, which means, for example, let's say, uh, a new joinee has joined your company, right? How do you onboard a person?

[13:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=806s) So you onboard them, uh, for, uh, one, two days. You give some initial orientation, and then you tell them like, "Okay, these are the Confluence links. This is the GitHub.

[13:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=814s) This is the some, some kind of a documentation. You have to follow things and all." Then you just assign a task to the person. So but you're not gonna tell, "Okay, go and gr- get graduated on, on this knowledge and come back, then I will give you work," right?

[13:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=828s) So you'll just assign the work item. And when you assign the work item, the person will start asking questions, fill the gaps. If the person is very much into documentation, he will also fill the documentation for you.

[14:00](https://www.youtube.com/watch?v=_QAVExf_1uw&t=840s) Uh, he gradually, uh, get his knowledge of, of the institutional knowledge, right? In the same way, we don't push all the knowledge to the agent. Rather than we start giving problems to agents, like work items, and let them actually pull the information from us.

[14:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=857s) And once, uh, pull the information, also ask them to document it,

[14:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=864s) uh, in, in a, in a better way, rather than in a monolithic structure. So if you s-- So that's the four layers. So you have a monolith, uh, a framework, and it actually pulls and actually creates a good better context, uh, blocks.

[14:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=878s) You can actually relate it to more into a, a legacy monolith to microservices directly if you have to have an, a knowledge of it. So this is how it works.

[14:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=888s) So this is one cycle of, uh, how...

[14:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=895s) The problem to an agent, and the, in the first attempt, the agent will fail to do it. So it will say, "You know what? You gave me a problem, but most of the, uh, documentation, I couldn't able to find anything.

[15:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=907s) I couldn't able to do it. Then these are the things I need to do [coughs] to finish this task," and it gives a checklist of things. So we fulfill the checklist, like we, uh, fulfill the checklist.

[15:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=919s) So once it is, uh, given, the problem is solved, it will take that knowledge, and also it will update. That means curate the knowledge in a particular place so that it can reuse or like other agents can also reuse.

[15:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=932s) This is one cycle. So the idea is if we can do it in multiple, uh, sessions with multiple problems, so it will gradually, uh, curate your knowledge, monolithic knowledge base, uh, and also document it for you.

[15:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=949s) Uh, you can also relate it to TDD. So how many are g- uh, do TDD? Or like... Nobody hates TDD, right? Before, before I, uh- [inaudible] Yeah. [chuckles] Okay. Okay. So, uh, in, in the same way, right?

[16:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=963s) In, in a TDD approach, what we do, we just write the fail test cases. We don't build the product, first of all. We just write the fail test cases.

[16:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=970s) Uh, we see what is the code that is missing, uh, for the fail test case to pass, and we just give that code. And we gradually build a, a product based on the fail test cases.

[16:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=980s) In the same way, we give problems that agent will definitely fail, and we gradually, uh, fill those gaps. And at a certain point, it becomes semi-autonomous with a good, uh, institutional knowledge already.

[16:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=995s) Okay. So I think I can jump into, uh, some kind of a, a demo already. Uh, so I will use terminal, so don't hate me. Uh, I, I think like all of you are from engineering background, so I think like you will like terminal.

[16:54](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1014s) Uh, let me switch to terminal. Okay. Okay.

[17:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1031s) So on the, on the far left, what, uh, right, what you see is how, uh, under the hood it works actually. So when you have given a problem, uh, how does the agent will fail?

[17:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1043s) How does it demand for the knowledge, uh, that the problem has to be solved? And a human, like a domain expert and all, f- uh, fills those gaps. And then it will curate, uh, a new knowledge base for you, which is, which is much better.

[17:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1058s) And then the agent succeeds, and you repeat on the next problems. So that is how one cycle, uh, uh, things are done. So how I-- it can be implemented, it can be implemented, uh, using any agent.

[17:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1069s) There is no, uh, it can be implemented on Claude or Copilot because it's an approach. You can, uh, do, uh, in any way you want. At work, I use Copilot, uh, so I implemented this way using Copilot.

[18:01](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1081s) Uh, but because everybody, I, I believe, loves more Claude Code, so, uh, I created this demo with Claude Code. Uh, and you can see it's just a combination of skills, rules, uh, agents, and hooks, and some kind of a, a place to save the knowledge base.

[18:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1097s) On the middle pane, what you're seeing on the top is your monolith, basically. Uh, this is a representation of your Confluence, Slack, GitHub, and all, but just for the sake of demo, I just put some flat files that look like them.

[18:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1112s) Uh, so that is how your monolith, uh, knowledge base will look like. On the down, what you're seeing is on a live. So when it's solving a problem, how it is actually adding the new knowledge, uh, to it.

[18:43](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1123s) So this is how, uh, uh... So let me...

[18:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1129s) So what I'm gonna do is I'm going to go to an agent.

[18:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1135s) Okay.

[19:12](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1152s) I'm gonna basically give an incident problem to do the root cause analysis, right? So, okay, what I did is... Uh, you remember in the previous slide, there is a Jira ticket samples that I showed, right?

[19:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1162s) It's a combination of, uh, uh, knowledge that is documented, not documented things and all. So this incident also represent the same kind of, uh, combination. So there is some knowledge that is documented already on your monolith.

[19:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1175s) Some it is not there or outdated or things like that. And most of it, it doesn't, couldn't, uh, wouldn't able to find because it's never written down actually. So when I gave this problem, so it uses those skills that I have developed using this approach, and it will try to actually first go to your, uh, monolith actually

[19:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1196s) on the knowledge base and try to find information on what is, uh, there. So,

[20:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1202s) uh, think about it like this. So, uh, first part is retrieval. That means it's already doing what RAG and MCP is doing, first part. But what else it is doing is, after it fetches the data, what it will do with the data actually.

[20:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1216s) So that is a missing part. For example, when you give, uh, new Confluence links to a new employee, the employee goes there, looks into it, but doesn't find information.

[20:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1225s) But it-- he doesn't stop there actually. He continue asking questions so-- uh, to solving the problem then, then just adding more knowledge and things and all, right? Those are the next steps missing right now.

[20:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1237s) It's-- We just stop at retrieval. So this is the next three steps it does actually. So you can see the confidence score is almost 1.5 because it says

[20:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1250s) these are the particular terminologies. I don't understand actually these terminologies and these business logics, uh, is not needed. So one thing you need to look at here is whatever it has said, this is the undocumented information.

[21:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1264s) That means it was never written down. So unless you don't do this way, you will never know w-what is not documented. For example, if somebody says like, "Okay, there is documentation missing, we need to write."

[21:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1275s) Okay, what do you want me to write actually? So tho-- there is so much in the people's head, I can't write so much, right? So-somehow it has to surface.

[21:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1283s) So when you give a problem, it actually surfaces what is not documented. And it tells me, "Okay, this is missing. I need to have a new information there." Uh, so what I will do is, so it does all the three steps.

[21:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1296s) So then what I will do is, I already have a pre-prepared answer.

[21:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1309s) Very high level prepare, uh, answer I gave it, to it, uh, of like what is the missing information. So, okay, this is the missing information you asked me, uh, to solve this problem.

[21:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1319s) Can you solve this problem now? Uh, I didn't expected this one. Okay. Uh, notifications is... Yes. I'll just say yes.

[22:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1334s) It asked also what fictional name should I use for it.

[22:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1337s) Ah, okay. I didn't see. See, when I did the test run, it didn't ask me the questions.

[22:21](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1341s) Yeah.

[22:21](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1341s) Yeah. Let's see. It knows it is a demo and-

[22:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1343s) It's smart for- [laughs]

[22:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1345s) You know, to... I trained it too well, I guess. Okay.

[22:30](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1350s) No, it is already-- What it is doing is already...

[22:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1356s) So you can see on the live, it is a-already adding the entities that, uh, the new knowledge base has been come into the place.

[22:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1368s) So the knowledge base is, uh, managed as a file system, right? As a, as a system files.

[22:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1373s) For the demo, I'm just showing it as a file system, but it's basically your MCP servers, uh, uh, the data will stay in Confluence, Slack, or things. Uh, you can just plug in and use the same MCP servers or RAG and all.

[23:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1388s) So it no need to be a f- uh, flat file.

[23:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1390s) So you treat this as your persistence layer for, for this agent, right?

[23:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1395s) Uh-

[23:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1395s) Just like a mem... Do, do you use any like a memory tool for, for this?

[23:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1400s) Yeah. I, I will show you on the next slide like how I'm going to save, uh...

[23:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1409s) Okay. So it started from fifty-six entities or something with the, uh, this one, right? Now, one problem actually surfaced six entities that are never been documented, and when I gave that information to it, it is able to actually discover, curate another five or six, uh, new entities that were never documented.

[23:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1429s) So it does discovery of the gaps. It also gets information from me and also stores information, new information and all.

[23:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1439s) Uh, this is one. Okay. S- Next, let's see.

[24:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1445s) This is a busy window. I tried to actually do things, but, uh, it didn't worked out. Okay. So what you are seeing on the, on the window is like fourteen incidents.

[24:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1455s) You have seen one problem that I solved with an agent, right? The communication. What if I took like fourteen incidents, uh, and I just go and have fourteen cycles of this thing, and how it does?

[24:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1466s) So if you see on the left side, it was the first incident, right? So right now it has- 1.5 confidence and everything is critical. Every-- So basically, nothing is documented.

[24:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1478s) So everything is critical, high, uh, the data is missing. So I started giving answers to-- on the first incident, then I repeated for the same second and third and continuously for, like, 14 incidents.

[24:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1489s) But on the 14 incidents, it basically actually able to go to a confidence level of 4.4 because first it discovered on every incident, it got the list of answers for me, and also it documented everything for me.

[25:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1503s) So it gradually, from 1.4 to almost like five range of knowledge, uh, it improved. Uh, so if you look at the traditional way, in traditional way, what we do is we solve all the context problem, right?

[25:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1516s) We have to deal with it first, then I have to give it to agent. In this one, we are moving agent from a consumer to a knowledge manager. So you just don't consume from me.

[25:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1526s) I'm gonna tell you, but the whole knowledge management is also your job and you have to, uh, do it for me.

[25:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1541s) Okay. I think we can get back to the slides a bit. Okay. So what we have seen is, uh, we have-- I have run one cycle and also have s- shown w- how it, uh, look like when I run in like 15 or 16 different cycles, right?

[25:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1558s) But if you ha- want to do it manually, it would be really painful because [chuckles] I tried it after 15 cycles. Like, uh, nobody would like to actually sit with an agent and, uh, you know, you have an incident, but you won't be sit with your agent and keep actually asking questions and telling you about your problems, right?

[26:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1574s) So, uh, that is super painful. Uh, so but the thing is, we can automate this process. So this is where actually it's, it's really good and gets interesting. So here is the thing.

[26:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1587s) You all-- We already have all the work items, right? We have Jira, we have incidents, uh, we have, uh, customer support tickets like that. All those kind of a work items already there, right?

[26:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1599s) Sitting in the archive. So why can't we take, uh, them and actually use the framework, uh, and, uh, validate across your monolith database, run an automation, and see actually what is the state of your content actually right now?

[27:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1622s) Okay. Let me see. Let me show you how it looks like. So

[27:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1627s) rather than actually doing it manually, at a scale, if we do this approach, so how does it look like? So the demo that you're seeing is almost like everything is preset.

[27:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1637s) Uh, for example, I have the demo. I have like a platform operations agent and, uh, I'm saying, okay, these are the recent incidents. Let's say I have 20 recent past incidents I have, uh, like an MD file or a JSON file, right?

[27:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1651s) It has all the details of description of it, things and all, uh, comments and everything. And the rest of the files are your knowledge base. So it's, it's a file system, but you can also actually connect with the same way, the Confluence and things and all.

[27:44](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1664s) Just for a demo purpose, I'm just showing it as a flat files. Now, what I'm trying to do is, I'm gonna take all those incidents and validate each incident across my knowledge base and ask the agent, "Okay, tell me, uh, how much of the document is good, how much of the documentation is I can't trust it or

[28:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1684s) like old or outdated, and how much is actually missing, not documented as per these incidents?" So let me

[28:13](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1693s) run it. Okay, it will take some time. So it will take three steps actually. So one is, uh, it generates probes, which means a basic test it will write to actually test your knowledge.

[28:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1705s) Uh, then it will run those tests and then analyze the gaps actually.

[28:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1716s) Okay. It's a little bit hot in the room actually. [sighs]

[28:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1727s) Before identifying the gaps, it's just like a clever prompt, I imagine. [lip smack]

[28:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1733s) Uh, okay. For example, let's, let's say you have an incident called the notification service is not sending, uh, customer, uh, messages to,

[29:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1743s) uh, the SMS service, right? So the notification service then you mention that the agent sees, is there a documentation related to notification service?

[29:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1751s) Mm-hmm.

[29:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1751s) So it doesn't find, that means you never wrote a documentation on notification service. I do understand what is a customer, uh, SMSs things and all. The customer notification service when you mention, it's a gap because it's never documented.

[29:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1764s) Or it takes the customer notification service, goes to Confluence and sees like the documentation, how old it is. If it is, say, it's like, uh, it's like [REDACTED:age], it will tell you, "Look, I looked into it, it's like [REDACTED:age].

[29:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1777s) I don't know whether I need to trust this documentation or not," or like incomplete, uh, documentation. So if you see, it scored each incident it took it, and it looked at all the knowledge, uh, base that you have connected and have a consolidated list of like scoring of like, okay, partially the agents can handle, uh, the basic

[29:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1798s) edge cases of the incidents that you give, uh, because your knowledge base is not complete and it will show you how much of the tribal knowledge is missing. System information, business process, what are actually are missing from your, uh, uh, institutional knowledge when whatever is not documented.

[30:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1814s) Uh, these are the probes. And it will also identify, uh, what is critical and what is high. This is really important because let's say there is some kind of an, um, example of notification service which I mentioned, right?

[30:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1834s) It is repeatedly, uh, appearing in like 20 incidents and you don't have-

[30:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1841s) This is the first that you need, you need to fix as per your documentation. So, uh, it will also help us actually understand when you're, uh, uh, breaking down your, uh, knowledge base, you need to understand what is critical actually, what I need to focus on first, what makes value for me.

[30:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1859s) So we'll organize into critical, high, medium. So this is what, like, uh, I showed the flat files, but you can also connect it to the various data sources that we have.

[31:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1875s) So the step one is basically what it does is demand extraction. That means every incident, it will extract, uh, the checklist of information, what is missing.

[31:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1886s) On the second step is what it will consolidate everything what is missing. So it will create like systems and APIs and all, and how many are clean, how many are stale, uh, which is incomplete, what is entirely missing, uh, something is tribal.

[31:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1902s) Uh, those kind of a classification it will also do.

[31:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1907s) And it will create a Kanban board for you. So what happens is, so if you want to fix your institutional knowledge base, basically, you just, just like Jira tickets, we finish it.

[31:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1916s) We actually has to document these missing pieces and all. And the, the moment you started to-- So it also saves in the context lake, so it also has to build its own, uh, uh, knowledge base.

[32:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1928s) And you can see the performance. So once you're fixing those tickets on the Kanban, uh,

[32:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1942s) the institutional knowledge. So that how-- So what we've seen is, one is the approach, first of all, which means not the pull, push approach, but the pull approach, how to do it.

[32:40](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1960s) One cycle or multiple cycle, how it look like. But if you put it in a scale of automation, then how much valuable it, uh, would be. [object clattering]

[32:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1975s) Okay. Now, uh, uh, the important question is-

[32:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1979s) Let's have another mic on you. Sorry about this.

[33:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1983s) Sure.

[33:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1984s) Double mic. Yeah, it's cutting out a bit, so I'm just gonna put that one on as well.

[33:09](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1989s) Should I do a voiceover later, actually?

[33:12](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1992s) Can you hear? Working well.

[33:13](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1993s) I think you need to repeat that from, [laughing] from the beginning.

[33:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=1997s) I have the patience. Who has the patience for a second round? [laughs]

[33:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2002s) So the question is, so I was all, uh, all the time I was talking about, okay, it, uh, receives the context, we give the information, it will store it.

[33:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2012s) But the question is: Where does it actually store it? So I have a very opinionated opinion, uh, hear me out. Uh, I prefer, uh, it has to go to a GitHub repository, uh, because eventually somebody will actually come up with a, you know, 20 million seed funded, uh, SaaS solution for you.

[33:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2029s) Uh, but before that, I prefer, uh, to actually put it in GitHub as a repository. Why? Because if you look at it a scale, if you want to do this, there will be multiple agents, multiple teams actually contributing to the same knowledge base, and there will be conflicts and resolutions, right?

[34:06](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2046s) So the, uh, the easiest way to do is using GitHub because it actually comes with inbuilt, uh, uh, PR processes, uh, review processes, things and all. So if multiple domain expert are sitting and uploading the files or like agents are contributing to it, the most efficient way to manage is in a GitHub, something like a structure like

[34:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2067s) this. And the other advantage is also if you put it on GitHub, you can also publish it to Confluence later or like Slack la- uh, wherever you want to publish it to under the, uh, solution that you want to use.

[34:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2079s) Uh, so I prefer to have it on GitHub, but if you want to directly integrate it to Confluence and all, you can also and so do it.

[34:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2087s) Next is, uh, a meta-model. Uh, how many are aware of the word meta-model and the... Okay, maybe I can quickly show you how does it look like.

[35:01](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2101s) So meta-model is basically something like this, right? So, and, uh, how does your, uh, uh, domain actually structured around? Like, uh, is a business process or, uh, how it is related to systems?

[35:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2114s) How systems are related to, uh, an APIs? And how is this, uh, business jargon or like tech jargons are actually linked to which one? So these kind of a relationship, uh, meta-model is really important.

[35:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2127s) It's not necessary, uh, for the approach that I have proposed, but it's an add-on. And why, uh, you need to have this one is right now, think of it as like a map.

[35:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2138s) Right now, your agents doesn't have any map actually to navigate with your, uh, knowledge base. Basically, what you're doing is you're dumping like, uh, these many number of files, and it need to figure out which file I need to need, right?

[35:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2149s) But your file structure is actually a representation of your meta-model. It actually knows how to navigate. For example, let's say, can you fix this system? It will understand if I ch- make changes, which business processes will be affected and which APIs I need to change or like touch thing, these kind of things.

[36:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2167s) So it's also important to have a meta-model. If you have it,

[36:12](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2172s) uh, then it will produce more value. So I strongly prefer to have a meta-model along with this approach.

[36:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2186s) Okay. So the last part, uh, is what is the value it created? So, uh, there's a lot of slides that you have seen, a lot of demos that you've seen.

[36:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2196s) So personally, I need to also share like what is the value that I see when we, uh, I was using it or like, uh, the other people who I shared with already were using it, uh, came back with the feed- uh, feedback and told me.

[36:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2208s) Uh, first, the most valuable thing is Knowing the unknown. So what is never documented is something can be surfaced only by this approach actually. Uh, otherwise, you will just end up in, um, an endless Miro board of like, uh, putting tickets on, okay, this is missing, this is missing, I need to add it, I need to add

[37:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2227s) it, and, uh, keep on doing it. So this is the fastest and better way to discover, uh, with your previous work items and all, what is never documented, uh, things and all.

[37:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2236s) Uh, second is, uh, basically, I can now give work to agents rather than I do all those things. Like, uh, rather than I becom-- I give the agents all this information, let it manage my knowledge management.

[37:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2252s) I don't want to be the knowledge manager of it, so let, let it do it. So those are the two big values that I have seen. If you want to u-use it, I think like you will also see the-- those two as the most valuable.

[37:44](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2264s) Uh, but these are the other things, uh, what I seen.

[37:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2271s) Now, okay, so I also need to, uh, tell you like what is the, uh, drawbacks of also using it, right? First of all, if you are coming from a small team or like if you say like, "No, no, no, my documentation, my knowledge base is really good," I'm like super happy for you.

[38:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2287s) Uh, you are the lucky ones in this world right now with agents. Uh, for you, it might not be really relevant unless you have a very, uh, very complicated, uh, uh, documentation that you have.

[38:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2300s) Uh, second is, I already mentioned, the manual-- manually doing is, it's very painful. I don't prefer anyone to do it. If you want to just try it for testing purposes, you can also do it.

[38:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2311s) But, uh, automation is the most best way to actually use this one.

[38:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2316s) Uh, this is very early, this approach, so by tomorrow morning on YouTube, somebody would have already posted something differently, uh, better than me [chuckles]. So, uh, in the A-- uh, in the, uh, era of AI, nobody knows like, uh, how long a thesis or an approach or an app or product going to survive.

[38:54](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2334s) So for now, I see this is the best approach.

[38:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2339s) Okay, so the whole workshop, so we started with, uh, o-one pipeline, right, on the ROI. And so the demand-driven context actually sits between this monolith and also the retrieval layer actually.

[39:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2355s) And what it does is, uh, it actually helps you build curated context blocks for you. You can also think of it like a, uh, cache database that you have.

[39:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2366s) So every time your agent doesn't need to go and, you know, s-boil the ocean for, uh, fixing an issue, rather than if you have a good context block of information, most of the time, eighty percent of the time that can be usable.

[39:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2379s) Because what I also believe is, it's always the eighty/twenty percent rule. So twenty percent of your documentation is most useful, eighty percent is some corner cases you have to look into it.

[39:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2390s) So rather than giving hundred percent of, uh, things, you need to figure out what is my twenty percent of that, uh, that is super helpful for agent and have it like a cache database, uh, the context block of it using it.

[40:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2402s) And rest of it, you can leave it like, uh, links. So whenever agent feels I need more information, then only it can go and check the, the whole, uh, monolithical, uh, institutional knowledge.

[40:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2415s) Okay. So from here, what you can take from this workshop is three things. One, uh, I hope, uh, I makes-- I made sense of this approach. So, uh, there is a GitHub repo, uh, uh, which I detailed it out and also a starter guide on it one, if you want to go home and try with it, you

[40:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2434s) can, you can try it. Uh, you already have know how the framework works, so you want to go home and just remix the whole approach, you can do it and let me also know.

[40:43](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2443s) Uh, I'll leave this one and I'll join with you for contribution. Uh, you have a context ga-gap scanner that I showed you, which is live already with presets. Uh, I think I added like twenty dollars on it, so hit it as much as possible as well.

[40:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2457s) Two hundred, right? Okay. Okay. So after twenty dollars, so, uh, first come, first served [chuckles].

[41:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2464s) Uh, so all these three you can use, uh, you can take away from this workshop.

[41:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2471s) Okay. So because this is a workshop, so I also would like to

[41:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2477s) want you to try something. Uh, what you can try is three things. One is either, uh, if, if you say like, "You know what? I'm so, so tired already.

[41:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2488s) It's almost like four, uh, it's almost about to go for a party. I don't wanna do it." So you can just go to the context gap scanner. Uh, everything is a preset here.

[41:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2497s) You can just try it out, hit it, and see how it works. If you think it can be done better, let me know so that we can work together.

[41:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2506s) Uh, or otherwise, uh, let's say, "No, I've-- I'm very technical, I want to know how it works under the hood." Uh, this is a GitHub repository. Uh, it's, it's under, uh, maybe I'll just take this out.

[42:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2525s) Uh, this is a GitHub repository, and it has all the information. Uh, plus there is a starter guide also if you wanna, if you wanna try it out. But, uh, if you still feel like, "No, I want much more simpler,"

[42:30](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2550s) you can also try this one. So you don't need to do anything. Basically, take this prompt, uh, take one of your Jira ticket or incident that you have right now.

[42:40](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2560s) If you already built MCP servers, uh, or like, uh, any other kind of a things, you just use that, uh, prompt, give it to your agent, uh, uh, with the incident or a Jira ticket and ask it Uh, give me the quality of the knowledge base that I have as per this incident or, uh, Jira ticket in

[42:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2579s) this way, and see how man- how much of it comes in the red, which is never documented. So you can, you can try also the simple one.

[43:09](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2589s) I'll just leave it like this. Maybe you can take a picture. Maybe I can switch to the slide if any- any- anyone want to.

[43:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2603s) Cool.

[43:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2609s) So while there's time for the hands-on with scanner, yeah.

[43:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2613s) This is yours line. You can-

[43:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2615s) Yeah. So I see some drawbacks in this approach.

[43:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2618s) All right.

[43:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2619s) And I find it very interesting. So my f- my first question is, have you already used this way of working at scale? Or... Because we've seen mostly toy examples, right?

[43:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2627s) Yep, yep.

[43:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2628s) In your, uh, slides so far.

[43:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2629s) Uh, I used it, uh, not at a scale. Uh, I started a bit simpler because you also need to see what is the scope of it. Let's say I have an enterprise, and I try it an enterprise level, I can't do it because it's multiple domains things and all.

[44:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2644s) Even if I do it at domain level, I need to understand. I tried it at domain level. Then even at a domain level, there is so much of a domain expertise I need to fill it up and, uh, uh, fill those gaps.

[44:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2656s) So again, cut down into maybe what is the smallest team that I have and the smallest teams Jira tickets, the smallest teams incidents, and the teams Confluence page, uh, with a bit of a scope.

[44:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2669s) Then if I s- drill down the scope, then I feel like it's more, uh, fast, more useful.

[44:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2675s) Mm-hmm.

[44:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2675s) But if I do it at a bigger scope, what happens is not one person has the whole, uh, domain expertise. So basically it again becomes like, uh, somebody has to come and, uh, you know, five or six people has to sit down and, uh, start doing these things.

[44:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2690s) Yeah. I'm, I'm a bit concerned that this might, uh, denial of service attack your, your team members-

[44:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2695s) Mm-hmm

[44:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2695s) ... in a certain way because our LLMs are fine-tuned to, uh, keep eliciting information, to keep, uh, getting more information out of us to ask follow-up questions.

[45:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2705s) Uh-huh. Okay.

[45:06](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2706s) Um, so I think it will be hard on the engineers that have to do the question answering. Um, and secondly, the, the scanner is nice, but that still builds on, uh, the assumption that all of your team members and the rest of the enterprise-

[45:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2718s) Mm-hmm

[45:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2719s) ... are still using your enterprise IT, well, as planned, that they're actually filling in their tickets with all the details and-

[45:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2726s) Yep

[45:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2726s) ... et cetera. And I know from practice that it is most of the time not the case.

[45:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2731s) Uh, that is true. That is true. I agree with you. Uh, even if I go-- My, my assumption is also even if I go to a leadership to buy in, like, "Hey, can you give me a bandwidth?"

[45:40](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2740s) Or like, uh, you know, I need these people to actually sit and fix the context. I don't think right at this point of time nobody will do. But I think it will happen because, uh, slowly, I think we are slowly moving towards an agent managers where, uh, agents are becoming semi-autonomous or autonomous, and we manage them.

[46:00](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2760s) But at that certain point of time, somebody has to fix that knowledge because it's not gonna come from anywhere. You have to. So then the enterprise focus will shift towards the gap.

[46:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2770s) Uh, that's what I started saying. I don't think nobody is looking into the problem yet. Uh, everybody is very focused with agent, how good the agent is, how good the retrieval is, but how good the context is, uh, y- y- you're not solving.

[46:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2786s) It to-- I think like in, down the line in a year or so, I think people will realize the importance of it and, uh, the Kanban board will definitely come into reality actually very soon.

[46:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2797s) Thank you.

[46:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2798s) Yep. Thanks. Yep.

[46:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2799s) Yeah. I think actually building on the same points, I think when we look at large enterprise, actually the s- source of truth is not actually the documentation, it's actually the code.

[46:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2808s) Just wondering, have you applied it to the code base?

[46:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2811s) Um, I did. Uh, I also applied the code base, uh, but I got a mixed result when I, uh... So, so the-- here is the thing. What happened is when I only use code base, uh, it is particularly good, or when I only use Confluence or like textual data, uh, like, uh, uh, it gives good results.

[47:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2831s) But when I combine it, somehow actually, uh, it conflicts because it, it creates a theory out of the GitHub repository, but the same GitHub repository documentation is also on Confluence.

[47:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2844s) So there it gets a conflict of, okay, what is the source of truth? Code says this. Should I implement it this way as per the documentation? So then aga- again, I need to create an additional skill or rules like, okay, what is the ranking that you need to give?

[47:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2857s) If you see it in GitHub, that means that is the source of truth. Or if you see it-- If you don't see it, then you have to, uh, look the information in Confluence things and all.

[47:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2868s) But those are still... I'm trying to fix those things actually. So seeing the gaps and fix those things. But I definitely see, uh, that issue combining those two.

[47:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2877s) And the second question is, um, quite interestingly, has actually applied the same approach with skills. Because what we find out is actually like you have your, um, kind of like your, um, like the, the process starts like by running agents, which is bring in a context and identify the right skills that need to be used, right?

[48:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2895s) Mm-hmm.

[48:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2895s) Then you go do the task and you fail-

[48:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2897s) Mm-hmm

[48:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2897s) ... I think, right? The moment you fail, you identify what it you need to solve.

[48:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2900s) Mm-hmm.

[48:21](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2901s) You go back, you curate, you, you fix, and then you're fixing the knowledge kind of base. Once we find out actually if we go back and fix the skill-

[48:30](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2910s) Okay

[48:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2911s) ... that increments kind of the, the next iteration. Sorry. So I'm not sure if skill-- 'cause I saw you had some skill, but-

[48:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2917s) Yep

[48:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2917s) ... is skills part of the, uh, the iteration loop or how's it-

[48:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2921s) Uh, I think right now skill that I have built is static, but what you are more proposing, if I'm not wrong, it's like se- evolving skill, right? If the skill fails, it has to evolve, right.

[48:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2931s) Uh, I agree with you. I never tried it, but I think like it has to be, uh, like that because I'm also more concentrating on, um, how to do it at scale.

[48:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2939s) Uh, the reason is also Uh, I, I want, I want the context to be fixed before retrieval itself,

[49:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2948s) not during operational. So first, when I started with it, I, I started doing with when operational, which means, oh, I have a work item, I will assign to it, it will fail, then I'll start giving context and all.

[49:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2960s) But it takes a lot of time, it takes a lot of patience for me. So rather than doing it, you know what? I'm gonna fix the context, but before retrieval.

[49:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2968s) So if I can, uh, uh, uh, while I was, uh, answering her question, if you take a team, the, the context that you need to fix is very small.

[49:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2977s) So you can use a, uh, context gap scanner, uh, kind of a thing. And maybe if you're good, have a good domain expert, I think like couple of weeks you can actually fix your documentation.

[49:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=2989s) Not like 100%, at least like 60, 70, 80% of a good quality that you can al-already build it. So my proposal would always be don't do it at an operational level, uh, at an, a real-time level, but do it before retrieval, uh, itself.

[50:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3004s) That is much better than this approach. Uh, yep.

[50:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3008s) I, I have a question. The retrieval part, like, especially at scale, if you have a lot of documentation and that documentation sitting in a GitHub repo. You may have situations where you ask questions that may need, I don't know, like five or six different docs.

[50:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3022s) So you will have like LLM going there, reading all the docs. Like, this takes time for retrieval of information, and that takes a lot of context also for the-

[50:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3033s) Uh-

[50:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3033s) ... LLM to get the right information

[50:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3035s) ... right now, after Claude Code announced one million of, uh, tokens in the context window, I don't... No, I don't have any problem. So I calculated it. Uh, at an average, it's like, uh, 96K tokens, because I tried with different domains actually.

[50:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3049s) Per domain, I see like around 96K tokens, uh, if I consolidate everything like Confluence things and all. Uh, so easily fits in the context window actually. Uh, I tried to do some experimentation around, you know, a graph RAG, put them there, rather than just take all the files, use a graph RAG, understand the intent.

[51:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3067s) But for me, just putting the whole context right now in the window, uh, gives you more results than actually doing, uh, uh, RAG. Unless you have a very big, uh, almost around a million tokens of a context that you want to fit in, maybe then you have to use a bit more, uh, retrieval mechanisms between it.

[51:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3088s) But otherwise, I think like, uh, it should be fine.

[51:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3092s) I, I have a question. Uh, I opened your paper, and could you explain this graph, like, uh, comparison between different techniques like domain knowledge, strategy knowledge, axis, um...

[51:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3107s) Uh, which one? Yep, sure.

[51:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3109s) This one.

[51:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3111s) Okay. So, uh, I also did the citations from other papers.

[51:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3115s) Okay.

[51:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3115s) Uh, so not directly related, but, uh, you have the paper of ACE, uh, uh, which is also, uh, does a similar thing. So, uh, but ACE is not exactly into, uh,

[52:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3130s) uh, how do you say, uh, discovery and curation actually. Uh, if I remember it correctly, uh, maybe I need to refresh my memory.

[52:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3138s) What do you, what do you mean between, uh... The difference between domain knowledge and strategy knowledge in here?

[52:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3144s) Okay. So s-strategic knowledge... Okay, so what ACE and all, uh, are doing is, when you are trying to have a conversation with, uh, AI, uh, you can see in the Claude Code and all, it updates its, uh, memory or like the relationship with you or things like that, right?

[52:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3162s) Mm-hmm.

[52:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3162s) So, and also from the chat history, it understands what is the most important context I need to remember, those kind of things. So when you are in communication with it, that operational conversations with AI improvement, they propose.

[52:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3175s) So what I, I, um, my proposal is not based upon your conversation with an AI, but rather your domain knowledge which is documented actually.

[53:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3188s) Mm-hmm. Okay.

[53:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3190s) Uh, somebody else has a question. Sorry.

[53:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3194s) Um, when, uh, relating to remote knowledge, uh, things on Confluence-

[53:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3197s) Mm-hmm

[53:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3198s) ... and other stuff, um, how do you ensure that your agent only points to the updated or like the, the, the relevant documentation as-

[53:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3208s) O-

[53:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3208s) ... because in your local file system, you, you have tags like outdated-

[53:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3211s) Mm-hmm

[53:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3211s) ... or piece of that, but remotely that's harder to tag, I guess. Right? So...

[53:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3215s) Okay. So when I wrote a pipeline for extracting from Confluence, it also allows actually to give you a date and also last updated, who created kind of an analytics on, on the space.

[53:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3225s) So you can use it, uh, to actually put a threshold of like, okay, on this particular date, w-whatever is, is old, consider it as an outdated one. And let me know.

[53:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3236s) Don't just consider it as an outdated one. You let me know, because sometimes the document can be stale for so long, but it could be an important document actually.

[54:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3244s) So it lets you know, but not like take decisions actually right now on this one. So you decide, uh, which one is stale and which one is not.

[54:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3251s) You, you don't have like an intermediate layer where you... In the repo you store this is still available.

[54:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3257s) Okay. So, uh, when it is curating the context, uh, also it updates with the, uh, date and also the state of the document, like stale, active, and clean. So it also looks into, okay, this is stale, I know I'm not gonna touch it, and I'll just go to just look for any other new other documents are there,

[54:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3273s) uh, in this one.

[54:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3274s) Great. Thanks.

[54:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3276s) Uh, did you think about, uh, how to manage access or permissions later to this knowledge? Like if you have some-

[54:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3282s) Mm-hmm

[54:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3282s) ... knowledge in the company that's sensitive and on- and only specific people can get access to it. Because currently I guess you just have all the knowledge and everything is accessible, but, uh...

[54:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3290s) Okay. So because it's not a product or a SaaS solution, it's just basically GitHub- For me right now, permissions and things are not difficult to implement because GitHub, uh, out of the box gives me who I can give the permission to as GitHub, who can have write, read access things and all, who can merge, uh, those things

[55:09](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3309s) and all. But in case if it evolves into a product, and for example, context, uh, gap scanner as a product, and I want you to test it because the, the reason why I was using presets, uh, for this, uh, workshop, not actually asking you to upload the files is because I don't want to take your IP data

[55:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3325s) on this one, right? So unless it becomes a product, you don't have any problem, uh, GitHub and all. But if you have a SaaS solution for this one, uh, then it's between how the SaaS solution will manage it ri- right now.

[55:39](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3339s) But the approach has nothing to do with the, uh, access things and all. How you implement those access on, on the knowledge base is up, up to you.

[55:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3348s) You have a question.

[55:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3350s) Yes. So this is of course about documentation, but did you give any consideration about using it on some like central tooling that a company would use? Like let's say that you have a platform team and you have a CLI that the different teams are using-

[56:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3365s) Mm-hmm

[56:05](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3365s) ... and so now it's used by different agents, right?

[56:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3368s) Okay.

[56:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3368s) And so the agents can also be like, "Well, this action is available for a resource, but I don't want to do five hundred calls just because I have a list of five hundred resource."

[56:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3376s) Okay.

[56:16](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3376s) It would be nice if the tool could do that. Uh, I don't know if, if you've given any consideration to this.

[56:21](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3381s) Uh, I think that is how it has to work. Uh, in an organization, you need to have a central solution for it. Uh, but how you want to do the solution is up to the organization.

[56:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3391s) For example, we are doing agile, right? So agile can do by Scrum, Kanban, or like Lean or something, and also you can do different apps to do it. The, the process is the same, but how you do it, which method you will choose, and which app that will you choose in your organization is different.

[56:47](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3407s) In the same way, what we have discussed is the approach. If you want to put it in the organization, you can use the approach, and you do-- you can do it in whichever way you want.

[56:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3419s) My point is more like, so with this you can identify gaps in your documentation-

[57:02](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3422s) Uh-huh

[57:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3423s) ... right? Could you use it to identify gaps on your tooling, so to say?

[57:09](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3429s) Okay. Uh, when you say tooling, it's the agentic-

[57:12](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3432s) Internal tools that, I don't know, maybe a team is building for the rest of the company.

[57:17](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3437s) Just infrastructure in general, right?

[57:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3438s) Maybe, yeah.

[57:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3440s) Uh, can you give me an example of, uh, like, uh, how it could-

[57:23](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3443s) Let's say that, uh, I don't know, you build some sort of abstraction on top of Kubernetes.

[57:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3447s) Uh-huh. Okay.

[57:28](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3448s) You don't want your developers to necessarily know what to do with that.

[57:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3451s) Okay.

[57:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3451s) And then you have a different CLI, or you, you have something, right? Um, but then like I say, I don't know, maybe you thought that they would list one of your custom applications or corporate applications-

[57:43](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3463s) Mm-hmm

[57:43](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3463s) ... one by one, but a team has grown into using more of that, and suddenly they have a lot, and they don't want to do that many calls, or perhaps even the agent is like, "Well, this is inefficient.

[57:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3473s) I would like this internal tool to work in a different way."

[57:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3476s) Okay.

[57:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3477s) And, and in that way, you would identify like gap in the tool or, or a performance improvement, kind of like these tasks for documentation.

[58:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3484s) Could be extended actually.

[58:06](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3486s) Mm.

[58:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3487s) So, uh, because, uh, we have seen the business processes also, right? So it can also document business process. Well, business process is nothing but how, uh, the process in the application it actually runs and does things, right?

[58:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3499s) So you can extend it to also find out the gaps in the business process or like how it works. It could be an extension to it.

[58:26](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3506s) Yeah. Okay. Yeah. Thank you.

[58:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3509s) How would you ensure that maintenance won't kill you?

[58:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3513s) Uh, sorry?

[58:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3514s) How would you ensure that maintenance won't kill you? Because the knowledge, the knowledge of a company changes with time.

[58:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3521s) Yeah.

[58:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3521s) So if the answer for a question today is B, tomorrow B1, on Friday it could be C, right?

[58:48](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3528s) Mm-hmm.

[58:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3529s) You have to-- at C, you have, you, you have to identify B1 and B and overwrite them. If it's stored as text, that's a problem.

[58:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3539s) Okay. So, uh, when I showed the context gap scanner, you also saw like an indicator of duplication, right? So if today you have a document, tomorrow you have version two point zero and something else, actually, it will find out the same information is having in three different...

[59:15](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3555s) It al- it also will find it. If you have only one, it is changed, it will take the latest updated one, because as a human, you changed it. So it will take it as a source of truth, right?

[59:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3565s) But if you have three versions of the same document, that's a duplication, and it will flag it as duplicate.

[59:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3573s) Right. But it, it's a search problem. How, how do you ensure it, it performs well? Let's say you have a document of one hundred thousand words.

[59:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3582s) Uh-huh.

[59:43](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3583s) You just say you just change a, a word like, uh, passport, let's say. It won't be there, but for example, right, it has changed.

[59:49](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3589s) Okay.

[59:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3590s) So you have to find the specific word, compare those three, in three documents-

[59:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3596s) Mm-hmm

[59:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3596s) ... check them and re-replace them, sort of, right? How it's feasible. Which tools would you use ma- to ensure it will, it will, it will not kill you cost-wise?

[1:00:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3608s) Um, I didn't quite get your question, actually. Uh, is it like the token usage you're worried about? Like, uh, the, that many tokens that we used? Uh, how, how is-- what is the cost saving, uh-

[1:00:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3620s) Yeah. Uh, uh, precisely. It, it, it will, it will grow-

[1:00:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3624s) Mm-hmm

[1:00:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3624s) ... presumably be, be big. And if there are a lot of changes, you would have to maintain, ma-maintain that whole database, right? So it has to be-

[1:00:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3631s) Mm, okay

[1:00:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3632s) ... some sort of structure. I don't know, maybe Graph or Ivy-

[1:00:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3635s) Okay

[1:00:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3635s) ... or whatever, right? Um, my question is, well, 'cause what you presented is sort of a happy path where you have a gap-

[1:00:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3645s) Mm-hmm

[1:00:45](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3645s) ... you fill it, and then you reuse it. But in a while, you will have a bigger problem where you pretend to have that gap filled, but actually it doesn't contain up-to-date information

[1:00:59](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3659s) and contains wrong information, right? So you want to prevent that

[1:01:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3663s) Ah, okay. Okay. Uh, so it can flag as per when it is created or the last updated. You can set such kind of a filters. But let's say you have a latest document which has a wrong information,

[1:01:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3679s) right?

[1:01:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3679s) But you don't know that, right?

[1:01:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3680s) No, that, that's true. Uh, but as, for example, as a human being, right? So you go and look into documentation. You told somebody to look into documentation, and the person looked into the documentation, and as per the documentation, this is being implemented in this way.

[1:01:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3693s) The person will do it, right? It's not an agent or a human issue.

[1:01:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3696s) But that's way too late, right? I mean-

[1:01:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3698s) Okay

[1:01:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3698s) ... you solved one ticket, one Jira ticket-

[1:01:40](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3700s) Mm-hmm

[1:01:40](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3700s) ... and the agent is trying to solve the second one. The assumption is the knowledge is pristine.

[1:01:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3706s) Okay.

[1:01:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3706s) So it's solving it, but the solution is wrong. So that's too late, right?

[1:01:52](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3712s) Okay.

[1:01:52](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3712s) So you say there is a scanner. Fine. Do you run it daily? How much will it cost you?

[1:01:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3718s) You, you, you can have another process spawned that would try to see, like, last updated-

[1:02:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3723s) Yeah

[1:02:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3724s) ... uh, try to see on some cadence, try to run a process that will up-update the knowledge.

[1:02:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3728s) Fantastic. How much will it cost you? Yeah.

[1:02:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3731s) Uh, for ex- for example, uh... Sorry, you were saying?

[1:02:14](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3734s) Less than, than doing s- more meetings, uh, every day to onboard someone or-

[1:02:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3739s) How, how-

[1:02:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3740s) ... update the knowledge

[1:02:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3740s) ... that's a strong claim.

[1:02:22](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3742s) But I don't think it will cost that much as is-

[1:02:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3744s) It's one premise of AI.

[1:02:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3745s) Yep. As I said, like, uh, when I tested it, uh, there is none of the domains which cross more than 100K tokens actually. So I don't think we will, for example, context gap scanner, right?

[1:02:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3757s) You-- I don't think, like, you have to do it, like, on a daily basis or anything. Even if you run daily basis, like, 100K tokens and do one scan.

[1:02:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3766s) For example, right, if you try to start hitting all of them, uh, all of you, the context gap scanner, I think you can't even burn, like, one, $1. I think so, if I'm not wrong.

[1:02:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3776s) It, it already had like... Oh, you're ready. Okay. Uh, okay. I'll, I'll cancel the subscription now. [laughs]

[1:03:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3783s) But I-

[1:03:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3783s) Because then I go back and depend two things. One is the complexity of organization you're trying to solve. Like a good example, like say airline-

[1:03:10](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3790s) Mm-hmm

[1:03:11](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3791s) ... industry. We're talking about like hundreds if not thousands of like repository system documentation third parties. So I agree with you. If you're in a s-specific domain-

[1:03:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3800s) Mm-hmm

[1:03:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3800s) ... solvable. The moment you scale, you will have to solve this, uh, cost question.

[1:03:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3804s) Okay.

[1:03:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3805s) Which is, which is gonna be depend also how fast the, um... To, to his point, like how fast the, the, the data change-

[1:03:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3812s) Mm-hmm

[1:03:32](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3812s) ... which I think most of the time not that much. So the moment you get to like 80%, ninety percent, you just continue kind of evolving the-

[1:03:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3818s) Yep

[1:03:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3818s) ... last one. Uh, I see it's different use case, different use case.

[1:03:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3822s) Yep, use case by use case. Yep. Yeah. Uh, any other questions?

[1:03:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3826s) Yeah. I was curious, so I, I ran the scanner for like finances domain, right?

[1:03:50](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3830s) Okay.

[1:03:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3831s) It has a bunch of recommendations. How do I know that that's enough?

[1:03:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3836s) Uh-

[1:03:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3836s) It actually makes it even better.

[1:03:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3838s) It, it-

[1:03:58](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3838s) How did you assess that?

[1:04:00](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3840s) It actually tries to detail out as much as possible. Right now, I haven't actually exposed everything what it did just for the UI purpose. Uh, but the, all the per ticket, what it actually found, like, uh, it writes like 100 or like 150 lines of, uh, Markdown files and save it somewhere.

[1:04:18](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3858s) So, uh, that gives you more details in case if you want to know actually. So for, for the demo, I just put the, you know, nice UX stuff on, on top of it, but you also have a detailed information at, uh, test.

[1:04:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3871s) Anyone else has any questions?

[1:04:33](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3873s) Can I have another one?

[1:04:34](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3874s) Yeah, sure. Go on.

[1:04:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3875s) Right. So you-

[1:04:36](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3876s) Easy one, huh? [laughs] At least try. [laughs]

[1:04:41](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3881s) Yeah. You had a chart where claims or red line, that's forty percent-

[1:04:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3886s) Yeah

[1:04:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3886s) ... that's the gap, right? And if I got the presentation right, your claim is with a finite amount of Jira tickets you can fill that gap.

[1:04:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3896s) Not just-

[1:04:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3897s) You said a couple of weeks, right? Couple of weeks and you will have filled that knowledge base.

[1:05:01](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3901s) Not fill the gaps, uh, but discover. But if you scope down to a team, uh, within weeks you can do it.

[1:05:08](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3908s) Right. Because, I mean, if you wouldn't fill it and would have to keep asking those questions, then solution doesn't make sense, right? So at some point you have the knowledge base that's created, available, and delivered by-

[1:05:21](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3921s) Yeah. So you do it one time, uh, first of all, or like m-multiple times at first. See the whole picture first of all. What is the state of your knowledge base?

[1:05:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3931s) First, fix it, uh, at that level. Then you go into operations, right? You still can actually, uh, also-

[1:05:37](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3937s) Sure. Having the-

[1:05:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3938s) You, you can still continue doing it, uh, with the agent, with skills.

[1:05:42](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3942s) But at some point you will, you will, you will kind of squeeze that, that forty percent gap, right?

[1:05:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3946s) Yep.

[1:05:46](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3946s) And, um, I think the same process of kind of providing that knowledge, because that knowledge you say is in people's heads, right?

[1:05:55](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3955s) Uh-huh.

[1:05:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3956s) It's not documented.

[1:05:56](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3956s) Oh, okay. Yeah, yeah.

[1:05:57](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3957s) That filling happens pretty frequently when a new member joins the organization.

[1:06:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3963s) Mm-hmm.

[1:06:03](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3963s) So wouldn't a replacement that be just get Zoom calls, transcribe them, and use them as a source, assuming all the calls or all the knowledge transfers or maybe introductions happen over Zoom?

[1:06:19](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3979s) Mm.

[1:06:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3980s) Or Teams or whatever the communication. Like if there's a new member joining-

[1:06:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3984s) Mm-hmm

[1:06:24](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3984s) ... asking questions and you have access to that-

[1:06:27](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3987s) You also have access to Zoom also

[1:06:29](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3989s) ... that's also a solution.

[1:06:31](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3991s) Okay. So you mean like, uh, you can also give all the transcripts rather than, uh, uh, doing the cycle, you mean?

[1:06:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3998s) Yeah.

[1:06:38](https://www.youtube.com/watch?v=_QAVExf_1uw&t=3998s) Uh, that can also be done, uh, if you're only have the... All the time you have a discussion in the meeting, everything is documented in meetings transcript itself. But I don't think like that's the same case for everyone at least.

[1:06:51](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4011s) Uh, uh, for-

[1:06:53](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4013s) Sure. But that should be easier to solve, right? If, if, if... 'Cause if I come across a Jira ticket that I don't know or I'm unsure about, I will call someone or chat with someone.

[1:07:04](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4024s) I think the amount of time people spending in Teams-

[1:07:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4027s) Absolutely

[1:07:07](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4027s) ... if you do use the transcripts, actually, those are the ones actually who, which, uh, have more tokens actually. There are so many useless meetings, uh, s- that transcripts actually, uh-

[1:07:20](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4040s) Totally agree. But the compression happens only once, and then that's part of the database.

[1:07:25](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4045s) Could be. Uh, again, it depends upon institution to institution, right? So are you like more into meetings, have solving problems within the conversations, and those conversation has the data?

[1:07:35](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4055s) Or like your Confluence or things has the data. If you have it, use th-those transcripts as your kn- knowledge base. And the, at the same time, like, yeah, the compression actually, that works actually.

[1:07:44](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4064s) That, that is more useful. Yep. Anyone else? Any questions? No. All good.

[1:07:54](https://www.youtube.com/watch?v=_QAVExf_1uw&t=4074s) Uh, then, uh, thank you so much, uh, for attending the session. [upbeat music]
