AI Engineer World's Fair 2026
Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates
Slide chapters
14 slide and demo groups · Timestamps are approximate.
Read the talk
Why We Killed Our Multi-Agent Pipeline
A pharmaceutical analytics pipeline found the right cause but recommended the wrong action. The redesign separated statistical detection from investigation and used a domain graph to constrain the search.
From a talk by Subbiah Sethuraman and Abhilash Asokan
Before you start: Familiarity with language-model agents, database queries and basic graph concepts will help; the pharmaceutical workflow is introduced in the article.
Why are prescriptions falling?
Why are doctors writing fewer prescriptions for a drug, and what should the company do about it? That is a commercial analytics question: after drug discovery and clinical trials, the work shifts to getting treatments to patients. At ZS, Subbiah Sethuraman, who introduces himself as head of AI Engineering, and Abhilash Asokan, Director of AI Engineering, work in a setting that includes brand performance across markets, sales-rep engagement, patient adoption and therapy switches.
An analyst moves through four connected steps:
- Signal: Detect a change, such as a drop in prescriptions.
- Why: Investigate whether a competitor entered the market, payer coverage declined or sales reps failed to communicate the drug’s benefits.
- Action: Choose an intervention that addresses the cause—for example, increase rep coverage if a region is underserved.
- Outlook: Estimate how the intervention could affect brand or sales performance.
The steps form a causal chain. The action depends on the explanation, and the outlook depends on the action.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turning the workflow into agents
ZS initially translated this human workflow into specialized agents. An orchestrator connected the stages, while the explanation step split into two distinct jobs: locating the decline and identifying its driver.
| Agent | Responsibility |
|---|---|
| Signal detection | Identify changes worth investigating |
| Source localization | Locate a national decline within a region or payer |
| Driver attribution | Identify why performance declined |
| Synthesis | Propose an action and its outlook |
| Orchestrator | Connect the agents |
This division mirrored the analyst’s sequence, but it also divided responsibility for understanding the result.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The right cause led to the wrong action
The pipeline produced an information packet with a signal, a reason, an action and an outlook. In Sethuraman’s illustrative packet, brand prescriptions fall 18% in a territory over roughly four weeks. The explanation identifies a payer coverage change that makes the drug more expensive for patients. Yet the recommended action is to send more sales reps to doctors, followed by a forecast of improved sales.
The packet looks complete until its parts are read together. More rep visits do not address the identified affordability problem. Because the action fails to address the cause, the recovery outlook loses its foundation too. Locally plausible outputs did not produce a coherent recommendation. No single agent owned the complete connection between evidence, cause, intervention and expected result.
Sethuraman attributes the failure to how the work was divided, rather than to the language model itself. Three architectural problems emerged in sequence:
- Unnecessary model judgment: A sales drop can be detected with statistical methods; a language model need not decide whether it exists.
- Context lost at handoffs: Driver attribution found the coverage problem, but synthesis failed to preserve its significance for patient affordability.
- Missing shared domain knowledge: The agents lacked a common understanding of business metrics, their relationships and why they rise or fall. Even a metric such as
TRx, total prescriptions, needs that context to be useful in an investigation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Observe the investigation before redesigning it
The first instinct was to redesign the topology, skills, tools, handoffs or schemas between agents. Instead, Asokan’s team opened an empty directory, ran Claude Code with Bash and database access, supplied an actual signal they had already identified, and watched how it investigated. The experiment began with the work itself rather than a new arrangement of agent roles.
The shell entry point for that kind of observation is small. Database access must already be available in the environment; the prompt gives the agent an investigation rather than asking it to discover a signal:
bash
mkdir pharma-investigation
cd pharma-investigation
claude 'Investigate the supplied prescription-decline signal using the available database. Examine where the decline is concentrated and which evidence supports or contradicts possible causes.'
The important output of this exercise is the sequence of operations the agent chooses, which can reveal where a dedicated tool or a deterministic process belongs.
Signal detection was the first responsibility to move out of the agent. The team had observed inconsistent behavior: sometimes the agent applied statistical methods, sometimes it barely inspected the data, and sometimes it confused noise with a signal. They replaced that judgment with an automated statistical workflow.
The new pipeline scans each KPI for anomalies and broken trends, applying guardrails, thresholds and prioritization before any investigator starts. It places selected signals on a queue; arrival of a signal wakes the agent. The agent investigates a detected change; it does not decide whether the change qualifies as a signal. This boundary gives the investigation a defined starting point.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep parallel work, centralize judgment
To address incoherent recommendations, the team consolidated the investigation around one main agent. Observing Claude Code also exposed repeated operations—writing a function and querying the database—that they packaged into a tool. The redesign reduced the number of reasoning handoffs without requiring all work to run serially.
Claude Code sometimes launched subagents dynamically for focused tasks, and ZS retained that pattern. A main investigator could delegate a question such as whether rep activity had changed in a particular region, then use the returned findings in the larger investigation. Subagents may reason within their tasks; the architectural rule is that the main agent retains final judgment about how their findings fit together.
That produced a lighter architecture, but consolidation alone did not supply business knowledge. A single agent could still misunderstand the relationships among entities and metrics. The remaining problem required an explicit domain model.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make domain relationships explicit
The agent had been inspecting database tables and inferring relationships from them. Asokan describes that approach as difficult to scale and prone to producing relationships that did not exist in the data. ZS brought in its commercial analytics domain experts to map the domain into a knowledge graph.
The graph connects geographic entities to payers and accounts, connects those entities to brands, and connects brands and payers to KPIs. It also represents relationships among primary, secondary and tertiary KPIs, including which metrics can drive others. This supplies more than entity definitions: it gives the investigator a structured account of which relationships deserve examination.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use the graph to bound the investigation
The graph’s operational role is a control plane: it determines what the agent can examine, which paths it can take and which hypotheses it can evaluate. It is not simply another source from which to retrieve facts.
Consider a national decline in TRx. The first question is where the decline is concentrated. It might be in a region, a territory or a combination involving a territory, payer or account. Exploring every combination creates a large search space. The graph guides source localization through the relevant dimensions.
Once the agent narrows down the location, it investigates why the decline occurred. KPI relationships provide the next candidate explanations: which metric could be driving the observed change? In Asokan’s design, each edge is a hypothesis the agent can evaluate, and the investigation stays within that graph-defined surface. An edge supplies a candidate explanation, not evidence that the explanation is true.
The resulting loop ties graph navigation to the underlying data:
- Start with an entity and inspect its graph neighborhood.
- Treat the neighboring edges as candidate hypotheses.
- Query the original data and examine the actual numbers.
- Reason about whether the evidence supports or contradicts each hypothesis.
- Follow a supported relationship and repeat the process.
- Stop when the available hypotheses are exhausted or a root cause is found.
The graph bounds the search, while the data determines whether a proposed path deserves further investigation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An architecture derived from the work
Asokan reports that investigations taking 50-plus turns and substantial tokens could produce analysis in roughly 20–30 minutes that might take an analyst three or four weeks. This is an approximate comparison from their pharmaceutical commercial analytics work; the talk supplies no evaluation dataset, quality criterion or cost breakdown. The shorter turnaround still involved a long investigative loop.
The closing design principle is to let the observed work shape the architecture instead of imposing the human workflow’s boundaries on it. In this system, deterministic operations establish the signal, and agentic work investigates it. One agent owns the reasoning end to end while invoking subagents, tools or skills for particular tasks. The graph then governs navigation and the next investigative decision. That places responsibility for the complete explanation in one agent while keeping its search grounded in explicit domain relationships.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Current guidance for delegating focused tasks to Claude Code subagents with separate contexts, tool access, and permissions.
Further reading
- Understanding formulary tiersDocumentation
Medicare’s explanation of drug formularies, tier levels, and their relationship to patient copayments.
Read the complete timestamped transcript
14 slide groups. Extracted from the recording; expand builds to follow changes. Timestamps are approximate.
- 0:00
[upbeat music] Uh, okay, so I'm Subbiah. I head AI Engineering at ZS.
- 0:17
Uh, I'm Abhilash. I'm Director of AI Engineering at ZS.
- 0:20
So ZS, we are a tech firm. We work with many of the top companies in the world, including a lot of the t- top pharmas, actually. Um, so today's talk, I think we wanted to, as I think already introduced, right?
Why We Killed Our Multi-Agent Pipeline
Subbiah Sethuraman and Abhilash Asokan share mistakes and lessons from building multi-agent pipelines for pharma commercial teams.
Text in image
Why We Killed Our Multi-Agent Pipeline Lessons from building AI for pharma commercial teams Subbiah Sethuraman · Abhilash Asokan ZS Associates AI Engineer World's Fair 2026
- 0:29
We wanted to talk from our experience, right? Building multi-agent pipelines. Uh, what are the mistakes we did, right? And what did we learn and how did we fix them?
- 0:38
So, um, so I'm going to orient it more on pharma commercial domain and for people in the room probably who are not aware, quickly, uh, pharma has two main functions.
- 0:46
One is R&D, right, the drug discovery, right, uh, and the clinical trials part. And then there's the commercial. How do you take a drug to a patient, basically. And, and within commercial, there are different functions.
- 0:58
Like, once you create a drug, right, um, what is the performance of a brand? How is the drug performing in different markets, right? Uh, then there are things around your field force, your reps.
- 1:07
How effectively are they engaging, uh, right, with everyone? There are things around patient journey. How are patients are adopting a drug, right? I think if there is any therapy switch which is happening.
- 1:17
So as you can think about, there is a lot of analytics which really happens in a commercial domain. And h-how do typically analysts work?
How an analyst works
A drop in prescriptions can prompt an analyst to check the signal, investigate its cause, decide on an action, and assess the outlook for sales.
Text in image
How an analyst works Four questions, answered in order. 1 Signal Is the change real? 2 Why What's causing it? 3 Action What should we do? 4 Outlook What happens next?
- 1:25
So there are four steps, right, what analysts do, right? So first, there is always something called a signal detection, right? So signal can be something like, okay, the prescriptions what a doctor is writing.
- 1:35
Maybe is there, is there a drop in the prescription? So that's a signal. So once you've got a signal, the second thing what an analyst does is why is this signal really firing?
- 1:45
What is the reason for it? So is it, like, there is a competitor drug which has come in? Because of that, is it, is it reducing? Is it, is it because maybe a payer coverage for the drug has reduced, or maybe the, the reps on the ground, there's no proper, uh, they're actually not taking the benefits to
- 1:58
the doctors. And once you arrive at the reason, the next step becomes, okay, what is the action do you take? So if reps-- suppose if reps, the coverage is not good in a particular region, should-- do we have to increase that?
- 2:12
And once you do that, what is the, what is the outlook, right? Is my brand-- is my sales performance, is, is it going to improve? Right? So these are the four things which happens.
The agentic system
An orchestrator connects four agents: one detects signals, two locate their source and attribute the driver, and synthesis determines action and outlook.
Text in image
We built an agentic system to mimic that One agent per step, an orchestrator running the team. Orchestrator runs the team SIGNAL Signal Detection WHY IT'S HAPPENING Source Localization Driver Attribution two agents to answer one question ACTION + OUTLOOK Synthesis
- 2:21
Now, for some of the top pharmas, what we have done is how do we in an agentic way, right? I think how do we actually mimic this, this analytics work?
- 2:29
So what we did, we built agents for every step, right? Signal detection. We said, "Okay, we'll have an agent for signal detection." It'll identify, identify the signals for me.
- 2:39
Second, what are the root cause, right, for the signals, right? So in this case we have two agents. One we call it a source localization. So for example, if my sales is dropping at a national level, is it because it's dropping at, say, a particular region, or is it dropping for a payer?
- 2:53
So we need to understand that. We are, we are trying to identify the source of it. And then once we understand what is the real reason, right, I think the, the sales performance has gone down.
- 3:03
That's another agent, the driver attribution agent. And then the last step is your synthesis, right? So depending on the cause, now what is the action you have to take and what is the outlook?
- 3:11
Like, typically how an analyst used to do. And all of this, we used to have an orchestrator agent which connects all these agents together.
- 3:18
So now what happened once we had this system, what did it generate, right? It generates something, an information packet like this, right? It clearly tells you the signal, right?
The packet it generated
Brand A’s new prescriptions fell 18% in Territory X over four weeks; almost all the drop was tied to Payer Y’s worse coverage tier. The packet recommends more sales visits and calls the outlook healthy heading into next quarter.
Text in image
The packet it generated SIGNAL Brand A's new prescriptions dropped 18% in Territory X over four weeks. WHY Almost all of the drop is in one insurer, Payer Y. They moved Brand A to a worse coverage tier, so patients now pay more — and many walk away at the pharmacy counter. ACTION Send sales reps to visit more doctors in Territory X to push prescriptions back up. OUTLOOK The overall outlook for the brand is healthy heading into next quarter.
- 3:27
So first it says maybe my brand's prescriptions have dropped eighteen percent in some territory, right, in some time frame, maybe four weeks. It then tells you the reason. Why did it, why did it drop, right?
- 3:38
The reason it says because a payer actually, right, the coverage for this drug has actually-- they moved it to a lower tier. So for patients it's expensive actually to, to buy this drug.
- 3:49
The action it says, okay, because doctors are writing less prescriptions, maybe send more sales reps to talk to doctors, right, and increase the number of, uh, prescriptions which you're writing.
- 4:00
And then if you take this action, maybe your outlook, your sales performance is, is going to increase.
- 4:05
So all of this looks good high level, but then if you, if you look at, at it closely, it's not very coherent, right? The cause is right. It identified the right cause, right?
It looks correct. Until you read it closely.
The pipeline identified an insurance affordability problem, then recommended more doctor visits and called the situation recoverable if rep activity improves. No single agent saw the whole picture.
Text in image
It looks correct. Until you read it closely. ✓ THE CAUSE Patients can't afford the drug after an insurance change ✕ THE ACTION Reps should visit more doctors — which says nothing about insurance ✕ THE OUTLOOK “Recoverable if rep activity improves” And here's the unsettling part The action doesn't match the cause. Nothing was made up. Every fact is real — but no single agent ever saw the whole picture.
- 4:14
Because patients can't afford the dug- drug, right? But the action it said, it didn't really focus on the payer part, the insurance part of it, right? It just said reps, right?
- 4:24
Uh, focus more reps actually, right? And then the outlook. Because the action is wrong, the outlook is not going to match. So at each level if you see, it is actually derived the right fact, but then there is no single agent which is owning-- which understands the end-to-end picture, basically.
Why it failed
The pipeline asked an LLM to judge a statistical signal, lost the reasoning behind the cause at handoffs, and gave agents no shared understanding of what drives prescriptions.
Text in image
Why it failed Not the model. The way we split the work. 1 A language model was deciding the signal We asked an LLM whether an 18% move was a real signal or normal noise — a question statistics should answer, not a model. 2 Context was lost at every handoff Each agent passed on its conclusion, not its reasoning. The clue that explained the drop never reached the agent writing the advice. 3 No shared understanding of the business Each agent inferred how pharma metrics relate from raw tables. Nothing told them what actually drives prescriptions, or in which direction.
- 4:40
So why did this happen? Like, why did it fail, right? So obviously it's not the LLM which failed, right? It's the way how we split the work, right? Because we tried mimicking the analyst behavior and we did it.
- 4:53
The first, first key issue is, like, a language model is actually det-determining your signals. So signals like things like your sales drop is a simple information which you can use statistical methods to actually go and fetch this information.
- 5:05
You don't need a language model actually, right, to fetch this information. Second is as your multi-agents, there is a lot of context handoff which is happening and context is actually getting lost at each of these handoffs.
- 5:17
So for example, the driver attribution agent is actually determining the right cause, but then the next agent, the synthesis agent, is actually is not able to understand why is that, right?
Why it failed
The pipeline asked an LLM to judge a statistical signal, lost the reasoning behind the cause at handoffs, and gave agents no shared understanding of what drives prescriptions.
Text in image
Why it failed Not the model. The way we split the work. 1 A language model was deciding the signal We asked an LLM whether an 18% move was a real signal or normal noise — a question statistics should answer, not a model. 2 Context was lost at every handoff Each agent passed on its conclusion, not its reasoning. The clue that explained the drop never reached the agent writing the advice. 3 No shared understanding of the business Each agent inferred how pharma metrics relate from raw tables. Nothing told them what actually drives prescriptions, or in which direction.
- 5:27
The, the, the patients are finding the drug to be expensive. It's not understanding the weightage of an insurance coverage going, going down. So that is a key information, right, which is getting lost.
- 5:37
And the last big piece is there is no shared understanding of the business domain knowledge for all these agents. All these agents don't understand metrics, right? So things like TRX, the number of transaction-- uh, number of prescriptions which a doctor writes, right?
- 5:50
What is the relationship between them? Why does it go up or down? So those are the, the reasons. So then what did we do? Uh, so I called Abhilash to come and solve for this.
Deriving the architecture
The team gave Claude Code Bash, a database and a real KPI decline, then watched its traces to derive what the agent touches. They decided where the math lives up front.
Text in image
So we stopped designing the architecture — and started deriving it. A bare agent. Bash and the database One real KPI decline. Then we watched. What the agent touches, we derived from traces. Where the math lives, we decided up front.
- 6:01
Uh, thank you, Subbiah. So we had, like, three problems. Uh, so we are thinking, how are we going to solve this? Uh, our first instinct was we'll go back to the drawing board.
- 6:12
Uh, we'll start designing it again. So maybe the topology was wrong, the skills were wrong, the tools were wrong, or maybe the-- maybe we have to de-de-define a better handoff-- design a better handoff or a better schema between agents.
Deriving the architecture
The team gave Claude Code Bash, a database and a real KPI decline, then watched its traces to derive what the agent touches. They decided where the math lives up front.
Text in image
So we stopped designing the architecture — and started deriving it. A bare agent. Bash and the database One real KPI decline. Then we watched. What the agent touches, we derived from traces. Where the math lives, we decided up front.
- 6:27
But we took a step back. We didn't do any of that. We, like all of us, we went back to Claude Code. So we opened a very pla-plain, empty directory and ran Claude Code, then give it just bash and the database.
- 6:43
Then give it an actual signal which we identified, then started observing what it is doing.
- 6:49
So while we look at what Claude does, we are able to figure out fix for all the three issues we discussed.
Statistics finds the signal
A deterministic workflow tests and prioritizes signals before the agent starts. The agent then investigates the signal, reasons about its cause, and writes a recommendation.
Text in image
FIX 1 · Language model determined the signal Statistics finds the signal. The agent explains it. STATISTICS Find the signal. Is an 18% drop real, or just noise? Deterministic tests and guardrails decide — before the agent starts. THE AGENT Explains it. Once the signal is real, the agent investigates, reasons about cause, and writes the recommendation. A language model is a reasoning engine, not a statistics engine.
- 6:55
The first, the first part. So what was happening was the agent was
- 7:01
looking at data and deciding on a signal.
- 7:05
Sometimes it applied some statistical methods, sometimes it barely look at the data and say, "This is a signal." Sometimes it's actually a signal, sometimes it's a noise. This is something we don't want an agent to do.
- 7:16
This is a completely a deterministic workflow. So we separated it out from the agentic system.
- 7:22
So we built a pure deterministic workflow with different statistical methods. We put guardrails, we put thresholds, we put prioritization. Everything happened before the agent even kick-starts. So we ran an automated pipeline which scans through the data, identify signals for each of the KPI.
Statistics finds the signal
A deterministic workflow tests and prioritizes signals before the agent starts. The agent then investigates the signal, reasons about its cause, and writes a recommendation.
Text in image
FIX 1 · Language model determined the signal Statistics finds the signal. The agent explains it. STATISTICS Find the signal. Is an 18% drop real, or just noise? Deterministic tests and guardrails decide — before the agent starts. THE AGENT Explains it. Once the signal is real, the agent investigates, reasons about cause, and writes the recommendation. A language model is a reasoning engine, not a statistics engine.
- 7:41
Is anything happening with that? Any anomalies which is happening? Any trend which is very, uh, any trend which is breaking? Based on that, we identify the signal, we put it on a queue.
- 7:50
The moment the signal comes to the queue, the agent wakes up. So the agent's job is to investigate, not to identify.
What replaced the pipeline
One analyst agent owns the investigation and judgment, delegating focused tasks to sub-agents. A knowledge graph supplies pharma context.
Text in image
What replaced the pipeline. FIX 2 · context was lost at every handoff Knowledge graph Signal Detection Analyst agent Sub-agent Sub-agent launched for one isolated task — e.g. verify a driver branch: did rep activity really fall? One agent owns the whole investigation. Sub-agents investigate. Knowledge Graph provides pharma context
- 7:59
The second part. So mainly the issue which we previously, what Subbiah was mentioning, there is no coherence in the output which the agents produced.
- 8:10
So we started consolidating. We look at Cl- how Claude Code operates. It's able to do a lot of operations, so we started designing around that. So it's repeatedly writing a function and querying database, so we give it a tool for that.
What replaced the pipeline
One analyst agent owns the investigation and judgment, delegating focused tasks to sub-agents. A knowledge graph supplies pharma context.
Text in image
What replaced the pipeline. FIX 2 · context was lost at every handoff Knowledge graph Signal Detection Analyst agent Sub-agent Sub-agent launched for one isolated task — e.g. verify a driver branch: did rep activity really fall? One agent owns the whole investigation. Sub-agents investigate. Knowledge Graph provides pharma context
- 8:25
So this consolidated the entire process into a single agent. That doesn't mean that we didn't do parallelism. We still do parallelism. What we removed is, do we need distributed reasoning?
- 8:36
We didn't want the judgment to be distributed between agents. That, we wanted to consolidate to a single agent. So that was exactly what we did. Then occasionally we observed Claude Code.
- 8:46
It's launching sub-agents dynamically for a very particular focused task. So we did the same thing. Because if-- suppose you want to understand if rep activity in a particular region, that is an investigation which you need to run.
- 9:00
That, you can still delegate to a sub-agent. You can get back the, uh, results back, not the reasoning or the judgment. That is still controlled by the, by the main agent.
- 9:09
But the investigation part of it, we de- we're delegating it to a sub-agent. So these are some of the things we kept based on observing what Claude was doing-- Claude Code was doing.
- 9:18
So that give us a more lighter architecture than what was initially there. But still it doesn't solve the problem. That's why you see a knowledge graph in the diagram.
What replaced the pipeline
One analyst agent owns the investigation and judgment, delegating focused tasks to sub-agents. A knowledge graph supplies pharma context.
Text in image
What replaced the pipeline. FIX 2 · context was lost at every handoff Knowledge graph Signal Detection Analyst agent Sub-agent Sub-agent launched for one isolated task — e.g. verify a driver branch: did rep activity really fall? One agent owns the whole investigation. Sub-agents investigate. Knowledge Graph provides pharma context
The business domain as a graph
The graph localizes an 18% NRx decline to Territory X, then traces possible drivers through coverage, patient cost and abandoned fills. It also connects rep reach to the investigation.
Text in image
FIX 3 · No shared understanding of the business We gave the agent the business domain — as a graph. WHERE · localize first National CONTAINS Region Territory X Account HCP Brand A OF NRx ↓ 18% measured at Rep reach ↓ small + small WHY · only after WHERE Payer Y SETS Coverage tier ↓ + raises Patient cost ↑ + raises Abandoned fills ↑ − suppresses Legend: geography · entity · driver · signal The graph didn't just add context. It structured the investigation.
- 9:28
So it still doesn't have the business context. It still in-- doesn't understand all the entities, the domain, the KPIs, how do they relate to each other.
- 9:42
So that's something we wanted to solve for because the agent was looking at data, looking at tables, then trying to infer the relationship that-- which was not scalable. And it often produced relationship which i-- which is not actually exists in the data.
- 9:57
So what we did. So we've been in this-- working in this field for a lot, lo-whole, whole lot of years, so we've been doing this for our clients. So we have a lot of domain experts who understand the pharma domain very well.
- 10:09
We are doing commercial analytics for the clients. So we start, sat with them, start building a knowledge graph. We sta-start trying to map out the domain.
- 10:19
So we are able to build a knowledge graph. We are able to identify different entities, their relationship between that. So if you see, there are, like, geographic entities. There are payers.
The business domain as a graph
The graph localizes an 18% NRx decline to Territory X, then traces possible drivers through coverage, patient cost and abandoned fills. It also connects rep reach to the investigation.
Text in image
FIX 3 · No shared understanding of the business We gave the agent the business domain — as a graph. WHERE · localize first National CONTAINS Region Territory X Account HCP Brand A OF NRx ↓ 18% measured at Rep reach ↓ small + small WHY · only after WHERE Payer Y SETS Coverage tier ↓ + raises Patient cost ↑ + raises Abandoned fills ↑ − suppresses Legend: geography · entity · driver · signal The graph didn't just add context. It structured the investigation.
- 10:29
How, how geographic entities connect to a payer or an account? How does this connect to a brand? Then from ba-brand or payer, how does it go to a KPI?
- 10:38
How does a KPI relate to like a, uh, secondary KPI, tertiary KPI? How does one KPI drives another KPI? So we started mapping out all of this information and then creating our, our knowledge graph.
- 10:51
Once you have the knowledge graph, then we want to let the agent to navigate the knowledge graph. So the knowledge graph is not just something the agent looks up for data.
The graph as the agent's control surface
The agent ranks children to locate concentration, then follows driver KPIs and checks evidence for each edge. Graph edges are hypotheses, not proof of cause.
Text in image
The graph is the agent's control surface. FIX 3, THE MECHANICS · the contract the agent must use GRAPH CONTRACT THE AGENT MUST USE (:Entity)-[:CONTAINS]->(:Entity) (:KPI)-[:EXISTS_AT]->(:EntityLevel) (:KPI)-[:HAS_DRIVER_CATEGORY]->(:DriverCat) (:DriverCat)-[:HAS_DRIVER_KPI {effect, diagnostic_q}]->(:KPI) WHERE PROTOCOL rank children by contribution return top_n = 5 stop if systemic / not concentrated WHY PROTOCOL walk driver KPIs, max_depth = 6 cycle guard classify SUPPORTS / CONTRADICTS from deterministic evidence checks — read, not improvised Every edge is a hypothesis the data still has to support — not proof of cause.- 11:01
It is a control plane for the agent.
- 11:04
So what do you mean by that? So the knowledge graphs dictates what the agent can look into, what path it can take, what investigation hypothesis, uh, it can evaluate.
- 11:17
So for example, if you-- when you do this analysis, the root of the problem, say something like your TRX is declining at a national level. This could be when you do source localization, so that's our, uh, contract for how to find the where.
The graph as the agent's control surface
The agent ranks children to locate concentration, then follows driver KPIs and checks evidence for each edge. Graph edges are hypotheses, not proof of cause.
Text in image
The graph is the agent's control surface. FIX 3, THE MECHANICS · the contract the agent must use GRAPH CONTRACT THE AGENT MUST USE (:Entity)-[:CONTAINS]->(:Entity) (:KPI)-[:EXISTS_AT]->(:EntityLevel) (:KPI)-[:HAS_DRIVER_CATEGORY]->(:DriverCat) (:DriverCat)-[:HAS_DRIVER_KPI {effect, diagnostic_q}]->(:KPI) WHERE PROTOCOL rank children by contribution return top_n = 5 stop if systemic / not concentrated WHY PROTOCOL walk driver KPIs, max_depth = 6 cycle guard classify SUPPORTS / CONTRADICTS from deterministic evidence checks — read, not improvised Every edge is a hypothesis the data still has to support — not proof of cause.- 11:31
It might happen within a region. It could be concentrated in a territory. It would be concentrated in a combination of a territory or a payer or an account. So there are a whole lot of dimensions which the agent needs to evaluate.
- 11:42
Now, it's a lot of permission-- permutation combination. So the knowledge graphs guides the agent on how to find where this is concentrated. Then the why, why part of it.
- 11:53
Why it is happening? When you figure out that, okay, something is concentrated at, say, the decline is concentrated at a particular region, now you need to figure out the why This is where, where the, uh, KPIs and the relations come, comes in.
- 12:05
Once the agent is able to narrow down the where, then it's able to go and figure out the, um, figure out the why part of it, which KPIs is driving what.
- 12:16
Graph acts as the control surface. So agent, every edge is a hypothesis, so the agent can go and evaluate that hyposi- hypothesis. Uh, it doesn't go outside of this.
The graph as the agent's control surface
The agent ranks children to locate concentration, then follows driver KPIs and checks evidence for each edge. Graph edges are hypotheses, not proof of cause.
Text in image
The graph is the agent's control surface. FIX 3, THE MECHANICS · the contract the agent must use GRAPH CONTRACT THE AGENT MUST USE (:Entity)-[:CONTAINS]->(:Entity) (:KPI)-[:EXISTS_AT]->(:EntityLevel) (:KPI)-[:HAS_DRIVER_CATEGORY]->(:DriverCat) (:DriverCat)-[:HAS_DRIVER_KPI {effect, diagnostic_q}]->(:KPI) WHERE PROTOCOL rank children by contribution return top_n = 5 stop if systemic / not concentrated WHY PROTOCOL walk driver KPIs, max_depth = 6 cycle guard classify SUPPORTS / CONTRADICTS from deterministic evidence checks — read, not improvised Every edge is a hypothesis the data still has to support — not proof of cause.- 12:28
So that gives a more bounded surface for the agent to investigate.
The agent's investigation loop
The agent checks graph edges against real numbers, keeping or rejecting each hypothesis before repeating. The illustrated surviving path runs from lower tier through higher cost and abandonment to lower NRx.
Text in image
FIX 3 · the contract, running The agent runs this loop. repeat — next candidate TRAVERSE the graph proposes the next move QUERY pull the real numbers REASON keep or kill the edge CONCLUDE · the surviving path: tier ↓ → cost ↑ → abandonment ↑ → NRx ↓ Traverse. Query. Reason. Repeat.
- 12:34
So like everyone was mentioning about loop, so we also build a loop. So what the agent does. So the agent first start with an entity. It goes to the graph,
- 12:46
it looks at the neighborhood of the graph, then it, it figures out the edges. So it's got some hypothesis. It'll go, it'll go back to the original data, evaluate that hypothesis, look at the actual numbers, then reason over it.
- 13:02
Then it'll, it'll either find it, find it contradicting or it fi- finding supporting the, the evidence supporting the hypothesis. If it's supporting it, it started traversing through the graph.
- 13:13
So this repeats this until it run out of all the hypothesis or it's able to figure out the root cause. So this, this concludes this run. This runs and it'll be able to pr- fi-figure out the, uh, root cause of the problem.
The agent's investigation loop
The agent checks graph edges against real numbers, keeping or rejecting each hypothesis before repeating. The illustrated surviving path runs from lower tier through higher cost and abandonment to lower NRx.
Text in image
FIX 3 · the contract, running The agent runs this loop. repeat — next candidate TRAVERSE the graph proposes the next move QUERY pull the real numbers REASON keep or kill the edge CONCLUDE · the surviving path: tier ↓ → cost ↑ → abandonment ↑ → NRx ↓ Traverse. Query. Reason. Repeat.
- 13:28
So once we built this, so maybe like after like fifty plus turns, a whole lot of tokens, it's able to produce something an analyst, uh, was able to produce maybe in three or four weeks in like maybe twenty, thirty minutes.
- 13:41
Thank you.
Key Takeaways
Derive the architecture, separate deterministic work, and give one agent end-to-end ownership of reasoning. The agent uses the graph to navigate and choose its next move.
Text in image
Key Takeaways 1 Architecture should be derived, not designed 2 Separate the deterministic from the agentic 3 Give one agent end-to-end ownership of the reasoning 4 Treat the graph as a control surface, and not just a lookup layer Subbiah Sethuraman · Abhilash Asokan · ZS Associates
- 13:44
So just to wrap it up, um, key takeaways. First thing is, I think we should not be introducing human constraints or design constraints into architecture. I think let the architecture be derived actually, number one.
- 13:57
Second, I think any complex workflows will have deterministic parts and agentic parts. Don't let agents actually run the deterministic part, right? So I think we need to break, break that off.
- 14:08
The third, you need to have one agent, right, which owns the reasoning end-to-end, right? This agent can actually take the call to use sub-agents, tools, skills to actually spawn off other, other tasks basically, right?
- 14:20
But you need one agent to own the, the reasoning. And the last, probably the most important, I think graph cannot be treated just as a lookup layer. I think graph has to be treated as a control plane which the agent uses to navigate and takes the next decisions basically.
- 14:36
Okay. With that, I think thank you. Thanks. Thanks for attending this. [audience clapping] [outro jingle]
AI Engineer World's Fair
White AI Engineer World's Fair logo and website on a black background.
Text in image
AI Engineer World’s Fair HTTPS://AI.ENGINEER