The 6 Pillars of an Agentic Harness for Production — Varun Krovvidi, Resolve AI
Loading slides…
Read the talk
The 6 Pillars of an Agentic Harness for Production
Varun Krovvidi explains Resolve AI’s approach to running and fixing software: match models to tasks, control context, demand causal evidence, govern actions, retain learning and evaluate the investigation itself. A Grafana-to-Slack demo shows why a plausible outage story needs more than coincidental timing.
From a talk by Varun Krovvidi
At a glance
Ideas worth remembering
Model selection belongs at the task level: an investigation combines different kinds of reasoning, and new models require continued evaluation.
Context engineering balances visibility and cost. Broad orientation can start an investigation; precise tool calls keep later queries relevant and limit token use.
A root-cause claim needs a causal chain of evidence. Missing links should lower confidence, and simultaneous events should remain hypotheses until connected to the failure.
Govern actions separately from diagnosis, retain useful context across investigations, and evaluate the agent’s reasoning path and confidence as well as its answer.
Writing code leaves the production problem unsolved
Using AI during on-call work is a much lower bar than letting it handle most of that work. Varun Krovvidi of Resolve AI opens by asking engineers whether they have tried AI on call, then whether they can comfortably say it does 90% of their on-call work. That second question frames the challenge: building agents that can run and fix software, rather than merely help an engineer discuss a problem.
Krovvidi’s explanation for AI’s progress in coding rests on three properties. Code carries information about its own behavior, so a model can parse it and work out a next step. Its modular structure lets the model reason over smaller chunks and extend them. And, in his framing, coding stays largely within one domain, making it easier to define benchmarks and assess a solution. Production diagnosis must connect several kinds of evidence whose relationships are less explicit.
The talk puts running and fixing software at 70% of engineering time. Treat that as Krovvidi’s framing of the workload; he does not supply a measurement method. His healthcare analogy usefully separates three kinds of operational work:
- Routine on-call and maintenance: Small recurring problems need attention, like taking Tylenol or Advil for pain.
- Incidents: A serious failure calls for the equivalent of surgery—systems are shutting down and everyone responsible needs to help.
- Preventive work: The “daily vitamins” include inspecting infrastructure, reviewing costs and doing platform work to support growth.
All three cross tools and domains. An agent may need to connect a symptom in telemetry with an infrastructure condition and the code that produced it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The last mile needs a domain-specific harness
Token cost makes the architecture matter even before an agent acts on production. Krovvidi calls this the end of “bottomless AI”: systems need to use models precisely and efficiently. A production investigation cannot simply keep gathering logs, asking questions and spending tokens until something sounds convincing.
The required outcome also changes the standard of success. An image, blog or website can admit many acceptable outputs. An incident investigation seeks a particular explanation of what went wrong. Krovvidi calls this the “single longest mile” of AI: reaching the correct answer requires architecture around a general-purpose model that keeps the investigation focused on the domain.
Production is also a multiplayer problem. Different teams own different systems and bring different expertise, so the investigation must support collaboration. Resolve AI’s stated ambition is for agents to do most of the operational work while engineers run those agents. Its product divides that work into three parallel categories:
- On-call agents: Handle software issues that arrive day to day.
- Incident agents: Help drive incident channels toward a root cause and fix.
- Ambient background agents: Monitor deployments, run analyses and investigate when specified conditions trigger them.
These are different entry points into the same underlying agent architecture, rather than successive stages of one workflow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Five cracks that appear beyond the demo
Why not point the biggest model at production data? A narrow use case can make that approach look good in a demo. Expanding to more teams and more kinds of incidents exposes a different problem: the model’s answer must remain useful as the task, available evidence and operating constraints change.
Anchoring and the model treadmill. An agent can anchor on an early explanation and continue producing a coherent story around it. Meanwhile, new models keep arriving, requiring fresh evaluation and model selection. Selection must happen below the level of an entire workflow: one investigation can contain hundreds of tasks, including image reasoning, SQL, log analysis and more deterministic steps. The best model for one of those tasks need not be the best model for the others.
Too much or too little context. A larger context window does not settle what belongs in it. Too much material can encourage overexploration: the model pursues unsupported hypotheses or loses the route to an answer. Too little hides plausible pathways. Production makes the balance especially difficult because teams can keep generating logs and metrics, then combine that telemetry with code and infrastructure. The task is selecting useful evidence from that expanding space.
Coherence without causality. A convincing explanation can still identify the wrong cause. User pressure makes this worse when a model agrees with whichever theory the engineer proposes next. The investigation needs the detective’s question: what sequence of events actually led to this failure? That sequence determines which issue to fix; a plausible story may only lead to a patch that leaves the cause in place.
Missing guardrails. A model might decide that deleting code, a filesystem or a database is the cleanest way to remove a problem. Whatever the model’s reasoning, the system must constrain its ability to carry out that action. The practical question is the least privilege an agent needs in the current situation.
No learning across people or time. SREs, platform teams and service engineers need shared context during an investigation. The agent also needs to retain useful context across investigations: the hundredth investigation should benefit from previous work. Otherwise, both the team and the system keep starting over.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Pillars 1–3: choose the model, narrow the search, establish the cause
The first pillar, model orchestration, has two layers: keep evaluating new models as they arrive, and match a model to the task currently being performed. Krovvidi gives Gemini for image-specific reasoning, OpenAI models for deterministic steps and Claude for open-ended investigations as examples of the distinction. These are examples of task routing, not a permanent ranking: the orchestration engine must keep evaluating which choice fits each task at the time.
The second pillar, context engineering, starts with a question about the investigation rather than the database: how much context does this particular problem need? Graph RAG and knowledge graphs are implementation choices. A graph-based retrieval approach might provide the relationships needed to begin an investigation; later steps may require narrowly defined tool calls that fetch particular logs, metrics, dashboards or code. Broad initial orientation and precise subsequent queries serve different purposes.
That narrowing also controls token use. Repeated tool calls can consume a large budget if each log or code query returns more material than the next reasoning step needs. Context engineering therefore has to manage both visibility and cost: provide enough evidence to investigate relevant alternatives without turning every query into another large context dump.
The third pillar, causal reasoning, sets the standard for a root-cause claim. Resolve’s stated design requires a chain of evidence connecting the steps that produced the issue. If the available data stops before that chain can be established, the system should lower its confidence and point the user toward another direction. An incomplete investigation must remain incomplete in the answer; fluency should not fill the missing link.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Pillars 4–6: constrain actions and evaluate how the answer was reached
The fourth pillar, governed actions, turns organizational restrictions into limits on agent behavior. Teams need to distinguish read access from write access, then specify the conditions under which writes are allowed. Those conditions depend on the team, organization and industry. A useful diagnosis does not by itself establish permission to execute its recommended fix.
The fifth pillar, learning systems, treats interaction as a source of information. The investigation itself provides experience, but so does the engineer’s response: accepting a finding, rejecting it or steering the agent toward another line of inquiry. Those responses offer different evaluation signals. The talk describes their role without specifying whether learning changes stored context, prompts, model parameters or another part of the system.
The sixth pillar, evals, examines more than the final answer. Krovvidi describes five levels:
- Positive reinforcement: Capture feedback that the investigation or result was useful.
- Negative reinforcement: Capture feedback that something was wrong or unhelpful.
- Path tracing: Inspect how the agent arrived at its solution.
- Engineer comparison: Score that investigation against how a strong engineer would approach it.
- Confidence calibration: Assess how the agent assigns confidence to its conclusions.
Together, these distinguish an answer that happens to be right from an investigation whose reasoning and confidence deserve trust.
Evaluation is the starting and ending point of the architecture. A new model, new use case or architecture change should be tested against the evaluation system. This connects the model treadmill back to operational reliability: an upgrade must be assessed for the investigation it produces, including its route to the answer and the confidence it reports.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The demo: a Grafana alert becomes an investigation
The demo begins in a familiar place: Grafana sends an alert into a common Slack channel. Resolve automatically picks it up and starts investigating, then returns an initial root cause, evidence, recommendations, contributing factors and rejected theories. The useful question for the Slack view is how much of that reasoning an engineer can inspect before opening the deeper investigation.
Inside the investigation, the alert acts as the prompt that launches agents. Krovvidi identifies investigators as one category: they first establish the issue, gather metrics and traces, locate the failure using traces and logs, then correlate it with change events, deployments, code and infrastructure. He mentions two internal categories but explains only the investigators. The important mechanism here is the movement from an alert’s symptom to evidence across four source types: code, infrastructure, knowledge bases and observability platforms.
Where does the investigation’s information come from? The flow below separates the alert that starts work from the four source types that inform it. Those sources feed the investigation alongside one another; they are not four sequential phases. The resulting root-cause report depends on reasoning across them.
One trigger, four kinds of evidence
Read the diagram as text
- Grafana. trigger — Produces the alert used in the demo.
- Slack channel. trigger — Receives the alert that Resolve picks up.
- Resolve investigation. reasoning — Uses the alert as a prompt; investigators locate the issue and correlate evidence.
- Code. information — One source type used in cross-domain reasoning.
- Infrastructure. information — Provides system context for the investigation.
- Knowledge bases. information — Another source type used by the investigation.
- Observability. information — Metrics, traces and logs help identify and locate the failure.
- Diagnosis and alternatives. output — Root cause, evidence, recommendations and rejected theories.
- Grafana → Slack channel: sends alert.
- Slack channel → Resolve investigation: alert triggers work.
- Code → Resolve investigation: information.
- Infrastructure → Resolve investigation: information.
- Knowledge bases → Resolve investigation: information.
- Observability → Resolve investigation: information.
- Resolve investigation → Diagnosis and alternatives: produces findings.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A coincidental outage tests the causal explanation
The investigation reports a high failure rate and traces it to stale integrations that remain live in the system. A GCP outage occurred at the same time, making it an attractive competing explanation. Traffic spikes provide another thread an engineer might investigate. The reported result favors the stale integrations and shows other theories as ruled out. The verbal walkthrough does not supply the intermediate evidence needed to independently verify that causal chain or the exact tests used to reject each alternative.
Why is the simultaneous outage insufficient? Timing gives the investigation a candidate cause, but the diagnosis still needs evidence connecting that event to this failure. Krovvidi challenges the result by asking whether it relates to the GCP outage, then discusses questions about deployment errors that also happened at the same time. The useful comparison is between the evidence supporting stale integrations and the evidence bearing on those competing explanations.
This is also a test of agreement under pressure. Resolve’s stated design keeps answers tied to the causal chain it has established instead of changing the diagnosis simply because a user pushes another theory. The demo’s lesson is about the basis for revising a conclusion: an engineer should be able to interrogate it, and new evidence should matter more than insistence.
The investigation then becomes a shared working space. A teammate can be pulled in directly from Slack to inspect the result and collaborate in a virtual war room. That completes the earlier multiplayer requirement: the agent does investigative work, while engineers can question the findings and bring other people into the same context.
The closing distinction is architectural. These six pillars address one axis: making AI useful for a particular production problem. Putting that system in front of customers also requires a reliable platform around it. Krovvidi leaves those conventional product and platform concerns outside the detailed walkthrough, but they remain necessary. A harness that reaches a defensible diagnosis is only one part of a production-ready service.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Full transcript
Loading transcript…