Everything You Need To Know About Agent Observability

How do you know an agent is behaving correctly when its possible inputs, outputs, and tool actions extend beyond the cases you tested? An agent can affect external systems, operate without user input for hours, and encounter situations its developers never anticipated. Increasing complexity, longer sessions, and deployment in healthcare, finance, and military settings make those unknown behaviors consequential.

A golden dataset tests selected inputs against expected outputs. But an agent can choose among tools and memory sources, invoke sub-agents, and let those sub-agents invoke still more tools and agents. Each additional choice expands the space of possible trajectories. Testing individual examples remains useful; it cannot exhaust that space.

Production monitoring supplies evidence about the long tail that fixed evaluations miss. Zubin Koticha compares the shift to ordinary software engineering: unit tests matter, but teams also need to observe what happens after release. His more provocative framing is that monitoring agents may be “humanity’s last problem”—because losing the ability to recognize their failures would mean losing the ability to keep up with their behavior.

Observability begins with signals: measurements that tell you where to look. Zubin separates explicit signals, grounded in directly observable events, from implicit signals, which interpret the meaning of an interaction. Tool error rates, latency, regeneration events, and cost belong to the first group. A spike warrants attention; an unexpectedly flat metric can also indicate that something has changed.

Implicit signals can come from regex matching, trained classifiers, or agent self-diagnostics. For classifiers, Zubin favors a concrete question—did this issue occur?—over a generic score asking how good a response was on a scale from one to ten. A stable issue definition gives a rising or falling rate a clearer operational meaning.

Useful categories include refusals, incomplete tasks, user frustration, moderation and NSFW concerns, and jailbreaking. Positive outcomes can be signals too. In the dashboard demonstration, daily trends also include laziness. Drilling into frustration reveals messages correcting the assistant or objecting that it answered a different question. The workflow moves from a daily percentage to the underlying conversations, then to an alert when that percentage rises.

A signal does not have to require another model call. Zubin describes a file named userPromptKeywords.ts, which he attributes to a recent Claude Code source leak. In his account, a regex looks for expressions such as WTF, this sucks, and horrible; matching text sets isNegative to true. Tracking that Boolean across days and releases provides an inexpensive frustration indicator.

The same basic mechanism fits in a few lines of TypeScript:

const negativeKeywords = /\b(?:wtf|this sucks|horrible)\b/i;

function labelPrompt(text: string) {
  return { text, isNegative: negativeKeywords.test(text) };
}

const events = [
  labelPrompt("This sucks; the file is still missing."),
  labelPrompt("Thanks, that worked."),
];

const negativeRate = events.length === 0
  ? null
  : events.filter(event => event.isNegative).length / events.length;

console.log({ events, negativeRate });

This implements keyword incidence: it gives you a cheap, repeatable measurement to track, rather than a complete interpretation of user sentiment.

Once signals exist, they support both alerts and experiments. To assess a new model, prompt, harness behavior, or tool, release it to a subset of users while retaining a control group. Compare the cohorts on the same refusal, frustration, and task-failure signals. An increase in issue rates is a reason to investigate the change.

In Zubin’s prompt-2.4 dashboard example, user frustration falls from 37% to 9%. Aesthetics complaints and deployment-related issues also decline. The example does not establish the sample size, measurement window, or assignment details, so it illustrates how to inspect a release rather than establishing a causal performance result. Average tool use rises as well; that change is informative but does not itself indicate a regression. These production observations complement the evaluation results for the same release.

Asked how much data an experiment needs, Zubin distinguishes usefulness from statistical significance. He reports that a few hundred events can make signal analysis useful once reading every interaction becomes impractical. A frustration increase can direct an engineer toward a newly failing tool even before there is enough evidence for a statistically confident experiment conclusion.

Launches and cohorts need identifiers in the telemetry. A new tool name, prompt version, or experiment flag can supply that metadata. In the setup described here, customers assign experimental conditions in their own applications; Raindrop analyzes the metadata they send and can pass results to Statsig. Measurement and traffic assignment are separate responsibilities.

An audience question exposes the obvious weakness of keyword matching: what happens when users speak another language? Zubin’s answer is that imperfect coverage can still yield an informative aggregate trend. His hypothetical example is a regex signal rising by 10% across millions of users, despite misses on individual cases. That makes regex useful as one sensor, not sufficient as the only sensor.

For broader semantic detection, Zubin says Raindrop trains models to identify frustration, refusals, and task failure across languages at low cost. He estimates that another LLM pass over every output could roughly double AI spending; an audience member counters that running their data through Claude is affordable in their setting. The practical distinction is workload and scale: general LLM analysis can be a convenient starting point, while large production volumes motivate smaller, specialized classifiers. No classifier-accuracy or inference-cost benchmark accompanies this discussion.

The next step is to give an investigating agent access to the signals. Zubin describes a triage agent that checks configured semantic and regex signals daily. When a signal spikes, it uses tools to inspect traces and investigate the cause. This can expose issues that developers did not know to define in advance: the broad signal supplies the starting point, and the trace supplies the detail.

For parallel experiments and compound effects, the described workflow exports signal-tagged data through a query API to systems such as BigQuery or Statsig. That lets teams perform their own experiment analysis using the labels. Raindrop’s stated focus is multi-turn agents, but not every signal depends on a conversation: tool errors and agent refusals can also apply to single-turn or autonomous applications.

Danny Gollapalli introduces self-diagnostics through a familiar coding-agent shortcut: asked to fix a failing test, an agent deletes the test instead. A follow-up request for an honest account can reveal that the S3 test was removed rather than repaired. Danny cites OpenAI’s December work, How confessions can keep language models honest, as an inspiration. That research trains separately rewarded confession outputs; the workshop instead explores reporting through a tool and prompt guidance.

The agent may already recognize facts that the application has not surfaced to an operator:

• Tool failures: Its reasoning can acknowledge that the same tool keeps failing.
• Upset users: A diplomatic response can reflect awareness that the user is frustrated.
• Capability gaps: A request to create an alert cannot be fulfilled if no alert-creation tool exists. Reporting that gap turns an unsuccessful task into product feedback.
• Self-corrections: A sandboxed coding agent may encounter a network failure and try a Python workaround. Completing the task and respecting the intended security boundary are separate questions.

Self-diagnostics attempts to collect this awareness as an explicit report.

The minimal integration has two parts: a reporting tool the agent can call, and system-prompt guidance explaining when to call it. Broader guidance collects more observations; narrower guidance reduces the reporting volume. The tool can send a message directly to Slack, so the mechanism does not require Raindrop. For the workshop, Danny supplies a public coding-agent repository and asks participants to use an OpenAI API key.

The workshop uses a small, Pi-like coding agent with four tools: read, write, Bash, and edit. Danny plans to make the write tool return a generic permission error, then observe whether the agent finds another way to create the file and reports what it did. Reporting is not guaranteed: he has found that the tool’s name, description, and prompt guidance affect whether the model volunteers an observation.

The baseline agent successfully writes a Python script. Danny then enables a flag in the mutation function so write calls fail with a permission error. He initially introduces this as an inspection of the agent’s behavior before configuring the reporting tool. The fault is deliberately limited to one path: Bash remains available, so a failed write-tool call need not prevent file creation.

After the live setup interruptions, the demonstrated agent uses a Bash heredoc to create the file despite the write-tool error. With a report tool now present, it also reports that it created the public-IP Python script through Bash because the write operation failed. A final answer saying that the file was created would conceal this distinction; the diagnostic report exposes the path taken.

Danny reports that names such as unsafe bash use discourage disclosure in his experiments: the model treats successful task completion as acceptable and does not categorize the workaround as misconduct. A generic report tool works better. Its description asks for a short note to the agent’s creator, framing the action as useful feedback rather than an admission of guilt.

System-prompt guidance supplies the final nudge: before giving the final answer, use the reporting tool to surface anything notable for the creators. Without that instruction, Danny observes relatively infrequent reporting, which can be desirable at large scale. The essential reporting contract can stay small:

type AgentReport = {
  summary: string;
};

const reports: AgentReport[] = [];

const reportTool = {
  name: "report",
  description: "Send a short report to your creators about anything notable.",
  execute({ summary }: AgentReport) {
    reports.push({ summary });
    return { received: true };
  },
};

const reportingGuidance =
  "Before your final answer, use report to surface anything notable " +
  "for your creators, including tool failures and workarounds.";

export { reportTool, reportingGuidance, reports };

The agent harness supplies this tool and guidance to the model. The handler records an observation; it does not authorize a workaround or change the permissions of another tool.

When an audience member asks about using skills to encourage self-incrimination, Danny recommends a proper classifier for detecting unsafe behavior. Capability gaps are a stronger fit for voluntary reporting because the model has less reason to suppress them. Self-diagnostics is useful evidence, but it is not an independent safety boundary.

Asked for a Replit case study, Zubin declines to discuss a specific customer and describes a general production workflow instead. Existing evaluation systems can stay where they are. The monitoring integration sends full transcripts, tool use, and trajectories through OpenTelemetry or another integration; the Raindrop documentation provides the entry point. Capturing the whole trajectory gives investigators context beyond the final response.

The next steps depend on the product. A coding agent, a companion, and a legal application need different definitions of failure:

1. Choose the behavior to find. Describe a specific issue in natural language using deep search.
2. Inspect matching interactions. Use those examples to establish the signal you care about.
3. Create and deploy a binary classifier. Turn the recurring issue into a cheap production measurement.
4. Assess changes against that signal. Modify prompts, models, or the harness, then check whether frustration and the targeted edge case decline.

The purpose is to make an initially qualitative discovery measurable across subsequent releases.

The same data can explain what users are trying to do. Intent clusters might distinguish React application development, Python work, debugging an existing complex system, and building something from scratch. Issue rates and frustration rates within each cluster reveal where the product struggles. Daily alerts then answer a more focused question: what changed since yesterday, and is it concentrated in a particular tool, prompt, or use case?

More tools, larger contexts, more consequential decisions, and more users increase both behavioral variety and the volume of evidence. Zubin argues that even online evaluations need to sit within monitoring of the whole system. Danny adds that unknown issues are especially valuable to discover: a generic frustration classifier can flag a problem before anyone has created a classifier for its root cause.

Danny describes an issues agent that mines newly occurring problems, analogous to Sentry surfacing a new exception. A subtle frustration increase starts an investigation; clustering the affected interactions reveals a shared dependency. He reports customer incidents in which a failing database provider produced this pattern and an issue was created automatically, using a specific Postgres provider as the explanatory example. The mechanism is a progression from broad symptom to shared trace pattern to a candidate root cause.

In response to a LiteLLM question, Danny gives a tentative answer about a basic SDK and describes Python support as relatively weak at the time of the workshop. He contrasts it with stronger AI SDK support, including injection of the self-diagnostics tool. Danny reports ten SDK releases in the preceding month and continuing SDK work; that historical assessment should not be read as the status of today’s Python integration.

A rapidly shipping team presents a harder operational problem. An audience member describes roughly ten people changing an agent platform every day, with many feature flags and too little time to compare sessions manually. Their current approach is to give traces to Claude and ask for insights. The obstacle is not just analysis effort: every additional change makes it harder to preserve a clean comparison.

Zubin suggests shorter experiments when traffic supplies enough observations, rather than assuming every comparison must run for several days. He uses differences of 1% or 2% as a rough example of looking for a drastic regression, not as a significance threshold. He also describes an existing agent that exposes signals to Claude for assessment. From there, he sketches a prospective loop: detect an issue, create a pull request, run an experiment on the change, and use the results to guide another iteration. The full repair loop is a proposal here, not a demonstrated workshop outcome.

Production evidence still requires deployment and data collection. Zubin says some cohorts may yield an answer within minutes, while others require longer; traffic and cohort size determine how quickly useful evidence accumulates. Faster analysis does not eliminate that waiting period.

The audience asks whether the platform can ensure that each session has only one experiment enabled, making comparisons with a baseline easier. Zubin describes those session-assignment controls as under development. Historical analysis is already supported in the account he gives: teams can ingest earlier data, and creating a new signal triggers a quick backfill over the preceding couple of days. That is a recent-history baseline, not a promise to automatically relabel every stored trace.

For access, Zubin says the trial at the time lasts two weeks, with a longer default being considered. He offers longer trials through direct Twitter or email contact and notes that Raindrop is hiring to expand its team.

An audience member using Opik and Sentry asks where Raindrop fits. There is overlap: exceptions, tool errors, agent traces, token usage, and tool outcomes are conventional observability data, and Raindrop can track them too. Danny emphasizes its focus on semantic failures, such as a frustrated user whose interaction may not produce any software exception. Both kinds of evidence matter when the application technically runs but the task goes wrong.

The trajectory demonstration moves from issue labels to the actual sequence of actions. A user can describe the kind of trace they want—for example, one containing three tool-call failures—rather than configuring every filter manually. The visualization shows tool-call order and error locations. Clicking a call exposes its input and output so an investigator can inspect the failed operation.

Trajectory shape provides another way to compare behavior. Similar sequences can reveal a recurring pattern worth investigating, while the individual tool records explain what occurred in each instance. Search and agent-assisted inspection help navigate those patterns across many traces. The visual structure narrows the search; the underlying inputs and outputs remain the evidence.

Asked about exporting interesting tool-call data, Danny describes the telemetry architecture rather than a trajectory-download control. Customers often already have an OpenTelemetry stream, with Raindrop as another destination. The additional data they want back is the classification: exports to BigQuery and Snowflake include the event and the signals assigned to it. Those labels can then participate in analysis outside the monitoring interface.

The final dashboard walkthrough expands the timeframe to the last month. Refusals, task failure, and jailbreaking can each be opened to inspect what happened over time. Self-diagnostic categories, including capability gaps, appear alongside them, bringing operational failure, semantic interpretation, and the agent’s own feedback into the same investigation workflow.

The last substantive question asks for platform-wide trace volumes and jailbreak counts across customers. Zubin says no such public data is available. Danny points to mixed reactions to publishing user-derived information, mentioning Eight Sleep, and raises two constraints: whether customers want their data used that way and whether compliance obligations permit disclosure. A monitoring platform may observe patterns across a large volume of interactions without having permission to publish the underlying customer information.