The Production AI Playbook: Deploying Agents at Enterprise Scale

Should the team use GPT or Claude? That question can consume a project before anyone decides how to tell whether the resulting application works. Sandipan Bhaumik introduces this problem from his work taking customer demos into production: at the time of the talk, he describes himself as a Data and AI Technical Lead at Databricks, following five years as an AWS Principal Architect. His experience spans distributed data platforms, B2B software and regulated financial services.

The familiar sequence starts with leadership pressure to do something with AI. A team chooses a model, builds features and demonstrates them against predictable data and limited scenarios. Leadership signs off. A few weeks into production, real customer questions produce answers that do not resemble the demo. The investment has bought an impressive demonstration without the machinery needed to understand or correct its failures.

Three missing capabilities explain why recovery is so difficult:

• Observability: The team cannot trace the decisions and operations behind an answer.
• Evaluation: Accuracy, latency and groundedness are discussed, but no one has defined the business outcome or built a system that measures it continuously.
• Governance: When something fails at three in the morning, responsibility is unclear—including ownership of the data that fed the response.

Bhaumik developed the framework from these gaps and reports applying it across multiple customer organizations.

Define the production system before choosing its model. The five responsibilities are evaluation, observability, data foundation, orchestration and governance. Consider all of them before starting, then build them gradually; their order is useful guidance rather than a rigid project schedule.

Evaluation precedes code, feature selection and model debates. Observability supports both engineering diagnosis and production approval: Bhaumik describes tracing as an onboarding requirement in many regulated organizations he encounters, including in Europe. That experience should not be read as a blanket legal requirement for every European AI application.

The data foundation has two parts. Question data supplies the information needed to answer users, whether through training or API access. Tracking data records what the system did and needs its own collection, storage and serving strategy as agent deployments grow. Multiple agents also introduce coordination dependencies: they exchange messages, wait for responses and require explicit orchestration choices. Governance closes the loop with accountability, data security and protection against injection that could cause harmful behavior or reputational damage.

A banking chatbot needs a more precise specification than “answer accurately.” What accuracy is acceptable? Which false positives can the business tolerate? How many simple customer queries should the AI handle without a human support agent? Deflection connects the chatbot to a business purpose: reducing the human workload for questions the system can safely resolve. Those outcomes need numerical targets and a way to track them.

Build the evaluation dataset with domain experts. Collect the answers human support agents actually give, including their handling of ambiguous questions, gray areas and edge cases. Then automate the comparison: submit a question, capture the AI response and assess it against the expected answer. Extend that pipeline to production responses so the same business goals remain measurable after launch.

Evaluation has three layers, each answering a different question. Start with inexpensive checks: regular expressions can validate email or phone formats, while established machine-learning models can identify entities, classify intent and detect personal information. These checks remove problems that do not require a generative model to judge.

The semantic layer asks whether an answer is safe, relevant and grounded. An LLM judge is a separate model given explicit assessment criteria, the primary model’s answer and, where appropriate, the expected answer from the evaluation dataset. Bhaumik points to MLflow as a way to run custom judges automatically on traces.

The behavioral layer checks the path taken to the answer: tool selection, loops, duplicate calls and retries.

In the account-balance example, the answer is correct, but the agent makes three database calls to obtain it. Failures or retries can hide behind a successful final response. At thousands of daily queries, repeated work becomes an operational expense that answer-only evaluation misses.

Consider a customer who believes an overdraft charge is illegitimate and asks the chatbot to waive it. Bhaumik uses a simplified trace from a retail-banking project to show the operations behind the response:

1. Classify intent, recording the classification, duration and confidence.
2. Retrieve the customer account through an API connected to the customer database.
3. Retrieve overdraft policy from a vector database used for retrieval-augmented generation, or RAG.
4. Reason over the account and policy to determine the response.
5. Run final guardrail checks, then respond to the customer.

The trace makes each stage inspectable instead of leaving the team with only the final answer.

If the customer disputes that answer, the team can investigate which account information and policy documents the agent used. Without that evidence, support may be reduced to making an uninformed concession. Traces also expose duplicate calls, turning an unexplained cost or delay into a specific operation that can be corrected.

Online monitoring can connect these observations to an explicit recovery policy. Bhaumik’s example allows at most three retries, then reports the failure or hands it to a human. The application must implement that control path: automatic trace scoring alone does not stop a tool call or enforce a retry limit. This distinction matters when using background LLM judges, which can evaluate a trace asynchronously while the application continues running.

Bhaumik estimates that data foundation consumes 60% of his time on typical projects. Enterprise data was largely organized for humans, who can notice a suspicious report and ask someone to correct it. An agent may instead consume the same error and confidently propagate it into a customer response. Data quality therefore becomes part of the behavior of the application.

Question data and tracking data need distinct plans. The first supports the answer. The second supports auditors, regulators, online monitoring and LLM judges. Trace schemas, collection and serving cannot remain incidental logging decisions when multiple teams depend on those records.

The Databricks architecture illustrates how these responsibilities can be layered. Bhaumik describes a foundation built on Apache Spark, MLflow and Delta Lake, with cloud storage on Google Cloud, AWS or Azure. Delta Lake supplies database-like behavior and incremental data management over stored datasets. More precisely, its transactional table mechanism uses Parquet data files and a transaction log; raw images or videos in object storage do not automatically become transactional tables simply by adding a generic manifest.

Unity Catalog adds centralized permissions, sharing, discovery, ownership and metadata. Table descriptions, column descriptions and PII tags provide context that an AI integration can use when querying governed data. Above that foundation sit Mosaic AI for model and application work, data warehousing, BI and Genie for natural-language SQL querying.

The same foundation can collect traces from heterogeneous deployments. Enterprises use multiple agent frameworks, including LangChain, and multiple clouds; a shared tracing layer gives their operational teams a common source of evidence. First-line support and defense teams need health dashboards. Genie can support text-to-SQL exploration, while Databricks apps and coding agents can provide custom workspaces. Bhaumik presents Agent Bricks and MLflow as supporting automated judges and drift monitoring. The architectural goal is to serve those different consumers from shared tracing data, wherever the agents run.

Moving from one agent to several introduces a control-flow decision. In an orchestrator-worker pattern, a central orchestrator distributes work to agents with specialized skills. Every request passes through that control point, so its logs provide a natural starting place for diagnosis.

In choreography, autonomous agents subscribe to events on a message bus. Work that does not depend on another agent’s result can proceed in parallel. For a mortgage application, one agent might inspect customer details while another examines approval details. Avoiding unnecessary round trips through an orchestrator can reduce latency when those tasks are independent.

Human-in-the-loop coordination introduces an escalation point: a threshold crossing or low confidence brings a person into the workflow to inspect the agent’s work and decide what happens next.

The pattern does not finish the architecture. Shared state, fault tolerance and scaling still need explicit treatment; Bhaumik develops those concerns in the companion talk From Chaos to Choreography.

AI governance extends beyond permissions on source data. Audit trails should capture actions, user connections and requests, while pre-validation checks personal information using techniques such as named entity recognition and regular expressions. Bhaumik reports detecting 47 PII breaches during the customer’s testing phase through this validation layer.

Treat prompts as code under change management. A prompt edit needs more than a Git commit: it needs the review and release discipline applied to application changes. Model upgrades require a similar process. A provider’s public benchmark results do not establish whether an upgraded model will work with the enterprise’s questions, data and risk constraints. Compare candidate models on the organization’s own evaluation dataset, and retain the ability to switch models rather than depending on one indefinitely.

Bhaumik presents Agent Bricks as Databricks’ effort to bring these production responsibilities into its agent tooling, making the five pillars easier to implement together. The surrounding platform still matters: evaluation, tracing, governed data and operational ownership are responsibilities of the complete application.

The retail-banking case makes the sequence concrete. Bhaumik reports roughly 20,000 customer calls per month, with about 60% involving simple questions such as account balances or overdrafts. The bank wanted to reduce reliance on human agents for those queries. An earlier proof of concept had consumed about 85K over six months without succeeding; the currency is not specified. The team lacked visibility into failures, a reliable measure of success and clear accountability.

The replacement goal was for the AI to handle the simple queries representing 60% of total user queries. In the new eight-week proof of concept, the team deferred model selection until week seven. During weeks one and two, it collected 200 cases from actual human-support interactions and defined the success criteria. Alongside the deflection goal, the accuracy target was approximately 85%; latency and other operational targets were also established. These were project targets, not achieved results.

Next came an automated feedback loop:

1. Capture the user’s question and the agent’s response.
2. Compare the response with the evaluation dataset and assign a rating.
3. Send below-threshold cases to a human for review.
4. Correct the underlying problem, which might involve a prompt or tool-calling behavior.
5. Add the failure to the dataset so future evaluations can catch a recurrence.

The evaluation dataset is a living system. The initial collection is a starting point, not a prescribed dataset size or a finished test asset.

Foundation work overlapped that early evaluation work. Bhaumik describes checking database access, security, traceability, distributed storage and trace collection in the second week. This project used direct API calls to query databases; MCP was not part of that implementation. Once the instrumentation was in place, testing could expose duplicate calls and help investigate declining customer satisfaction.

In weeks seven and eight, model selection became an empirical comparison: run candidate models against the same questions, compare their responses with the expected answers and calculate accuracy. That shortened the earlier open-ended model debate. Production launch followed the integration of observability and the evaluation layers into a system whose behavior was visible, measurable and accountable.

The six-week post-launch results slide reports 87% accuracy, 62% deflection, a 65% reduction in response time and CSAT of 4.4/5. These are the customer-case results presented in the talk; the scoring definitions, sample sizes and response-time baseline are not supplied.

A subsequent incident showed why the measurement system mattered. The bank changed interest-rate policies and notified customers by email and through its mobile app. Customers then asked the chatbot follow-up questions and received incorrect answers. Thumbs-down feedback exposed a drop in customer satisfaction.

The team detected that decline through its measurement system and followed the traces to the documents used for retrieval. The agent was consulting an outdated policy: the new document’s embeddings had not reached the vector database. The team fixed the update problem after identifying this stale-data path. The failure was therefore actionable at the data layer, rather than remaining an unexplained decline in chatbot quality.

A production incident playbook defines what happens when the system fails. Bhaumik includes one among the artifacts accompanying the talk. Its workflow connects the capabilities developed so far:

1. Detect the problem through evaluation dashboards.
2. Diagnose it through traces.
3. Contain the impact, for example by withdrawing a problematic prompt version or diverting work to humans.
4. Fix the underlying issue using judge reports and evaluation results.
5. Preserve the failure as a regression case in the growing test library.

For more involved recovery, he points to saga, compensation and circuit-breaker patterns covered in the companion orchestration talk.

The playbook must connect to existing IT service management, or ITSM, systems. Detecting a failure is insufficient if no responsible person receives the alert. Integration supplies that operational handoff and helps coordinate action before downstream systems are affected.

The first implementation can be small: define success in business terms, collect examples of good answers and automate a comparison in Python. Capture a response and make its assessment repeatable before expanding the system.

For a structured account-balance response, a small fixture can separate answer correctness from unnecessary database work. Here, 125.00 and the record shape are illustrative teaching values; the three repeated calls follow the earlier account-balance example. The output records the two findings independently, so a correct answer cannot hide repeated work.

from decimal import Decimal


def evaluate_balance(case: dict, run: dict) -> dict:
    answer = run["answer"]
    expected = case["expected"]
    balance_calls = [
        call for call in run["tool_calls"]
        if call["name"] == "get_account_balance"
    ]
    return {
        "case_id": case["id"],
        "answer_correct": (
            Decimal(answer["balance"]) == Decimal(expected["balance"])
            and answer["currency"] == expected["currency"]
        ),
        "balance_call_count": len(balance_calls),
        "repeated_balance_lookup": len(balance_calls) > 1,
    }


case = {
    "id": "account-balance",
    "question": "What is my account balance?",
    "expected": {"balance": "125.00", "currency": "USD"},
}
run = {
    "answer": {"balance": "125.00", "currency": "USD"},
    "tool_calls": [
        {"name": "get_account_balance"},
        {"name": "get_account_balance"},
        {"name": "get_account_balance"},
    ],
}

print(evaluate_balance(case, run))

A production pipeline can replace the fixture with captured responses and traces, while retaining the explicit expected result and assessment fields.

As the test library grows, assign an owner and categorize cases by the problems they cover. A failure to require authentication before answering a protected account question belongs in a security category. Those categories make it possible to relate a change to the failures it should address, rather than treating the library as an undifferentiated collection of examples.

Prompt history needs enough detail to explain the behavior of each version. Record when a prompt changed, why it changed, the failure that triggered the change and the correction expected from the new version. A vague commit message preserves the text difference but loses the operational reason for it, making later investigations harder.

Behavioral evaluation also needs a cost policy. Fixing a wrong tool call can require replaying agent behavior across a growing dataset; repeatedly running the full library makes each change expensive. Bhaumik suggests running a small subset for prompt changes in continuous integration, then running the full suite when merging to main. This preserves frequent feedback while reserving the broadest, most expensive check for the integration point.

The closing resource slide points to Google Drive templates, an evaluation checklist and guidance for setting up tracing with open-source tools. Their intended use is practical: establish an evaluation and inspect traces in a test environment before committing to a toolset.