Building an Agentic Platform

How do you make AI useful over an enterprise’s content while preserving the protections that made the enterprise willing to store it there? That is the starting constraint for Box, an unstructured content platform serving large organizations. Ben Kus, Box’s CTO, reports more than 115,000 enterprise customers and two-thirds of the Fortune 500. For some customers, he says, Box was their first AI deployment because they were concerned about security and data leakage.

The AI capabilities sit above existing infrastructure for managing and protecting content. Kus describes that infrastructure as holding over an exabyte of data and hundreds of billions of files. Adding intelligence therefore means extending a content platform customers already trust, with its infrastructure and services still underneath.

Box began its generative AI effort in 2023, developing document Q&A, data extraction and AI-powered workflows. Extraction became a particularly useful test of the architecture: it looks like a function that accepts a document and returns fields, rather than a conversational assistant. Yet the work required to make that function reliable would bring agentic techniques well beyond chatbot interactions.

Data extraction takes information embedded in documents, contracts and project proposals and turns it into structured metadata. Kus frames the opportunity as roughly 90% unstructured data versus 10% structured data in enterprises; this is his characterization of the problem, not a measured universal ratio. Much of that unstructured content is critical, but historically difficult to automate.

A contract is the concrete example. The document remains authoritative, while its key fields become usable for queries, workflow triggers, search and filtering. The slide shows an agreement beside metadata such as contract type, version, participant, number of parties and expiration date. Extracting those fields gives software a way to act on information that previously required someone to read the document.

This was already the purpose of intelligent document processing, or IDP, which Kus describes as a multibillion-dollar industry. Traditional approaches required specialized models, narrow content categories and large training corpora. Custom vendors or custom ML models could solve particular problems, but the resulting systems were expensive to develop and brittle enough that many companies left critical content unautomated. The practical response was often to collect structured information wherever possible instead of attempting to understand arbitrary documents. Box itself had used ML for years before generative AI offered a more general route.

The first generative approach was straightforward. Working with what Kus calls GPT-2/GPT-3-style models, Box preprocessed documents with optical character recognition, or OCR, then asked a model to extract specified fields. A single call, sometimes with prompt refinement, could use the model’s understanding of language and context without training a document-specific extractor.

Kus reports that generic models from multiple vendors outperformed earlier specialized models in Box’s experience, though he supplies no controlled benchmark. The system supported multiple models, improved as those models improved, and handled varied customer content. OCR appeared to be a manageable preprocessing step. Early customer examples worked well enough to encourage a much broader promise about what the system could extract.

Successful examples raised expectations. Customers brought a 300-page lease with 300 requested fields, complex digital assets and questions that required risk assessment rather than straightforward extraction. These were tasks that could challenge a human reader too: finding a value is different from interpreting a large document well enough to make a judgment.

Input quality then became a separate failure source. Scanned pages contained handwriting and crossed-out text; PDFs and other formats introduced additional complications. When OCR failed, it supplied bad information to the model downstream. Supporting international customers also exposed language problems. Better reasoning over corrupted text could not reliably recover what the preprocessing step had lost.

Even with usable text, the model could lose track of the extraction instructions. Kus contrasts 10 fields over 10 pages with 100 complex, separately instructed fields over 100 pages as illustrative workloads, not measured capacity thresholds. The difficulty was maintaining attention across all the requested fields and their individual requirements at the accuracy enterprises expected.

Accuracy itself was hard to communicate. Traditional ML systems might return a confidence value such as 0.865; an LLM’s assessment of its own answer is not automatically a reliable confidence estimate. Box introduced LLM-as-a-judge checks, but a warning that an extraction might be wrong did not finish the customer’s job. Customers needed correct results, delivered quickly and affordably, including on the harder documents they now expected AI to handle.

Waiting for the next Gemini model or a stronger model from OpenAI was part of the answer: models did keep improving. But model upgrades alone did not resolve the fragility of the surrounding pipeline. Box began treating extraction as an agentic task, even though its user-facing behavior did not resemble a chat agent.

In this architecture, an agent combines instructions and objectives with a model, tools, secure access and memory. A directed graph orchestrates the work. The model may create a plan, or engineers may specify the sequence because they already know which steps the task requires. Agentic execution therefore does not require handing every planning decision to a model; it can include deliberately designed workflows whose individual steps use AI.

That change was controversial internally. Engineers proposed improving OCR, adding postprocessing regular expressions, training specialized ML models or fine-tuning. Each could address a local failure. Kus’s concern was that accumulating such fixes would gradually sacrifice the generality that made the generative approach valuable in the first place.

Box developed a LangGraph-style approach; Kus uses the framework as an architectural comparison, without identifying it as Box’s internal implementation. The external contract stayed the same: document and requested fields in, answers out. Inside that boundary, the team explored reflection, criticism, task decomposition and multiple agents.

The first important step was preparing and grouping the fields. Related values must be extracted together. For a contract, independently requesting parties and addresses can produce three parties and two addresses with no reliable correspondence. Splitting the workload reduces the amount of instruction a model must handle at once, but splitting along the wrong boundaries destroys relationships between the answers.

A concrete way to represent that grouping is to request party records, each with its own address, rather than unrelated lists. The following JSON expresses the field relationship; it is a grouping specification, not an extraction result:

{
  "groups": [
    {
      "id": "contract_parties",
      "recordFields": ["party_name", "party_address"],
      "instructions": "Extract each party together with its address. Keep the association explicit."
    }
  ]
}

With related fields kept together, the system can issue multiple focused queries against the document instead of one oversized extraction request.

The remaining work checks and improves the candidate answers:

1. Cross-check the evidence. Use tools to examine extracted results. OCR can supply text, while images of the original pages provide another view of the source.
2. Consult multiple models where useful. Kus gives an example in which two of three models from different vendors agree on a difficult answer. That agreement is supporting evidence, not proof of correctness.
3. Turn judging into feedback. A judge identifies a problem and sends the task back for another attempt, instead of merely attaching an uncertainty warning to the answer.

These checks take additional time. Kus reports improved accuracy from the architecture, but gives no numerical accuracy result or latency measurement.

The graph also changes how the system evolves. A newly discovered failure may call for a different prompt at one node or another check near the end. It no longer automatically requires rethinking the entire extraction pipeline. A function with a simple interface can contain an intelligent workflow that adapts as its failure modes become clearer.

The same correction loop applies to document Q&A. Box already had a judge assessing whether an answer was good. The next step was to use a poor assessment to trigger reflection and another attempt before presenting the answer to the user. Kus associates that extra work with higher accuracy, while acknowledging that it also increases system complexity.

The shared foundation then supported the announced deep research capability over content stored in Box, analogous to the internet research experiences from OpenAI and Gemini. The workflow searches for data, repeats that search as needed, selects relevant material and double-checks it, then creates an outline and a plan for the report. The displayed graph continues through prose generation and quality review, with feedback loops for information capture and accuracy critique. These are coordinated stages of a larger task, made possible by the same agentic foundation used for extraction.

After this experience, the agentic layer looked like a clean architectural abstraction: an intelligent directed workflow uses models at its steps to accomplish a task. Kus qualifies the recommendation—it is useful for some tasks, not everything. It also addresses a different problem from distributed systems design.

Kus uses one document versus 100 million documents per day to illustrate the distinction, not to report measured throughput. Both concerns matter, but separating responsibility lets one team improve the agent’s behavior while another scales the generic process that runs it.

The deep research system supplied a concrete example of local improvement. Its output was initially sloppy. Rather than redesign the whole workflow, the team added a final node that summarized and reformatted the result according to instructions. Kus describes this as a quick fix, without specifying a duration: the existing research process could remain intact while another step improved its presentation.

Established engineering teams also need experience building these workflows. Hands-on work helps engineers recognize where another intelligent step can solve a problem, and it improves their judgment about the platform they expose to customers. At Box, that includes thinking about MCP servers, the usability of tools and agent-to-agent communication.

Kus’s recommendation is conditional: when a set of AI models can plausibly help solve the problem, build the agentic architecture early enough to benefit from continued improvement and reuse. Looking back at Box’s journey, he wishes the team had established that foundation sooner.

The first audience question asks whether these capabilities are available through an API. Kus confirms that Box takes an API-first approach: callers can invoke agents with arguments, and APIs and tools expose the capabilities for programmatic use. His answer establishes the integration approach without detailing an endpoint schema or deployment procedure.

Evaluation is broader than asking another model to judge an answer. In response to the next question, Kus describes several sources of evidence:

• Standard evaluation sets cover the tasks the system already needs to handle.
• Challenge sets probe difficult requests that customers do not commonly make yet, but may bring as their expectations grow.
• LLM judging and customer feedback add evidence alongside the datasets. Feedback is especially useful because Box has limited visibility into what happens inside enterprise usage.

The challenge sets keep evaluation from stopping at today’s easy cases. Together, these methods assess present readiness and the ability to handle more demanding work; Kus does not provide scores or a detailed manual-review protocol.

The final question asks whether Box has tried fine-tuning agents. Kus initially interprets it as asking why agents are necessary if a model can be fine-tuned; the questioner clarifies that they are asking about Box’s experience with fine-tuning. Kus answers with the team’s position at the time: it generally avoided fine-tuning because every evolution of the underlying models could require repeating that work.

Box supports Gemini, Llama, and models from OpenAI and Anthropic. Maintaining consistent fine-tuning across that range is difficult, while a newer base model may already improve the result. For these particular use cases, Box therefore favored prompts, cached prompts and agentic orchestration. The decision follows from maintaining a platform across providers and model generations; it is not a universal claim that fine-tuning has no place.