The Half Life of Agent Infrastructure — Ben Kus, Box

Ben Kus approaches agent infrastructure as Box’s CTO and a career enterprise-software builder. That context sets the scale: Box holds more than an exabyte of data for tens of millions of users across hundreds of billions of files and other pieces of unstructured content. Tokens introduce another dimension; he puts current usage around a trillion tokens and expects it to reach 10 trillion sometime soon, without specifying a measurement period. The lesson is aimed broadly, but its sharpest constraints come from large enterprises with extensive data, many users, and long-lived technology commitments.

The internet, mobile computing, cloud infrastructure, and now AI have each created openings for new companies while forcing established ones to adapt. Kus’s conventional advice through those transitions followed a compounding sequence: choose a scalable, reliable stack; become deeply competent in it; use that competence to deliver customer value; then optimize the product for speed, cost, and capability. Staying put preserves expertise and avoids repeatedly paying migration costs.

That advice now has an exception. Earlier waves retained relatively stable foundations—HTTP still underpins the web, while iOS and Android remain the dominant mobile foundations in Kus’s comparison. Generative AI exists as a recognizable category, but many of the models, retrieval methods, agent patterns, and execution environments beneath applications keep moving. Deep specialization can therefore accumulate around a layer that is already losing its lead.

Kus’s own previous conference talk supplies the uncomfortable example. He recommended graph-based agents: developers arrange model-powered nodes into a workflow, and the agent traverses the graph to complete the task. The architecture makes execution inspectable and gives developers direct control over the workflow. An attendee later thanked him because it fit his problem exactly. Kus still likes the pattern, but now considers it somewhat dated. The advice was reasonable when given; a more capable approach emerged afterward.

The same compression appears across three parts of the stack:

• Model selection: Training or fine-tuning gave way to hosted frontier models; cost encouraged open-weight models and self-hosting; enterprise contracts created bring-your-own-key and bring-your-own-model requirements; adaptive selection now routes work among larger and smaller models according to task performance.
• Agent control flow: A single model response became chain-of-thought reasoning, then developer-authored graphs, then agents that create their own plans. Dedicated subagents compete with general recursive agents equipped with skills, while sandboxes let agents write and execute code. Bring-your-own-harness support moves the integration boundary outward again.
• Retrieval: BM25 and keyword search gave way to embeddings and approximate nearest-neighbor retrieval, then graph approaches, hybrid lexical-semantic retrieval with rank fusion, and agentic search that iterates over those mechanisms. Kus’s current preference retains hybrid retrieval underneath an agent, showing that a newer layer may compose an older technique rather than erase it.

Each change moves a different responsibility. Adaptive routing turns model choice from a one-time configuration into a runtime decision. Planning agents move workflow construction from developers toward the model. Sandboxes give the agent an execution environment, not merely a list of remote tools. Agentic retrieval turns search into an iterative activity in which the agent can reformulate queries and inspect results. Kus presents these as successive leading candidates, not permanent winners; the talk provides no comparative evaluations that establish a universal best architecture.

The preferred architecture can move because several inputs move together. Better instruction following can make autonomous planning practical where explicit graphs were previously safer. Faster and cheaper hardware changes how many tokens an application can afford. Enterprise customers may standardize on one agent, demand support for several platforms, or use both. New techniques and large investments keep adding alternatives. The result is not merely faster model releases; it is continuing movement in the application boundary around the model.

Other infrastructure does not move on the same clock. Large databases, identity and access controls, engineering-team design, and multicloud storage can remain useful for years. Switching those systems is expensive and risky: migrations take longer than expected and break things. A technically better option does not automatically justify replacing a working database or storage layer.

Kus characterizes ordinary infrastructure as having a three-to-five-year half-life, while AI technology may have one measured in months. This is a planning metaphor rather than a measured decay rate: it means that a few months after adopting a leading approach, a team may face a significant chance of wanting to replace it. What changes when those two clocks coexist? The comparison below makes the mismatch visible: stable surrounding systems reward depth and rare migration, while the agent layer requires routine reconsideration.

The cost lands differently across roles. Engineers watch recently learned skills and completed implementations age. Startups can disrupt an incumbent and then be disrupted by the next AI wave. Buyers risk signing long contracts for a technical direction they soon want to leave. Investors may discover that a seemingly decisive opportunity has moved. The shared requirement is no longer picking correctly once. It is becoming good at changing without changing indiscriminately.

Box’s agentic search makes the cost concrete. An engineer completed the requested search and deep-research behavior using one approach. The observable result worked. Kus nevertheless asked for a rebuild using a newer, looping agent style because the delivered system was not as capable as the team now wanted. From the engineer’s perspective, the requirement had changed after successful delivery; from the product perspective, a newly available architecture had raised the attainable quality bar.

The engineer rebuilt it, and the new version was good. Two months later, the product was scheduled to ship on Tuesday. Kus proposed starting another rebuild on Wednesday. The team’s reaction was rational: perhaps the existing implementation only needed more tuning; perhaps leadership’s next choice would change again; perhaps finished work no longer meant anything. Kus’s answer was that the architecture probably would change again.

This is why rapid replacement is more than a technical migration. Repeatedly superseding working systems can make engineers distrust requirements and leadership decisions. It can damage morale even when each migration improves the product. A team needs an explanation for why work is being replaced, evidence that the improvement matters, and permission to treat the previous choice as reasonable rather than mistaken.

The first intervention is organizational: tell teams in advance that change is expected and does not mean they failed. Technology reviewers cannot promise that today’s selection will remain best two years from now. They still have to choose. Explicitly accepting that uncertainty removes an impossible requirement from the decision and lets the team optimize for learning and future replacement.

The corresponding technical mechanism is a stable abstraction over swappable internals. Box uses an agent abstraction that allows underlying components to change while the customer-facing agent continues to behave through the same product boundary. Such an interface does not eliminate migration work—the talk does not specify its contract or claim that every component is interchangeable—but it localizes change and reduces how much of the product must move with the implementation.

Box also schedules reassessment. AI technology receives a six-month review even when the team likes the current choice, compared with roughly three years for other technology. A review is not an automatic migration. It creates a regular moment to compare alternatives without interpreting reconsideration as an emergency or an admission that the original choice was bad.

What prevents readiness from becoming trend-chasing? The decision flow below separates review from replacement. A new paper or technique can trigger exploration, but a switch depends on repeatable evaluations using the same inputs, expected outputs, and customer-relevant grades for cost, speed, quality, and capability. Better results justify serious consideration; equal or worse results justify keeping the current system or running a bounded follow-up experiment.

No team can track every model, harness, retrieval method, and execution pattern. Vendors and platforms absorb part of that burden. Current capability remains the first test, and the roadmap still matters, but Kus adds a retrospective question: what happened when the market changed six months or a year ago, and how did the provider respond?

Several vendors Kus trusts had reinvented themselves three times in the preceding year. Their history suggested familiarity with agent technology, evaluations, and observability—the supporting systems needed to navigate another transition. Past adaptation does not guarantee future performance, so the buyer still has to determine whether the platform remains good. It does provide more evidence than a roadmap promise alone.

The closing bet is deliberately broader than startups. A company founded recently—or an existing medium-sized or large company—may become dominant during this transition, but its current technical approach will probably change several times along the way. The defensible capability is therefore the ability to replace parts, learn from evaluations, and keep the team committed through successive migrations. Kus calls adaptability the moat, then adds the appropriate qualifier: “until that changes.”