# The Half Life of Agent Infrastructure — Ben Kus, Box

Ben Kus, Box

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/sM1iYgz93HI-half-life-agent-infrastructure) · [Video](https://www.youtube.com/watch?v=sM1iYgz93HI)

## The Half Life of Agent Infrastructure

Ben Kus, CTO of Box, explains why agent stacks can age in months, how repeated rebuilds strain engineering teams, and how replaceable layers plus customer-relevant evaluations turn adaptability into an operating discipline.

### Enterprise advice meets a faster cycle

[0:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=12s)

Ben Kus approaches agent infrastructure as Box’s CTO and a career enterprise-software builder. That context sets the scale: Box holds more than an exabyte of data for tens of millions of users across hundreds of billions of files and other pieces of unstructured content. Tokens introduce another dimension; he puts current usage around a trillion tokens and expects it to reach 10 trillion sometime soon, without specifying a measurement period. The lesson is aimed broadly, but its sharpest constraints come from large enterprises with extensive data, many users, and long-lived technology commitments.

![Marks the talk’s pivot: the traditional advice breaks down because generative-AI infrastructure changes much faster.](https://ai.engineer/api/talk-essay-media/b4f24758c4228b31bfc3e7b6d1a98bad5a6b29e534d94bca57ee4753912103e9)

Marks the talk’s pivot: the traditional advice breaks down because generative-AI infrastructure changes much faster.

The internet, mobile computing, cloud infrastructure, and now AI have each created openings for new companies while forcing established ones to adapt. Kus’s conventional advice through those transitions followed a compounding sequence: choose a scalable, reliable stack; become deeply competent in it; use that competence to deliver customer value; then optimize the product for speed, cost, and capability. Staying put preserves expertise and avoids repeatedly paying migration costs.

That advice now has an exception. Earlier waves retained relatively stable foundations—HTTP still underpins the web, while iOS and Android remain the dominant mobile foundations in Kus’s comparison. Generative AI exists as a recognizable category, but many of the models, retrieval methods, agent patterns, and execution environments beneath applications keep moving. Deep specialization can therefore accumulate around a layer that is already losing its lead.

### Last year’s best answer can become this year’s migration

[4:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=270s)

Kus’s own previous conference talk supplies the uncomfortable example. He recommended graph-based agents: developers arrange model-powered nodes into a workflow, and the agent traverses the graph to complete the task. The architecture makes execution inspectable and gives developers direct control over the workflow. An attendee later thanked him because it fit his problem exactly. Kus still likes the pattern, but now considers it somewhat dated. The advice was reasonable when given; a more capable approach emerged afterward.

![Representative frame for the rapid progression in model selection, culminating in adaptive routing among models.](https://ai.engineer/api/talk-essay-media/c2a659dfe6c6a36febffddb23bae7b3e301950bc6551dab775fa8d3a138d58df)

Representative frame for the rapid progression in model selection, culminating in adaptive routing among models.

The same compression appears across three parts of the stack:

- **Model selection:** Training or fine-tuning gave way to hosted frontier models; cost encouraged open-weight models and self-hosting; enterprise contracts created bring-your-own-key and bring-your-own-model requirements; adaptive selection now routes work among larger and smaller models according to task performance.
- **Agent control flow:** A single model response became chain-of-thought reasoning, then developer-authored graphs, then agents that create their own plans. Dedicated subagents compete with general recursive agents equipped with skills, while sandboxes let agents write and execute code. Bring-your-own-harness support moves the integration boundary outward again.
- **Retrieval:** BM25 and keyword search gave way to embeddings and approximate nearest-neighbor retrieval, then graph approaches, hybrid lexical-semantic retrieval with rank fusion, and agentic search that iterates over those mechanisms. Kus’s current preference retains hybrid retrieval underneath an agent, showing that a newer layer may compose an older technique rather than erase it.

Each change moves a different responsibility. Adaptive routing turns model choice from a one-time configuration into a runtime decision. Planning agents move workflow construction from developers toward the model. Sandboxes give the agent an execution environment, not merely a list of remote tools. Agentic retrieval turns search into an iterative activity in which the agent can reformulate queries and inspect results. Kus presents these as successive leading candidates, not permanent winners; the talk provides no comparative evaluations that establish a universal best architecture.

### A half-life measured in months changes the planning model

[9:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=541s)

The preferred architecture can move because several inputs move together. Better instruction following can make autonomous planning practical where explicit graphs were previously safer. Faster and cheaper hardware changes how many tokens an application can afford. Enterprise customers may standardize on one agent, demand support for several platforms, or use both. New techniques and large investments keep adding alternatives. The result is not merely faster model releases; it is continuing movement in the application boundary around the model.

![Summarizes the implications for buyers and investors before Kus gives the operating imperative: get good at changing.](https://ai.engineer/api/talk-essay-media/2edabc3b8cc23ea2d4cd4a0990f8a56f861d2542057bfa70389b9ccb59dc185a)

Summarizes the implications for buyers and investors before Kus gives the operating imperative: get good at changing.

Other infrastructure does not move on the same clock. Large databases, identity and access controls, engineering-team design, and multicloud storage can remain useful for years. Switching those systems is expensive and risky: migrations take longer than expected and break things. A technically better option does not automatically justify replacing a working database or storage layer.

Kus characterizes ordinary infrastructure as having a three-to-five-year half-life, while AI technology may have one measured in months. This is a planning metaphor rather than a measured decay rate: it means that a few months after adopting a leading approach, a team may face a significant chance of wanting to replace it. What changes when those two clocks coexist? The comparison below makes the mismatch visible: stable surrounding systems reward depth and rare migration, while the agent layer requires routine reconsideration.

The cost lands differently across roles. Engineers watch recently learned skills and completed implementations age. Startups can disrupt an incumbent and then be disrupted by the next AI wave. Buyers risk signing long contracts for a technical direction they soon want to leave. Investors may discover that a seemingly decisive opportunity has moved. The shared requirement is no longer picking correctly once. It is becoming good at changing without changing indiscriminately.

Two infrastructure clocks
Conventional infrastructure still rewards accumulated expertise and infrequent migration. Agent infrastructure may need reassessment within months, so replaceability becomes part of the design.
- Conventional infrastructure — Typical planning half-life: three to five years; go deep and switch rarely because migrations are costly.
- Agent infrastructure — Planning half-life may be months; a recently adopted approach can soon face a more capable replacement.
- Operating response — Optimize stable layers while designing fast-moving AI components for reassessment and replacement.
- Conventional infrastructure → Operating response (Rewards specialization and rare migration)
- Agent infrastructure → Operating response (Requires readiness to change)

### The rebuild after shipping exposes the human cost

[12:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=774s)

Box’s agentic search makes the cost concrete. An engineer completed the requested search and deep-research behavior using one approach. The observable result worked. Kus nevertheless asked for a rebuild using a newer, looping agent style because the delivered system was not as capable as the team now wanted. From the engineer’s perspective, the requirement had changed after successful delivery; from the product perspective, a newly available architecture had raised the attainable quality bar.

![Kus identifies rapid replacement as a leadership, morale, team, and company problem—not merely a technical one.](https://ai.engineer/api/talk-essay-media/780fd460ac6ec8d0d433c9e71a5f9f9c5ae3cc0c4b1f59a3a6976fb5a66ce8d1)

Kus identifies rapid replacement as a leadership, morale, team, and company problem—not merely a technical one.

The engineer rebuilt it, and the new version was good. Two months later, the product was scheduled to ship on Tuesday. Kus proposed starting another rebuild on Wednesday. The team’s reaction was rational: perhaps the existing implementation only needed more tuning; perhaps leadership’s next choice would change again; perhaps finished work no longer meant anything. Kus’s answer was that the architecture probably would change again.

This is why rapid replacement is more than a technical migration. Repeatedly superseding working systems can make engineers distrust requirements and leadership decisions. It can damage morale even when each migration improves the product. A team needs an explanation for why work is being replaced, evidence that the improvement matters, and permission to treat the previous choice as reasonable rather than mistaken.

### Make change routine, replaceable, and evidence-driven

[14:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=864s)

The first intervention is organizational: tell teams in advance that change is expected and does not mean they failed. Technology reviewers cannot promise that today’s selection will remain best two years from now. They still have to choose. Explicitly accepting that uncertainty removes an impossible requirement from the decision and lets the team optimize for learning and future replacement.

![Defines the evidence gate for migration: consistent evals covering cost, speed, quality, and capability rather than trend-following.](https://ai.engineer/api/talk-essay-media/5f5e010b4d73ed03097ec40da0425ed19009cf583c83e8a177334d9ca71deb2a)

Defines the evidence gate for migration: consistent evals covering cost, speed, quality, and capability rather than trend-following.

The corresponding technical mechanism is a stable abstraction over swappable internals. Box uses an agent abstraction that allows underlying components to change while the customer-facing agent continues to behave through the same product boundary. Such an interface does not eliminate migration work—the talk does not specify its contract or claim that every component is interchangeable—but it localizes change and reduces how much of the product must move with the implementation.

Box also schedules reassessment. AI technology receives a six-month review even when the team likes the current choice, compared with roughly three years for other technology. A review is not an automatic migration. It creates a regular moment to compare alternatives without interpreting reconsideration as an emergency or an admission that the original choice was bad.

What prevents readiness from becoming trend-chasing? The decision flow below separates review from replacement. A new paper or technique can trigger exploration, but a switch depends on repeatable evaluations using the same inputs, expected outputs, and customer-relevant grades for cost, speed, quality, and capability. Better results justify serious consideration; equal or worse results justify keeping the current system or running a bounded follow-up experiment.

Review often; migrate only on evidence
A regular review creates opportunities to test alternatives. Customer-relevant evaluations, rather than novelty alone, determine whether the migration proceeds.
- Scheduled review — Reassess AI technology every six months.
- Candidate approach — Explore a new model, method, or architecture without committing to it.
- Customer-relevant evals — Use consistent inputs and expected outputs; grade cost, speed, quality, and capabilities.
- Strongly consider switching — Proceed when the alternative performs better on what customers care about.
- Retain or investigate briefly — Do not migrate when results fail to improve; optionally run bounded additional work.
- Scheduled review → Candidate approach (Identify an alternative)
- Candidate approach → Customer-relevant evals (Test instead of following hype)
- Customer-relevant evals → Strongly consider switching (Improves eval results)
- Customer-relevant evals → Retain or investigate briefly (No demonstrated improvement)

### Choose partners for their change history, then keep evaluating

[16:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1014s)

No team can track every model, harness, retrieval method, and execution pattern. Vendors and platforms absorb part of that burden. Current capability remains the first test, and the roadmap still matters, but Kus adds a retrospective question: what happened when the market changed six months or a year ago, and how did the provider respond?

![Adds change history to vendor selection: examine how a provider handled transitions six months or a year earlier.](https://ai.engineer/api/talk-essay-media/5f7c8af86ece808f17adfa82a4ced573dbae86fc9877c2a0163394e9ebc35daf)

Adds change history to vendor selection: examine how a provider handled transitions six months or a year earlier.

Several vendors Kus trusts had reinvented themselves three times in the preceding year. Their history suggested familiarity with agent technology, evaluations, and observability—the supporting systems needed to navigate another transition. Past adaptation does not guarantee future performance, so the buyer still has to determine whether the platform remains good. It does provide more evidence than a roadmap promise alone.

The closing bet is deliberately broader than startups. A company founded recently—or an existing medium-sized or large company—may become dominant during this transition, but its current technical approach will probably change several times along the way. The defensible capability is therefore the ability to replace parts, learn from evaluations, and keep the team committed through successive migrations. Kus calls adaptability the moat, then adds the appropriate qualifier: “until that changes.”

## Key takeaways

- Agent infrastructure can require reconsideration within months even while databases, identity systems, and storage remain on multi-year planning cycles.
- A better architecture does not make the previous choice a mistake. Teams need that distinction stated explicitly because repeated rebuilds affect trust and morale.
- Place a stable customer-facing abstraction over replaceable models, retrieval methods, agent loops, and execution components where practical.
- Review fast-moving AI technology regularly, but migrate only when customer-relevant evaluations show meaningful gains in cost, speed, quality, or capability.
- Evaluate vendors by present capability and by how well they handled previous technical transitions; adaptability is evidence, not a guarantee.

## Resources

- [Building an Agentic Platform](https://ai.engineer/talks/12v5S1n1eOY-building-agentic-platform) — Kus’s prior Box talk provides the concrete graph-based architecture he revisits here, including document extraction, corrective judging, deep research, and the reasons Box initially favored orchestration.

## Related talks

- [Your agent architecture has a half-life of 6 months](https://ai.engineer/talks/X1kp-ABIIxQ-your-agent-architecture-has-half-life-6) — Dan Farrelly develops the same short-horizon problem through stable execution primitives, resumability, sandboxing, and observability.
- [On Engineering AI Systems that Endure The Bitter Lesson](https://ai.engineer/talks/qdmxApz3EJI-on-engineering-ai-systems-that-endure-bitter) — Omar Khattab explores how modular abstractions and interchangeable models can help AI systems survive rapidly changing models and optimization methods.
- [Five Hard-Earned Lessons About Evals](https://ai.engineer/talks/a4BV0gGmXgA-five-hard-earned-lessons-about-evals) — Ankur Goyal gives a practical companion to Kus’s migration rule by explaining application-specific datasets, scorers, and rapid model adoption.

## Slides

### Build for change, not just for now

Box CTO Ben Kus introduces a talk on adapting infrastructure as AI agents evolve.

[0:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=19s) [Box title slide introducing Ben Kus and infrastructure adaptability in the age of AI agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/f3f332ca95f4bf777474976c9dadecadcb819a13d1e9793a3c0ab484ca80599e.webp)

box

Build for change, not just for now:
Infrastructure adaptability in the age of AI agents

Ben Kus
Chief Technology Officer, Box

Interface text:
box
Search with Box AI
Files > Product Development
All Files
AI
Hubs
Notes
Updated
New
Share
CONFIDENTIAL
Product Roadmap
75%
03
Vendor Agreement
Contract generated
9:41
Files
All
Recents
Offline
Box AI
Open
Share
Jennifer

### The scale slide: Box

Kus frames Box’s infrastructure challenges around its scale: over an exabyte of customer data under management, hundreds of billions of secured and governed files, and roughly a trillion tokens and climbing.

[0:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=54s) [Box reports over an exabyte of customer data, hundreds of billions of files, and one trillion tokens.](https://assets.ai.engineer/sites/aiecode2025/sha256/7f44d6a1f8d456c091525ace1fb53487bfdf21f46b12904692b3ef06e9be46ac.webp)

The scale slide: Box

1+ Exabyte
of customer data
under management

100s of Billions
of files, secured
and governed

1 Trillion
tokens
and climbing

### Major disruptive technology trends are opportunities

Ben Kus groups the internet, mobile, cloud, and AI as disruptions: big companies must adapt or die, while smaller companies can disrupt, take bets, and grow.

[1:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=101s) [Internet, mobile, cloud and AI disruptions create pressures for large companies and opportunities for smaller companies.](https://assets.ai.engineer/sites/aiecode2025/sha256/b96fca3ba61967a0d3a2bb69d590d1577422bbb827bb0e70289f525c13b58210.webp)

Major disruptive technology trends are opportunities

Technology disruptions
Internet
Mobile
Cloud
AI

Change brings opportunity
Bigger companies
Adapt or die

Smaller companies
Disrupt, take bets, grow

### For years, the engineering approach was clear

Kus describes the traditional engineering playbook: master a selected technology stack, apply it across products and customers, then make it faster, cheaper and more capable.

[2:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=163s) [Three steps: build scalable, reliable platforms; use the technology across products and customers; and optimize over time.](https://assets.ai.engineer/sites/aiecode2025/sha256/3240f5cf9028958c1c46b6e09ea7c5e20f1145d42f88e0999132b922a012699c.webp)

For years, the engineering approach was clear

1
Build scalable, reliable platforms based on select technologies

2
Leverage your technology across your product and customers

3
Optimize relentlessly over time

### With AI, this approach needs updating

Ben Kus argues that the traditional engineering approach needs updating: the rate of change is much higher today, and most things powering generative AI are changing dramatically.

[3:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=225s) [The slide states that the rate of change is much higher today.](https://assets.ai.engineer/sites/aiecode2025/sha256/b04573ed31cd7cd5ec6ba54551187429299b2a6d9fc1e85123de0599a1d494a9.webp)

With AI, this approach needs updating.

The rate of change is much higher today.

### Last year: Agentic graphs are the key technology

Kus revisits his earlier agentic graph: an extraction supervisor coordinates field preparation, extraction and quality feedback. He now considers the approach a little out of date.

[4:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=269s) [A slide shows an agentic data extraction workflow from a previous talk.](https://assets.ai.engineer/sites/aiecode2025/sha256/6dcf79194469cdbce195dc0c60d47f419330e9b57b52d6b5b366cd789fc294ba.webp)

Last year: Agentic graphs are the key technology

Rearchitecting for agentic approach

AIE
box
Agentic routine for Data Extraction

Document
[illegible]

Box AI Extract Agent
Extracted Data
[illegible]

Extract supervisor

Prepare fields
• Intelligently group fields
• Break fields into groups
...

Extract field data
• Tools to interact with Document (eRAG, OCR, images of docs)
• Uses multiple models
...

Quality feedback
• LLM as judge
• Reflection / recommendations for updates
...

### A better way soon emerged

Kus says the approach from his previous talk lasted months before a better one emerged. Looking through last year’s conference talks, he found the same pattern across many other good ideas.

[5:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=343s) [Text says better approaches emerged within months, alongside an image of a conference agenda.](https://assets.ai.engineer/sites/aiecode2025/sha256/0ecf3f08ed7daaee986a650ce15d78f7ec2e1dfcadfcf435a7bb0a47b89b077d.webp)

A better way soon emerged

Approach lasted months before something better emerged.

And it wasn’t just my talk.

Walk back through most conference agenda from last year. In many cases, there is a better approach today

Conference agenda screenshot:
Intro to GraphRAG — Zach Blumenfeld
Advanced: Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han
Introduction to LLM serving with SGLang — Philip Kiely
A2A & MCP: Automating Business Processes with LLMs — Damien Murphy
Beyond Benchmarks: Strategies for Evaluating LLMs in Production — Taylor Jordan Smith
The AI Engineer’s Guide to Raising VC — Chelcie Taylor
Building Voice Agents with Gemini and Pipecat — [illegible] Kramer
Automating Escrow with USDC and AI — Corey Cooper
Mastering AI Evaluation: From Playground to Production with Braintrust — Doug Guthrie
Building Multimodal AI Agents (From Scratch) — Apoorva Joshi
Solving for the hardest Eval challenge: Building Metrics that actually work — David Karam
Ship Agents that Ship: A Hands-On Workshop for SWE Agent Builders — Jeremy Adams - Casañas
[Additional agenda details illegible]

### Agentic technologies are experiencing rapid change

Ben Kus traces changing approaches to models, agents, and retrieval. Retrieval moved from BM25 and keyword search through RAG and hybrid search to agentic search; each was arguably a leading approach in its moment.

[6:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=373s) [Three columns list approaches to AI models, building agents, and context retrieval.](https://assets.ai.engineer/sites/aiecode2025/sha256/ac3acaf5644956f78167b91c44574153be5fa7643611c77b87e6cce59157f2b6.webp)

Agentic technologies are experiencing rapid change
List of the “leading approaches” AI technology changes in the last couple years

Utilizing Latest AI Models
• Train your own
• Fine-tune
• Frontier model choice
• Open-weight cheap models
• BYOK
• Adaptive model selection

Building Agents
• Single-shot LLM
• Chain-of-thought / reasoning
• Graph-based agent systems
• Looping agents with dedicated subagents
• Recursive agents with skills
• Agents with dedicated sandbox (RLM-style)
• BYO-Harness?

Context Retrieval
• BM25 + keyword
• RAG: Embeddings + ANN
• GraphRAG
• Hybrid (lexical + semantic + rank fused)
• Agentic search

Each of these approaches was arguably the best approach. Until a new approach come along.

### What you learn today about AI may not last long

Ben Kus expects today’s best AI approaches to change as model capabilities, hardware, enterprise adoption, and techniques evolve.

[9:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=543s) [Four boxes identify model capabilities, hardware, enterprise adoption patterns, and new techniques as sources of change.](https://assets.ai.engineer/sites/aiecode2025/sha256/6dec76e70cbb7722568d591a8beef3dd35e6a1ec1b68aded36e16eac4940ed96.webp)

Likely: what you learn today about AI won’t last long
Tremendous investment and change

New model capabilities
Hardware limits and leaps
Enterprise adoption patterns
New techniques

### AI infrastructure is different from other infrastructure

Kus contrasts AI’s rapid change with other infrastructure fields, where approaches from a few years ago still apply. Examples range from databases and access controls to engineering teams and cloud storage.

[10:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=613s) [Five boxes list databases, reliability and observability, identity and access, engineering team scaling, and secure multi-cloud storage.](https://assets.ai.engineer/sites/aiecode2025/sha256/182f16394e2a306a3f98fb6c972f1c1a76a28227e2fcd85622aa8ee291f1f2e3.webp)

AI infrastructure is different from other infrastructure
Approaches from a few years ago are still very relevant

Large-scale databases
Reliability and observability
Enterprise identity and access models
Scaling engineering teams
Multi-cloud secure storage

### Previous wisdom: Optimize for using specific technologies

Kus describes the old infrastructure strategy: go deep on a technology and switch rarely, because migrations, instability and rebuilding expertise are costly. Major infrastructure decisions have a 3–5 year half-life.

[10:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=643s) [The slide advises choosing a technology and switching rarely, citing migration costs and a three-to-five-year infrastructure decision half-life.](https://assets.ai.engineer/sites/aiecode2025/sha256/290028f08945ec47fcc8247bb5b19a389757e9c7873557604213531783777fe9.webp)

Previous wisdom: Optimize for using specific technologies

The old wisdom: pick a technology, go deep, switch rarely.

• Migrations are painful
• Team expertise is expensive to rebuild
• Every switch risks instability
• Switching costs are high

3–5 yrs
Half-life of a major infrastructure decision

### AI technology half-life

Kus argues that even the best available AI technology may face a significant chance of replacement just months after adoption.

[11:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=685s) [The half-life of AI technology is now measured in months.](https://assets.ai.engineer/sites/aiecode2025/sha256/10fe0cbd2e44a16de2484f8d5ca9c13158964261721e1525c616a087beff6885.webp)

The half-life of AI technology is now measured in months

### This challenge affects everyone

Kus warns that engineers’ stacks and buyers’ tools can quickly age, startups can be disrupted by the next wave, and VCs’ bets can lose their appeal.

[11:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=712s) [Four cards describe how rapid change affects engineers, startups, buyers, and venture capitalists.](https://assets.ai.engineer/sites/aiecode2025/sha256/2085284e8656ec39fb176771a33f7cb704976349453ea473e64ca1eca1413f6d.webp)

This challenge affects everyone

As an engineer
The stack you just mastered is about to be out of date

As a startup
You’ll disrupt someone, then get disrupted yourself

As a buyer
You’ll buy the best tool, then be forced to move

As a VC
You’ll fund the hottest idea, then watch it evaporate

### Get good at changing technologies

Kus advises getting good at changing technologies. He says change gets harder as the pace picks up.

[12:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=767s) [Advice: Get really good at changing technologies.](https://assets.ai.engineer/sites/aiecode2025/sha256/6df662b26045027d69a52e59b19fa750d0f127756233802c1f3c06e43cfafda9.webp)

Advice:
Get really good at changing technologies

### Changing often is hard on people and products

Kus recalls asking an engineer to rebuild a working agent, then announcing another rebuild two months later. Repeated changes can undermine morale as people question their discarded work and the next approach.

[12:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=774s) [Three questions express resistance to change; a note describes managing change as a leadership challenge.](https://assets.ai.engineer/sites/aiecode2025/sha256/26e680ba2b86d5769b236c4d6f44e92f665ea6e281509499456e8ab63c5e7aee.webp)

Changing often is hard on people and products

“Why are you asking to throw away my work?”

“If you give me more time, I can make our previous approach work better.”

“How do you know the new approach won't change again?”

Managing change is a leadership challenge: change can destroy morale, teams, and companies

### How to build for change

Kus introduces three approaches to building for change after warning that repeated change can erode morale.

[14:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=863s) [Box slide introducing three approaches to building for change.](https://assets.ai.engineer/sites/aiecode2025/sha256/bf9db0b2006fb000dbfcfc07419f2af766e8f5aa4f93a9fa8584db1b91730bdb.webp)

How to build for change:
Three approaches to consider

box

### Prepare everyone for change

Kus urges teams to expect change and build abstractions that can themselves be abandoned. Box reviews agent technology every six months, versus roughly three years for other major infrastructure.

[14:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=872s) [Build teams that expect change, use replaceable abstractions, and expect mistakes. Box reviews agent technology every six months and other major infrastructure about every three years.](https://assets.ai.engineer/sites/aiecode2025/sha256/c098742220ebb5448d722af037bec19cb607d29d1cde5f7d6a852b8915fe8427.webp)

1. Prepare everyone for change

Build teams that expect to change, make it normal, not a crisis

Build abstractions that let you swap what’s underneath and be ready to abandon those too

You’ll make mistakes and change again. That’s expected, not failure

At Box: we review agent technology every 6 months, other major infrastructure more like every 3 years

### Define what makes you change

Box evaluates new approaches against the same inputs and expected outputs, weighing cost, speed and quality. It considers switching when an approach performs better on its evals.

[15:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=947s) [Slide identifies cost, speed and quality as criteria for change and says Box switches when a new approach beats its evaluations.](https://assets.ai.engineer/sites/aiecode2025/sha256/67c89f69f5cea4c1331691a8619b090df28bccaed78b75eb3da79b91fd5e60d9.webp)

2. Define what makes you change
Not a trending post. Not a new arXiv paper. Change only when it matters.

Cost
Speed
Quality

At Box: we switch when a new approach beats our evals.

### Judge technology providers by how they handle change

Check whether a provider meets your needs now and how it handled past changes. Kus flags vendors who are sure their way is best and resist change.

[16:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1002s) [Slide advises assessing both current needs and a provider's history of handling change, and flags resistance to change.](https://assets.ai.engineer/sites/aiecode2025/sha256/563695ac4a886986420f70a9c0fcc3cc979daf3c3f185413657e4155f367a731.webp)

3. Judge your technology providers based on how they handle change

At some point you bet on a platform to manage this change for you

Don’t just ask
“Does it meet my needs today?”

Ask instead
“Can it meet my needs now?
“How have it handled change before?”

A vendor who is sure their way is best and resistant to change, is a red flag

### Technology foundations will change

Ben Kus bets that companies born today can become major successes, but their technology approaches will probably change multiple times along the way.

[18:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1103s) [Slide says many companies born this year will succeed, but most will change their technological foundations along the way.](https://assets.ai.engineer/sites/aiecode2025/sha256/7ec906510f39f91771fd837b3f2e65f26ec749ec378464275ad7413f9f50de36.webp)

Many companies born this year will become huge successes.

But most will change their technological foundations along the way.

### AI Engineer World's Fair

[19:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1158s) [AI Engineer World's Fair logo in white on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/f930c1425014f572b477712f5b72fcc1a03cbed39f0e84e2cdfd16129bc48e2f.webp)

AI Engineer
World’s Fair

HTTPS://AI.ENGINEER

## Full transcript

[0:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1s) [music]

[0:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=12s) Hi everyone, I'm Ben Kuss. I'm CTO of

[0:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=15s) Box and today I'm going to be talking

[0:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=16s) about uh building for change and

[0:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=19s) specifically around uh AI agents and how

[0:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=22s) to continue to adapt uh your

[0:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=24s) infrastructure as we are all in the

[0:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=26s) middle of this journey.

[0:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=28s) Um, so before I get too far, I will

[0:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=30s) quickly sort of set a little bit of like

[0:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=32s) who I am and uh sort of what I do. Um,

[0:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=36s) for Box, I'm CTO and one of my jobs in

[0:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=39s) my job for my whole career has been to

[0:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=41s) build enterprise software. And so if

[0:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=43s) today um you're from a consumer company

[0:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=46s) or you're uh not involved in enterprise,

[0:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=48s) I hope that a lot of it is still

[0:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=49s) relevant. But in many cases um a lot of

[0:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=52s) the lessons I've learned are enterprise

[0:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=53s) uh specific.

[0:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=56s) So um I sort of will highlight um when

[1:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=60s) I'm thinking and talking about

[1:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=61s) infrastructure when I'm talking about uh

[1:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=63s) the kind of challenges that we face um

[1:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=65s) I'm typically talking about things that

[1:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=67s) are sort of in a scale of like uh like

[1:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=69s) for box we have over an exabyte of data

[1:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=72s) not a gigabyte not a terabyte not a

[1:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=74s) pabyte but an exabyte um and then

[1:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=76s) oftentimes we're I'm thinking in the

[1:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=77s) tens of of millions of users uh the

[1:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=80s) hundreds of billions of things in our

[1:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=82s) case files or content or unstructured

[1:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=84s) content um and then the new stat that is

[1:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=86s) sort of the the one that we talk about

[1:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=88s) is uh tokens. Um so we are now in the

[1:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=91s) ballpark of trillion tokens probably

[1:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=93s) will be 10 trillion tokens sometime

[1:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=95s) soon. Um and this of course is uh uh

[1:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=98s) some of the new and interesting

[1:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=99s) challenges that this kind of scale

[1:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=100s) brings.

[1:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=103s) Um so in my career and um I think maybe

[1:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=106s) many of us here um we've kind of lived

[1:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=109s) through these this these technology

[1:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=110s) changes. And so taking a quick step back

[1:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=112s) um I started uh my career when the

[1:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=115s) internet was sort of becoming a thing uh

[1:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=118s) lived through mobile and sort of this

[2:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=120s) idea of like you know carrying these

[2:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=122s) different devices move to the cloud

[2:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=123s) where you could kind of store and

[2:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=124s) maintain all your data and then of

[2:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=126s) course we're all in the middle of this

[2:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=127s) AI change and I think when you see these

[2:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=129s) kind of technology disruptions when

[2:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=131s) you're sort of thinking about this idea

[2:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=133s) of like all of these kind of have

[2:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=134s) changed all of our lives um and then uh

[2:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=137s) and you're thinking about it from the

[2:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=138s) perspective of a technology leader or a

[2:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=141s) startup or a engineer. Um, you kind of

[2:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=144s) see that like these are where like major

[2:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=147s) companies are born. Um, you see that

[2:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=149s) like big companies adapt or die. Um,

[2:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=152s) small companies are here to disrupt

[2:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=153s) things. They're here to take bets.

[2:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=154s) They're here to grow. Um, I've had two

[2:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=156s) startups. I've been acquired twice. Once

[2:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=158s) in IBM, once in a box. Um, and um, so

[2:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=161s) that we're in the middle of this kind of

[2:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=162s) major opportunity.

[2:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=164s) Um but for um a long tu uh no matter

[2:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=168s) what you kind of come from and what area

[2:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=170s) you're at I uh t typically if you were

[2:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=172s) to ask my advice um and about kind of

[2:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=175s) what makes it you successful as an

[2:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=177s) company as an engineering organization

[2:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=179s) as a technology startup or as a like a

[3:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=182s) company who has a a technology division

[3:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=185s) um I would say no matter what there's

[3:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=187s) kind of three things the first is you

[3:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=190s) need to build scalable reliable uh

[3:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=192s) platforms and select the technology that

[3:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=194s) you care about. Meaning that like

[3:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=196s) there's a lot of ways to do things, but

[3:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=198s) get good at something. Get good at that

[3:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=199s) technology, at that system, at that

[3:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=201s) stack, and then and then keep going with

[3:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=202s) that. Um, and then you leverage this

[3:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=205s) technology so that you do more for your

[3:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=207s) customers. Um, you build your better

[3:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=209s) product, you develop the capabilities,

[3:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=210s) and then you optimize it, make it

[3:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=212s) better, make it faster, make it cheaper,

[3:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=215s) make it uh more capable. And this was

[3:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=218s) sort of the generic enterprise advice,

[3:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=220s) uh, the generic energy advice that many

[3:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=222s) many people would follow. And I think

[3:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=224s) this works really well

[3:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=227s) except now

[3:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=230s) I don't know if this is good advice. It

[3:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=232s) has been across all these major

[3:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=234s) disruptive changes over time. Unclear.

[3:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=236s) In fact, I don't think it's good advice

[3:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=238s) right now because there's a funny thing

[4:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=240s) happening right now which that didn't

[4:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=242s) happen in those previous trends which is

[4:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=243s) that the rate of change is dramatically

[4:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=245s) higher. Um I and you might say look

[4:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=248s) technology always is changing like you

[4:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=250s) know there's other trends things change

[4:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=251s) a lot but not that much um the internet

[4:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=255s) is still based on HTTP the uh uh mobile

[4:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=258s) devices are still um iOS and Android

[4:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=260s) based and so on and so but nowadays um

[4:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=263s) other than the fact that like uh

[4:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=264s) generative AI exists most things that

[4:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=266s) power it are changing and changing

[4:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=268s) dramatically.

[4:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=270s) So last year I was here at this uh at

[4:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=273s) the AI engineering world fair and I gave

[4:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=274s) a speech and I said uh after spending a

[4:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=278s) lot of time on this and thinking through

[4:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=279s) this I think there's a key to this which

[4:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=282s) is a gentic uh uh graph-based approach.

[4:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=285s) The idea was um uh you have a large

[4:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=288s) language model and these nodes and then

[4:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=290s) you sort of put them together and you

[4:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=291s) have the sort of the AI traverse this

[4:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=293s) graph that you set up you build the

[4:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=294s) graph. This is the key approach that and

[4:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=297s) if you use this this is going to really

[4:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=299s) help you sort of build agents because

[5:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=301s) what is anything that we do in life it's

[5:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=303s) a agent it's a workflow um uh it's a

[5:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=306s) it's a way and then if you have an

[5:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=308s) intelligent uh uh a agent it can

[5:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=310s) basically traverse this is the key

[5:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=312s) approach and I believe that at the time

[5:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=314s) and a lot of people did and I still love

[5:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=316s) this approach but nowadays it's sort of

[5:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=320s) a little bit out of date in fact I

[5:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=322s) remember um uh a guy came up to me after

[5:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=324s) my speech last time and he was like the

[5:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=326s) problem you talked about the the answer

[5:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=329s) is just this exactly you were speaking

[5:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=331s) to me thank you so much and and I was

[5:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=332s) happy I I you know I gave a good speech

[5:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=334s) and gave somebody some good advice um

[5:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=336s) and then I remember when we made a

[5:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=337s) change I was like I wonder what happened

[5:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=339s) to that guy I wonder if he's here uh

[5:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=340s) it's uh uh so um but the problem is is

[5:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=344s) that not that it was wrong that that was

[5:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=345s) the best approach but a new way emerged

[5:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=349s) um in fact I started to like look

[5:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=352s) through like all of the last year's uh

[5:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=355s) events and actually go to other

[5:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=356s) conferences like what what did people

[5:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=357s) talk about a year ago and most of them

[6:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=360s) were again nothing much wrong all good

[6:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=362s) speeches all all good ideas but most of

[6:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=365s) them have now a better way there um and

[6:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=369s) uh so um and this is be sort of the gist

[6:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=373s) of of the challenge so if you look at

[6:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=376s) our journey of the technologies the kind

[6:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=378s) of things that we care about um I'll

[6:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=380s) just kind of rapid file here like so

[6:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=382s) let's say that you want to utilize AI

[6:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=383s) models and let's just look the last

[6:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=384s) couple years a long a while ago probably

[6:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=386s) distant memory now like people would say

[6:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=388s) train your own my models or maybe fine

[6:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=389s) tune them like nah that doesn't that's

[6:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=391s) too why bother just use a frontier model

[6:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=393s) use something from openi use something

[6:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=394s) from anthropic use something from from

[6:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=396s) Gemini um and then that's great but it's

[6:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=399s) kind of expensive okay great let's just

[6:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=400s) use openweight models they're pretty

[6:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=401s) close you can use them you can host them

[6:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=403s) yourself you can get some good GPUs um

[6:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=405s) but then uh some companies will come to

[6:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=407s) you and they'll be like look we just did

[6:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=408s) this big deal with open orthropic like

[6:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=410s) can we use our own key or bring our own

[6:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=411s) model like sure you can do that too but

[6:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=413s) then nowadays is probably the best

[6:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=415s) approach is to do an adaptive model

[6:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=416s) selection where you basically are

[6:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=418s) picking the big and smaller models what

[6:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=419s) which model does well and this is kind

[7:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=421s) of the cool new thing maybe I could give

[7:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=423s) a talk on that um or let's say you're

[7:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=425s) building agents like we lot lot of

[7:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=427s) things here like um I mean the word

[7:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=429s) agent hasn't really been around for that

[7:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=430s) long but in that time it used to be like

[7:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=432s) a singleshot uh LLM response call that

[7:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=435s) an agent if you feel like it then you

[7:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=436s) have to say no okay we're chain of

[7:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=438s) thought reasoning now we're going to

[7:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=439s) make a graph-based agent system like I

[7:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=441s) presented last year uh no then it turns

[7:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=443s) that why are you bothering to make

[7:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=444s) graphs when you could actually have an

[7:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=445s) agent just figure out what to do? Make a

[7:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=446s) plan. That's the new approach. That's

[7:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=448s) kind of the way that um Claude sort of

[7:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=450s) uh laid the approach there. And you're

[7:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=452s) like, okay. And then now it's like,

[7:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=454s) well, you if you want to use dedicated

[7:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=456s) sub agents, maybe, but then maybe why

[7:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=457s) not just make a generic agent and have

[7:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=459s) it recursively work and then give it

[7:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=460s) skills. Skills are very generic. They're

[7:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=462s) super helpful. Um and then maybe now

[7:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=464s) it's maybe the idea is not just to do

[7:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=466s) that, but to do it with a agent sandbox

[7:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=468s) so the agent can write code and execute

[7:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=469s) it because that's super useful because

[7:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=471s) agents are great programmers. I'd have

[7:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=472s) them sort of just live in their own

[7:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=473s) computer. Um, and then arguably now

[7:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=475s) that's the best approach or maybe even

[7:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=477s) we're in the world now of like don't

[7:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=479s) even bother with any of that. Just bring

[8:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=480s) your own harness like like like let

[8:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=482s) people select if they want to use one of

[8:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=483s) these other systems and then you know

[8:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=485s) not even just building agents but the

[8:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=487s) technology around context retrieval

[8:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=488s) things like um you know in the old world

[8:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=490s) we were like BM25 and keyword search

[8:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=492s) that's the way to do things but that's

[8:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=493s) like distant memory. Uh obviously the

[8:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=495s) future is is retrieve augmented

[8:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=497s) generation embeddings approximate

[8:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=499s) nearest neighbor that's how we're going

[8:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=500s) to find data. Turns out that doesn't

[8:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=502s) really scale well and it kind of almost

[8:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=503s) mimics randomness as you keep going. So

[8:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=504s) then maybe it's about graphs. It's

[8:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=506s) difficult to get working well. It's

[8:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=507s) probably not the best. Um so then it's

[8:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=509s) about hybrid. You want a lexical and you

[8:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=511s) want to do semantic search and rank fuse

[8:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=513s) those together. Arguably not. Arguably

[8:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=515s) agents are actually way better at

[8:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=517s) finding data because they can find

[8:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=519s) things and apply their intelligence to

[8:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=520s) get to it. So each of these things I

[8:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=522s) just mentioned is arguably the leading

[8:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=524s) approach for that moment over time.

[8:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=528s) If you asked me to give a speech right

[8:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=529s) now on any one of these, I would pick

[8:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=530s) the last. I'd pick the adapted models

[8:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=533s) with dedicated RM style agent and

[8:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=535s) agentic search powered by hybrid.

[8:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=537s) But uh is this the end of this journey?

[9:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=541s) This is not that long of that time here.

[9:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=544s) And so um my guess is the stuff that

[9:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=547s) you're learning today likely won't last

[9:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=550s) that long. Not that it's not wrong, not

[9:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=552s) that it is not the best answer right

[9:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=553s) now, but probably something's going to

[9:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=556s) change. The big thing that changed last

[9:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=558s) year was in my mind Opus 40 to Opus 45.

[9:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=563s) When you did that, suddenly you got to a

[9:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=565s) model that could do instruction

[9:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=566s) following and really high scale. This is

[9:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=568s) kind of to me the beginning the epic of

[9:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=570s) like the new agent models. Uh also uh

[9:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=573s) hardware is getting better, faster,

[9:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=575s) cheaper. Maybe we'll start to use more

[9:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=576s) tokens. Like token usage is off the

[9:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=577s) charts of course and is that good or

[9:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=579s) bad? What's going to change there?

[9:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=580s) Enterprises are adopting things

[9:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=582s) differently whether or not a company has

[9:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=584s) decided to go all in on one agent to

[9:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=586s) rule them all sort of like claude or

[9:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=588s) maybe codec style agent um or maybe they

[9:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=590s) want to utilize uh agents from different

[9:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=592s) platforms and different systems or both.

[9:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=595s) This is going to affect your lives um in

[9:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=596s) addition to things like just the new

[9:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=598s) techniques new interesting uh approaches

[10:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=600s) new technology to power these things. So

[10:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=603s) the fact that everybody in is working so

[10:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=606s) hard on this trillions of dollars

[10:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=607s) investment is actually leading to a lot

[10:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=609s) of this change and again it's happening

[10:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=610s) way faster than than I've ever seen for

[10:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=613s) sure.

[10:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=615s) Um, now if you look back, um, it's not

[10:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=618s) this way with everything else. Like if

[10:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=620s) you go see some of these other

[10:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=621s) discussions, like I've given a speech on

[10:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=623s) some of these topics. I looked at them

[10:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=624s) come some of a few years old. They're

[10:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=626s) pretty good. I still think they're very

[10:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=627s) relevant. You want to talk about large

[10:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=628s) scale databases about uh identity access

[10:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=631s) controls, how to scale engineering

[10:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=632s) teams, how to do multi cloud storage,

[10:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=635s) probably um these are still relevant

[10:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=637s) things today. These do not change as

[10:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=639s) fast despite being high-scale uh

[10:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=641s) interesting powerful uh technologies.

[10:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=645s) So the previous wisdom of saying

[10:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=647s) optimize for specific technologies

[10:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=650s) go deep switch rarely right this what

[10:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=652s) this the reason you do that is because

[10:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=654s) switching is hard migrations suck

[10:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=656s) whenever you migrate you break something

[10:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=658s) every time no matter what um it's always

[11:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=660s) harder than you think even [music] if

[11:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=661s) you know that um and uh and the

[11:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=663s) switching cost is basically high so

[11:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=664s) basically don't do it for most things

[11:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=666s) you're kind of uh just because

[11:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=668s) something's better out there that's not

[11:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=670s) the answer for most infrastructure so

[11:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=672s) typically If you say half life of an

[11:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=675s) agent infrastructure,

[11:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=677s) 3 to 5 years, reevali, see what's out

[11:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=679s) there. We've been using databases like

[11:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=681s) my SQL databases for a long time. It's

[11:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=683s) still pretty good and probably need to

[11:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=684s) replace it soon. But um but now with AI

[11:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=688s) technologies, arguably the halflife is

[11:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=691s) measured in months, meaning a few months

[11:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=693s) after you've adopted what might be the

[11:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=695s) best possible thing, there's a

[11:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=697s) significant chance that you're going to

[11:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=699s) have to replace it coming soon. And this

[11:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=701s) is I think shocking.

[11:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=704s) Maybe I see from some of your reactions

[11:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=705s) that like you're kind of like have

[11:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=707s) experienced this a little bit. Um but

[11:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=708s) this is a different aspect of the way

[11:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=711s) that you build technology.

[11:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=713s) So if you're an engineer, this really

[11:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=716s) sucks because the thing that you just

[11:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=718s) learned and that you're making is now

[12:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=720s) probably going to be out of date soon.

[12:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=722s) Uh no engineer I know likes this. Um uh

[12:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=725s) as a startup, you bet on something.

[12:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=727s) You're like, we're going to go all in.

[12:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=728s) We're going to go on the technology this

[12:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=729s) approach and then we're going to

[12:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=731s) basically uh uh disrupt somebody which

[12:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=733s) probably will but then you see and see

[12:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=735s) now like the first phase of AI companies

[12:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=737s) are starting to get disrupted by the

[12:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=738s) next phase. If you're a technology

[12:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=741s) buyer, you're a leader of a company, you

[12:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=742s) buy technology, you you you select uh

[12:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=745s) open source models, you select vendors,

[12:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=748s) there's a significant chance that

[12:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=749s) whatever you just bought is not going to

[12:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=751s) be the approach you're going to invest.

[12:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=753s) That's you know, good luck doing a

[12:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=755s) three-year deal like on on things about

[12:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=757s) about this kind of stuff. Or if you're

[12:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=758s) VC, uh maybe the coolest best thing that

[12:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=761s) everybody agrees is the greatest

[12:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=762s) opportunity is no longer going to be the

[12:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=764s) opportunity soon um because everything's

[12:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=767s) changing.

[12:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=769s) So here's my advice. Get good at

[12:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=771s) changing.

[12:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=775s) It's almost silly to say because you

[12:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=777s) know obviously technology changes.

[12:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=778s) Obviously it's something that is um you

[13:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=781s) know built in. Of course we're all going

[13:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=782s) to change. We've done this for a long

[13:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=783s) time. Um it's hard. I think it's really

[13:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=786s) hard and the faster that you do it the

[13:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=788s) harder it is. Uh when I was going

[13:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=790s) through that uh like oh yeah we switched

[13:12](https://www.youtube.com/watch?v=sM1iYgz93HI&t=792s) from the graph based agent to a um to

[13:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=794s) the more looping style deep style agent.

[13:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=796s) I remember very well the conversation

[13:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=798s) with the engineer. He just he's like, "I

[13:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=799s) did it. I got a tic search working and

[13:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=801s) this approach does deep research. It

[13:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=802s) does all this stuff just like you

[13:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=803s) asked." Okay, we're going to switch

[13:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=806s) rebuild it again in this new technology.

[13:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=808s) And he's like, "Wait, what?" Like, "It's

[13:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=810s) working. You did what you're talking."

[13:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=812s) Yeah, but it's not as capable as we

[13:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=813s) wanted it to be. Like, what do you mean?

[13:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=814s) You didn't tell me that before. Like,

[13:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=815s) and then and then so convince him like,

[13:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=816s) "Okay, this is a new approach." And

[13:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=818s) then, you know, he does it and it's

[13:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=820s) good. Two months later, uh, we we're

[13:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=822s) we're actually shipping the product uh

[13:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=824s) on on Tuesday. And then I was like,

[13:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=826s) "Okay, uh, guys, on Wednesday we're

[13:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=828s) going to rebuild it again on the new

[13:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=829s) approach." And they're like, "What are

[13:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=830s) you talking about?" Like, like, um, then

[13:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=833s) they'll say, like, it's it's almost hard

[13:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=834s) on everybody like, "Wait, wait, give me

[13:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=835s) more time. I'll I'll make the new way

[13:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=836s) the old way do it better." Um, uh, like,

[13:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=839s) and then also they're skeptical. Like,

[14:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=841s) now you say that, but like this is going

[14:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=843s) to change again, right? Like, who are

[14:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=844s) you to like make these choices? And the

[14:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=846s) answer is, yeah, I'm pretty sure it's

[14:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=847s) going to change again. So, this is, I

[14:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=850s) think, a leadership problem. It's a

[14:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=851s) technology problem. It's a morale

[14:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=853s) problem. It's a team problem. It's a

[14:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=855s) company problem and if you're not

[14:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=857s) careful, it is actually can destroy you.

[14:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=859s) It can destroy a lot of things because

[14:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=861s) people lose faith, they lose morale.

[14:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=862s) It's a problem.

[14:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=864s) So, um if my advice is change um and be

[14:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=868s) ready for change, how are you going to

[14:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=870s) do it? Three things to give you.

[14:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=873s) One, um you just got to prepare people.

[14:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=875s) Well, this is a kind of a people

[14:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=877s) challenge. So when you build your teams,

[14:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=879s) when you talk to them, when you prepare

[14:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=880s) them, if they're in AI world, you got to

[14:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=882s) tell them like expect change. It's

[14:44](https://www.youtube.com/watch?v=sM1iYgz93HI&t=884s) normal. It's not a problem. It's not

[14:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=887s) that you did something wrong. This is is

[14:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=888s) weirdly like um like helps people like I

[14:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=891s) have a a technology review team and and

[14:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=893s) then and then they're like we can't like

[14:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=894s) change. We don't know. We're not sure.

[14:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=895s) We can't tell you that in two years from

[14:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=897s) now this is going to be best. Like

[14:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=898s) that's okay. Uh we're gonna we're gonna

[15:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=900s) build these things that change. So just

[15:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=902s) go with it. You have to pick something.

[15:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=904s) Um, also whenever possible if you can

[15:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=906s) build an abstraction so that it lets you

[15:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=908s) swap out what's underneath. We have an

[15:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=909s) agent extraction in box and you're able

[15:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=910s) to go through and be like uh like select

[15:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=913s) things underneath and the agent still

[15:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=914s) works the same for the customers but it

[15:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=916s) it's better underneath. Um, and the idea

[15:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=919s) is that change is not a mistake and and

[15:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=922s) I highlight like that's very hard for

[15:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=924s) most people and and I and I would sort

[15:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=926s) of just you just I tell them all the

[15:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=927s) time change is not a mistake. You

[15:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=929s) wouldn't nobody knew six months ago.

[15:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=931s) Nobody today will know six months from

[15:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=932s) now. It's Seems very true. So uh at Box

[15:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=936s) we are now in the habit of reviewing

[15:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=937s) every six months no matter what. This is

[15:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=939s) great technology. We love it. Review in

[15:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=941s) six months like because uh which is just

[15:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=943s) completely crazy for everything else

[15:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=945s) that we're doing. Everything else is

[15:46](https://www.youtube.com/watch?v=sM1iYgz93HI&t=946s) like three years. Um also even though

[15:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=950s) change is critical you um you need to

[15:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=953s) define what you mean when change. If you

[15:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=954s) just change all the time there's a new

[15:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=956s) paper it's awesome. You know our CEO

[15:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=959s) Aaron is very active on all the newest

[16:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=961s) things. He's like check this out. Like

[16:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=962s) don't change just because of that. Like

[16:04](https://www.youtube.com/watch?v=sM1iYgz93HI&t=964s) don't change just because it's a trend.

[16:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=965s) Change because you know it matters. And

[16:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=968s) how do you know it matters? Probably

[16:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=969s) pitch you on eval sets. If you're

[16:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=971s) building agents, if you're building AI,

[16:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=973s) make sure that you know what people

[16:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=974s) have. You have the ability to give the

[16:15](https://www.youtube.com/watch?v=sM1iYgz93HI&t=975s) same input, expect certain output. Grade

[16:17](https://www.youtube.com/watch?v=sM1iYgz93HI&t=977s) that cost, speed, quality, capabilities.

[16:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=979s) These are the things that you probably

[16:21](https://www.youtube.com/watch?v=sM1iYgz93HI&t=981s) are going to to be wanting. So for us,

[16:23](https://www.youtube.com/watch?v=sM1iYgz93HI&t=983s) it's easy. Does the new approach work

[16:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=987s) better for our eval sets? What the

[16:28](https://www.youtube.com/watch?v=sM1iYgz93HI&t=988s) customer cares about? If the answer is

[16:30](https://www.youtube.com/watch?v=sM1iYgz93HI&t=990s) yes, strongly consider switching. If the

[16:32](https://www.youtube.com/watch?v=sM1iYgz93HI&t=992s) answer is no, don't bother like or or

[16:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=994s) keep working on a little bit of work to

[16:35](https://www.youtube.com/watch?v=sM1iYgz93HI&t=995s) see if you can make sure that you you've

[16:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=997s) fully explored it. Um and then so the

[16:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=999s) idea is uh build a system that lets you

[16:42](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1002s) be able to change. And then the third uh

[16:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1005s) and final piece of advice here is um

[16:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1009s) almost certainly none of us can keep up

[16:51](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1011s) with everything. It is very hard. Um I

[16:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1014s) think I heard uh Andre Kaparthy uh he he

[16:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1016s) was like everything changes so fast I

[16:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1018s) can't keep up. and you're like you're

[17:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1020s) sort of quite famously good at keeping

[17:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1021s) up and so like what's the hope for

[17:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1023s) everybody else if if that's the case. Um

[17:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1025s) and so but then so what you do is you

[17:06](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1026s) rely on somebody else. You rely on a

[17:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1028s) technology, you rely on a vendor, you

[17:10](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1030s) rely on a platform. Um you when you

[17:13](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1033s) select it and um and then here I think

[17:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1036s) very use I mean like whenever you

[17:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1038s) whenever anybody's bought technology in

[17:19](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1039s) the past I would had advised them like

[17:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1042s) look at what they do now. Double check

[17:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1044s) the road map. Make sure it's good. Make

[17:25](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1045s) sure it's on the path you want but just

[17:27](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1047s) focus on what's available now. But I

[17:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1049s) think something else here is um should

[17:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1051s) do that of course that's most important

[17:33](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1053s) thing but like look back how have they

[17:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1056s) handled change what's their attitude

[17:37](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1057s) towards change how can what can you when

[17:39](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1059s) you talk to them when you read about

[17:41](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1061s) their stuff like what happened six

[17:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1063s) months ago what happened a year ago how

[17:45](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1065s) did they handle that transition many of

[17:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1067s) the vendors that I really like right now

[17:49](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1069s) have reinvented themselves three times

[17:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1070s) in the last year and I now trust that if

[17:53](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1073s) something else comes along they're very

[17:55](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1075s) good at this they understand agent

[17:56](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1076s) technologies they understand the the

[17:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1077s) eval sets they understand the the

[17:59](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1079s) observability systems and then you can

[18:00](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1080s) say ah okay good I hope that they keep

[18:02](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1082s) up and then I now my sort of thing I

[18:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1085s) need to do is just evaluate whether or

[18:07](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1087s) not that's a good platform

[18:09](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1089s) so um making sure that you have this

[18:11](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1091s) sort of platforms that do well is is is

[18:14](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1094s) critical um and um if anybody's

[18:16](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1096s) interested in unstructured content and

[18:18](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1098s) AI associated with it uh Box has a booth

[18:20](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1100s) downstairs happy to talk to you about

[18:22](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1102s) those kind of things

[18:24](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1104s) um and then um I I'll leave you with

[18:26](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1106s) this is um I actually I fully bet and I

[18:29](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1109s) believe that um a company that's born

[18:31](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1111s) this year was born last year um will or

[18:34](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1114s) maybe even a company a medium-sized

[18:36](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1116s) company or a big company will will they

[18:38](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1118s) they'll shoot very high the company that

[18:40](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1120s) will dominate tomorrow is is is now born

[18:43](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1123s) today. Uh but I kind of bet you that the

[18:47](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1127s) technology approach that they have right

[18:48](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1128s) now is probably going to change multiple

[18:50](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1130s) times before they do that. So

[18:52](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1132s) interestingly it's like the challenge

[18:54](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1134s) the advice the thought here is build for

[18:57](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1137s) change

[18:58](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1138s) adaptability arguably that's the moat

[19:01](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1141s) that you have

[19:03](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1143s) until that changes.

[19:05](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1145s) Okay thank you everyone.

[19:08](https://www.youtube.com/watch?v=sM1iYgz93HI&t=1148s) [applause]
