Rise of the AI Architect — Clay Bavor and Alessio Fanelli

Why does getting help from a business so often mean fighting to reach a person? Businesses want to provide good customer care, but the cost makes that difficult. Clay Bavor captures the resulting experience with someone he passed on his walk to work: an iPhone in hand, AirPods in, repeatedly shouting “Representative” into a call. Sierra’s starting problem is this gap between the service businesses want to provide and the service they can afford.

At the time of the conversation, Bavor reported hundreds of customers a little over a year after launch and projected serving hundreds of millions of consumers that year. His examples include ADT, SiriusXM’s Harmony, and a large mortgage originator. Harmony makes the distinction between answering and acting concrete: Bavor describes an agent that answers the phone and can send a satellite signal to get a subscriber’s radio working again.

That capability leads to a broader product thesis: a business’s branded agent could become the interface that follows its website and mobile app. In Bavor’s forecast, companies will need to make that agent both useful and recognizably their own.

Alessio Fanelli frames coding and customer support as leading applications of AI, then asks who owns the personality and integration of a customer-facing agent. The useful historical comparison is the webmaster: someone responsible for a company’s digital storefront, including both its implementation and how it looked and felt. An AI architect similarly owns more than the underlying technology.

The title emerged inside Sierra’s customers rather than from a prescribed organization chart. As Bavor recalls in connection with Bret Taylor’s earlier AI architect conversation, a customer team responsible for building, managing, coaching and improving its agent began calling itself the AI architect team.

The first responsibility is capability judgment: understanding what agents can do well enough to make useful decisions. That does not require pretraining a model or even having worked directly with LangChain. The second is experience and brand judgment: choosing the voice, values, tone and persona through which the company meets its customers. A generic company virtual agent may fit one business; Chubbies’ Duncan Smuthers, with irreverent, bro-like jokes, fits a different brand.

The third responsibility is business judgment: deciding which outcomes the interaction should produce. These responsibilities belong together because technical capability, customer experience and commercial purpose all shape the same agent.

Bavor predicts that this combination will make AI architect one of the fastest-growing job types over the next five years. That is his forecast for an emerging role, not an established labor-market result.

The role does not have a single departmental entry point. Bavor has seen AI architects emerge from technical teams and customer-experience teams, but the latter are especially prominent in his account. People who already understand good service can combine that judgment with enough hands-on technical fluency and an understanding of the business.

Experience in service, support, care or retail supplies something a framework alone cannot: familiarity with what customers actually need and how a good interaction unfolds. Building and coaching an agent gives those employees a way to apply that knowledge across the company’s customer experience, with a new level of responsibility.

A mandate to develop an AI strategy leaves the actual work unspecified. Fanelli asks what distinguishes the people who turn that mandate into something useful. Bavor starts with a willingness to experiment: probabilistic software introduces uncertainty, so adopting it requires some tolerance for risk rather than waiting for perfection.

The next requirement is a concrete customer or business problem. Start with a valuable task, even if it is narrow. One early customer deployment began with processing a return. The milestone worth celebrating was an agent answering the phone, arranging shipment of replacement shoes and giving the customer a return shipping label to print. That completed workflow supplied a place to learn and expand; complexity was not the initial goal.

Once an agent is doing useful work, the team’s responsibilities also change. Bavor describes successful customers reorganizing customer service around helping the agent perform better, rather than simply transferring their old structure into a new tool.

At one customer, Bavor says a team reviews a couple hundred conversations a day. The reviewers coach the agent on wording, decisions, empathy and judgment. This creates a distinct operating responsibility: examining how the agent handles real interactions and turning that evidence into improvements. The organizational design follows the work the agent now needs, rather than the roles that happened to exist before it.

The build-versus-buy question often begins with an incomplete inventory. Fanelli jokes about internal builds taking longer and costing more; Bavor responds with Sierra’s agent iceberg analogy. Above the surface are familiar choices: a language model, LangGraph or LangChain, an embeddings model, a vector database and tool integrations. Those choices can make the project appear close to finished.

Below the surface is the work required to maintain a customer-facing system.

Voice makes the gap particularly clear. An agent must distinguish the primary caller from another speaker and handle interruptions, in addition to reasoning about the customer’s problem. Model selection does not resolve those interaction requirements.

Sierra’s answer combines development in code with tools for people who own the customer experience. Agent OS is the platform; its programmatic and no-code surfaces are Agent SDK and Agent Studio. Engineering teams can build sophisticated behavior, while nontechnical users can build, refine, coach, edit and update the agent. The significant property is that these contributions interoperate on the same system, across chat and phone.

Bavor describes roughly two years of work resolving the problems beneath the surface, and recounts companies returning after nine months of attempting their own builds. These are a vendor’s observations, not controlled estimates of the cost or duration of building internally. Their useful implication is about scope: assess ownership of the full agent lifecycle, including the tools that let customer-experience staff contribute.

Fanelli raises the possibility of experimenting with agent personalities and A/B testing. Bavor answers at the level of the development lifecycle: before optimizing a personality, a team needs a way to test nondeterministic behavior. A single input and expected output cannot adequately exercise an extended conversation.

Sierra uses AI to simulate users, with dozens of personas, simulated accounts and simulated devices. A troubleshooting scenario can include a concrete device condition, such as whether an amber light is on. The test therefore has more context than a question string: the simulated customer has circumstances that the conversation must address.

The development process then follows the customer journey:

1. Understand the goal. Identify what the business is trying to do for its customers and which journeys matter.
2. Model the journey in code. Express the intended behavior while leaving room for unexpected requests and changes of topic.
3. Constrain what must be exact. Use deterministic behavior where needed, particularly for required compliance language.
4. Exercise the agent in simulation. Use the resulting conversations to find missing knowledge and poorly handled corner cases.

Bavor reports tens or hundreds of thousands of simulated conversations before an agent first goes live. This is a reported testing volume, not a count of successful customer resolutions or a reliability score.

After launch, customer-experience and engineering teams need visibility into the point where an agent reaches its limits and hands off to a person. Those interactions feed a closed loop of review, coaching and updates. Improvement here means using evidence from past mistakes to change the agent; it does not require assuming that live conversations automatically retrain the underlying model.

An agent’s limits are not stationary. Fanelli points out that a model or application can fail at a task and then work a month later. Bavor’s approach to staying current starts with research, social discussion and attention to adjacent fields. Veo 3 is his example: Sierra was not using video models at the time, but their capabilities offered clues about what might become possible elsewhere.

Reading is only part of the practice. Hands-on use develops a feel for what the tools can actually do. Bavor places particular weight on the rate of capability change, rather than a snapshot of current performance. Sierra’s early planning anticipated both expanding capabilities and falling token costs; product decisions depended on where those trends were heading.

His concrete routine is simple: keep a Google Doc of problems that GPT-4 could not solve, then revisit them with models such as o1, o3 and o3-mini. The point is to repeat the same difficult problems as models change, not to declare a universal winner from a casual trial. Those checks help inform a development decision: if work starts now, might the required capability and latency be available when the product is ready? Anticipating that intersection is how Bavor thinks about building at the frontier.

The closing question moves from model capability to the interface through which people encounter it. Drawing on Bavor’s Google, AR/VR and Lens background, Fanelli asks what follows text and voice: video, glasses or more ambient agents?

Bavor expects agents to outgrow interfaces that resemble instant messaging or a phone call. His vision is an agent that can summon text, voice, video, imagery and a user interface as the interaction requires. The interface would change form with the task instead of keeping every interaction inside a chat window.

For a trusted personal assistant, he sees glasses and wearables as the eventual vehicle. They could let the assistant see what the user sees and hear what the user hears, then respond through a whisper or a visual nudge. This is a prediction about personal AI, not a description of Sierra’s current customer-service product.

The practical issue is access friction. Retrieving a phone from a pocket or bag and unlocking it creates a separate interaction each time help is needed. If an assistant becomes an extension of how someone navigates the world, Bavor expects it to remain available throughout the day. That is the reason glasses and wearables matter in his closing vision: they could put assistance alongside the experience itself.