Rise of the AI Architect — Clay Bavor and Alessio Fanelli

Talk transcript

1. 0:00

   [on-hold music] So welcome.

2. 0:15

   Uh, I, I know a lot of people might be fam-familiar with you and especially all the amazing work you've done at Google in the past and with Sierra, Sierra overall, but maybe just give a one-minute intro on what Sierra is, who you serve, the scale of it.

3. 0:27

   Yeah. Uh, so Sierra, in a nutshell, we help businesses build better, more human customer experiences with AI. And, uh, concretely, what we're trying to do is bridge this age-old gap between businesses wanting to provide great care, customer service, customer experience on the one hand, and the impossibility of doing that on the other hand because of cost.

4. 0:48

   And I think we've all been there of just, like, being on hold, and it's like 00000. Or, uh, I was walking into work a couple weeks ago and there was a dude, uh, on a call holding his iPhone with AirPods in yelling, "Representative, representative, representative, representative." [laughs]

5. 1:04

   And, um, like, I'm not sure what circle of hell involves, you know, be- waiting on hold and, and trying to get a problem solved, but I'm sure it's one.

6. 1:10

   And we're trying to, uh, bridge that gap. And so, uh, concretely, you know, we're, uh, a year in change after launch. We have hundreds of customers. We'll serve hundreds of millions of consumers this year.

7. 1:20

   Um, uh, we work with folks like ADT, the largest home security company in the country, built an AI for them. If any of you have a SiriusXM subscription and contact them, you'll speak with Harmony, an AI agent that we built with and for them who picks up the phone and can help you, uh, get back up and

8. 1:35

   running, including sending a satellite signal from space, um, to, to get your radio u-up and running again. We work with, uh, one of the largest mortgage originators in the country, and then a lot of, uh, local tech companies that you would have heard about.

9. 1:48

   And so, um, that's, that's what we do in, in a nutshell. And, um, uh, what excites us is we think that every company in the future is gonna have an agent, its own, uh, branded customer-facing agent, and we think it's what comes after the website, what comes after the mobile app, and we want to help the, the

10. 2:07

    great companies of the world build their own and do it well.

11. 2:10

    So talking about AI architect, I would say the killer use cases of AI are coding and customer support today. Um, the coding, I would guess vibe coding and some of these ideas-

12. 2:20

    Yeah, yeah

13. 2:20

    ... are kind of take-taking hold.

14. 2:21

    Yeah.

15. 2:21

    On the customer support side, you know, Bret mentioned this idea of the AI architect where instead of managing software, you're almost like the personality coach and, like, really thinking through-

16. 2:31

    Yeah

17. 2:31

    ... what's the vibe that the agent should have, how does it integrate. So w-what is an AI architect?

18. 2:37

    So a-a-around the emergence of the internet and the web, there was this role of webmaster, right? And you don't really hear webmaster thrown around a lot these days, but it was someone who was creating a company's, in essence, digital storefront, right, if, if they're a business and thinking about not only the technology, right, uh, uh, are, are

19. 2:56

    you building an ASP or you have static pages or, or whatever in the dark ages of the web, uh, but also it was like, what does it look like, what does it feel like, and, and so on.

20. 3:06

    So I think the AI architect, I would say, is kind of the AI era and AI agent version of the webmaster in a way. And it's, um, it's actually, Bret shared on the podcast, it's a role that, uh, we've seen emerge organically across a set of our customers.

21. 3:23

    And we heard it first from one of our largest customers, where the, the team of folks who are responsible for managing, coaching, improving, building their agent came to calling themselves the AI architect.

22. 3:36

    And I think there are three parts to it. One is you've got to understand the, the technology, have a little bit of a feel for what agents can do and so on.

23. 3:43

    Doesn't mean that you've, you know, pre-trained a, a trillion-parameter model or, uh, e-even been hands-on with a, a LangChain or, um, even vibe coding, but having a feel for the capabilities, number one.

24. 3:56

    Number two is, and a pattern we've seen, we-- there's some interesting things, is a company's agent needs to not only manifest the functional capabilities of the company but also be something of a brand ambassador, right?

25. 4:08

    What's the voice? What are the values? What's the tone? How do you come across? How do you create a connection with the customer? So there's a real, I, I would say, aesthetic and taste element to it.

26. 4:17

    How should it sound? Uh, it-- Does it have a persona, right? Some of our customers' agents are the X company virtual agent. Uh, in contrast, uh, we work with a company called Chubbies.

27. 4:29

    Uh, they make great very short shorts, uh, that I am not cool enough to wear. Um, their agent is named Duncan Smothers and will tell kind of irreverent bro-y jokes and so on and speak in a, a really funny tone.

28. 4:41

    So m-making decisions about that a-and, and many other ways that the agent comes across is the second part of it. And then third, uh, ultimately, a business wants to, in engaging with its customers, drive business outcomes.

29. 4:54

    So what business outcomes are you driving towards? So it's this three hats. It's technology, it's experience and aesthetics and design, and it's business. And I, I think it will be one of the fastest-growing job types in the next five years.

30. 5:06

    And from what I've seen, you know, I have a front row seat on this, one of the most interesting as well.

31. 5:12

    Where most of them already in the customer support org, or are some of these people coming from more technical teams and kind of like creating this blend of a role?

32. 5:22

    All of the above, but the, the area I've been most excited about is seeing individuals in customer experience teams. And, you know, engineering teams are often celebrated and held up and so on, your, your CX team less often so.

33. 5:36

    Uh, but what's emerged are people who really do have a feel for what a great customer experience looks like and are hands-on with the technology enough and have a sense for what the business is trying to do.

34. 5:48

    So, uh, the answer is folks emerge from all of those teams. I would say the most common and the one I'm most excited about are the folks who have been close to the customer in service, support, care, retailing settings, and so on, who, uh, kind of put this badge on and, and become the, the AI architect, uh,

35. 6:06

    or one of the AI architects for their, for their business.

36. 6:10

    I'm sure there's a lot of people in the audience that have been tasked to figure out the AI strategy of their company-

37. 6:15

    Yes

38. 6:15

    ... whatever-

39. 6:15

    Yeah

40. 6:16

    ... whatever that means.

41. 6:16

    The board says we need an AI strategy.

42. 6:17

    Uh, the board said-- They're gonna be really angry.

43. 6:19

    Yes.

44. 6:19

    Um, so when you think about all the AI architects that you work with, what are like some of the traits of the most successful one? Are they really curious?

45. 6:27

    Do they try a lot of products? Are they very structured in how they evaluate? Are they maybe more vibe-based on how they think about what tools to use?

46. 6:34

    I mean, there's a lot in what, what has made customers we've worked, companies we've worked with successful in developing an AI strategy and actually a-applying it. I, I think, um, broadly across the businesses we work with, the most successful have not let perfect be the enemy of the good.

47. 6:51

    You think about large language models and agents, these are probabilistic pieces of software that could say or do anything. And, uh, uh, so necessary in adopting them is some amount of risk tolerance and being willing to, uh, you know, step into the pond and, and try things out.

48. 7:09

    So a spirit of, uh, exploration, trying new things, and taking some risk, uh, is, is number one. Number two is a, a deep, uh, focus on actually solving customer problems and real business problems.

49. 7:25

    I think, uh, too often there's this, "Hey, let's apply some AI to that, and we'll have, you know, emerge from that our AI strategy." No, no, no, no, no, like start with a concrete, valuable problem to solve, and it can be very narrow.

50. 7:38

    Uh, w- you know, we-- To give you a sense for like where we and one of our s- customers started was something as simple as processing a single return.

51. 7:47

    And like we celebrated and they celebrated when their AI, you know, picked up the phone and successfully drop-shipped someone a new pair of shoes and printed-- and gave them a shipping label, label to print.

52. 8:00

    Right? It is not, you know, the pinnacle of complexity and so on, but you start somewhere, um, and, and learn and grow from there. And then the last thing I would say is,

53. 8:09

    uh, uh, not shoehorning the way you've built teams in the past or done things in the past into the AI era. And so our most successful, uh, customers and partners have actually re-architected their customer experience and customer service teams around supporting the AI agent in being and doing better.

54. 8:30

    So there's a set of people, for instance, at one of our customers who will review a couple hundred conversations a day and basically coach and refine the agent on how to do it better, how to say it better, how to make better decisions, how to have greater empathy, how to have better judgment, and that was not a

55. 8:45

    team that has existed, you know, anywhere in the past. And so really thinking from first principles and, and not just trying to translate naively the old to the new would be a third element of it.

56. 8:56

    You know, architect, that's kind of like a NRN technical connotation in enterprise. It's like, hey, you're like the software architect. What were some of the build versus buy fallacies that you've seen working with customers where you maybe had the customer support team that just wants something today, and the engineering team is like, "Oh, we can just build

57. 9:12

    this. It's just gonna take three times as long and cost twice as much"? That's my example. [laughs]

58. 9:16

    The, the multiples are more than that, but yes. Yeah. So, uh, it's such an important question, and, uh, we get the, "Oh, we're gonna build our own," uh, you know, "Why should we work with you?"

59. 9:26

    all the time. And it's funny, we, we have a slide, [laughs] we call it the agent iceberg, where I think technical teams think, "Oh, awesome, we're gonna choose our language model.

60. 9:34

    You know, should we LangGraph or LangChain, take it off the shelf? Uh, you know, what, uh, what embeddings model will we use? Uh, what vector database? And, you know, maybe we'll integrate some tools, you know.

61. 9:46

    We're, we're done." And then you put on your scuba tank, and you kinda... Y- you go under the surface of the ocean, you're like, "Oh, my God," you know.

62. 9:52

    There, there are hundreds of things from how do you do regression testing and unit testing? How do you do model migration and model upgrades? Uh, in voice, it gets just shockingly complicated, where how do you separate primary from secondary speaker, handle interruptions, and a thousand other things in the agent development lifecycle.

63. 10:10

    And so, um, uh, we, we come to our customers with, uh, what we call Agent OS, our platform for building agents, and it's, it's a very sophisticated toolkit for in-code building very sophisticated customer-facing agents.

64. 10:27

    Now, the, the architect, right, there's this whole other side to building excellent customer-facing agents, which is the experiential, the, the brand, the mark- So paired with that, we have a set of no-code tools that enable non-technical users to build, refine, coach, edit, update their agent.

65. 10:45

    And importantly, these two seamlessly interoperate. And, a- and so I think when folks approach us and say, "Hey, we're just gonna roll our own," they look at, oh my goodness, all of the things under the surface of the iceberg, the problems that we have spent, you know, the better part of two years, uh, running into and then

66. 11:02

    solving and pulling together in a very coherent platform to build these scaled customer-facing agents that can pick up the chat, pick up the phone, um, and handle a high degree of complexity, and the set of tools for non-technical users to contribute, uh, to, to the agent as well.

67. 11:18

    And that rings quite true. And where we have had, um- Companies we've interacted with go down the path of build your own. We've had many of them come back nine months later and it's like, "Hey, uh, it was deeper and darker than we expected [laughs] under there, you know.

68. 11:32

    Can we talk?" So, uh, that's kind of the journey and the, the pattern that we see.

69. 11:36

    Yeah. What's the agent-building iteration process, like when people are building on Sierra or when you're seeing people build agents, like how should people think about how to push the envelope?

70. 11:46

    And you can also do things like you couldn't A/B test a customer support person before.

71. 11:50

    Yeah.

72. 11:50

    Now with Sierra you can kind of have different personalities. Like do you see people be very creative with that?

73. 11:55

    Yeah. It's a couple levels. One, uh, we've had to essentially event... invent a new software development lifecycle. We call it the agent development lifecycle, where you have this non-deterministic piece of software, so how do you test it?

74. 12:08

    Well, one of the things we've discovered is the solution to most problems with AI is more AI. And so [laughs] when you're testing a company's agent, how do you do that?

75. 12:18

    W- we-- You can't just put in a single input and hope you get the, the right output. We've built a whole user simulation testing harness where we can create dozens of different personas with simulated accounts, uh, even simulated devices that they're troubleshooting, and, you know, the amber light is on or off and, and so on.

76. 12:37

    And so first and foremost have had to think through all of the parts of the software development lifecycle. Um, with that as the foundation, you then have this approach to building out every, every business' agent, which starts with deeply understanding what they're trying to do with their customers, what are the key customer journeys.

77. 12:55

    And then we have a variety of techniques for modeling those in code, uh, i- in a way that is very expressive, uh, a- and lets the agent simultaneously kind of hit the curveballs and flex.

78. 13:08

    If someone comes in on one topic and goes to another, do that. But then when it matters, right, be, you know, down to fully deterministic where needed. Like there's no hallucinating, you know, compliance regula- uh, compliance language that, that you want.

79. 13:21

    From there, uh, we, we then use, uh, the, uh, simulations testing harness to, in essence, have tens or hundreds of thousands of conversations-

80. 13:29

    Mm

81. 13:29

    ... with the agent before it's live for the first time. And from there we can tell, oh, you know, it doesn't know enough about this part, or it needs to be able to handle this corner case better and so on.

82. 13:39

    And then, um, it, it really gets interesting when we go live and we have a, a set of tools that give CX teams and engineering teams deep insight into where does the agent realize like, oh, like I'm, I'm beyond my abilities on this, I'm gonna hand off to, to a person.

83. 13:56

    Uh, and then we have this closed loop, uh, set of tools where the agent can learn from its past mistakes, it can be coached, it can be improved, and you end up with this kind of upward spiral of performance and capability.

84. 14:09

    Yeah. Yeah, you mentioned beyond my ability. What's your process for like staying up to date on the ability of the models?

85. 14:14

    Yeah.

86. 14:14

    I think there's a lot of people that try a model or try an app and it's like it doesn't work, but then it works a month later because the models improve so quickly.

87. 14:22

    Yeah.

88. 14:22

    Uh, what, what's your process for staying up to date on it?

89. 14:24

    Yeah. Well, first of all, i- if you feel like things are changing faster than they ever have before, it's because they are. Um, I, I, I feel like whether it's, I don't know, Dance Dance Revolution or Beat Saber or just like new models, new agent frameworks, uh, uh, new benchmarks and so on are just coming down the

90. 14:43

    pike at, at an incredible and increasingly fast rate.

91. 14:48

    Mm.

92. 14:48

    So I, I, I think one thing is I think fairly typical, uh, I think we all do, dipping into Twitter, X, uh, reading the l- latest research papers, uh, and, and really trying to, uh, just immerse oneself in things that are even adjacent to what specifically you're doing.

93. 15:04

    So we don't yet use video models, but gosh, what the, you know, VO3 models are ca- capable of and, uh, what's emerging. And it's like, okay, it gives, it gives you a hint of what's gonna be around the corner, maybe in the area that, that you're directly, uh, working in.

94. 15:19

    And then I just think there is no substitute for hands-on and using it. And so, uh, really being hands-on with the tools, whether or not, right, again, you're directly applying them in, in your work or what it is you're building, I think is so important.

95. 15:35

    And I, I would argue that, uh, understanding where things are going is even more important than understanding where things are today. So the first derivative is more important than kind of the absolute, uh, state of capability.

96. 15:50

    Uh, an example o- of a decision we made early on was like we had a strong sense when we started the company that the, uh, cost per token was going to plummet, that model capabilities were gonna expand.

97. 16:02

    And, and so you want to be building to where the puck is going as opposed to where it currently is. And, and so almost having a ritual where on a, on a cadence you're checking in with the capabilities of the model.

98. 16:16

    I keep in a Google Doc some problems that were too hard, right, for GPT-4 to solve, but you know, o1 or o3 can, can o3-mini do it? Um, and, uh, y- you're, you're checking in on the capabilities of these models to basically plot, a- again, the slope, uh, so that you can understand, cool, if we start building

99. 16:37

    this now, this will intercept, you know, at this period of time, and we could have this level of model capability, but with this latency. And I think that's, that's how you build truly great products that are at the frontier.

100. 16:48

     It's by anticipating the frontier.

101. 16:50

     Yeah. Um, I know we only got a minute and a half left, but you spent 18 years at Google. You kind of started the AR/VR projects.

102. 16:57

     Yeah.

103. 16:57

     You started the Lens project.

104. 16:58

     Yeah.

105. 16:58

     What, what do you think about the next interface for AI? So we had text, now we have voice.

106. 17:03

     Yeah.

107. 17:03

     Obviously video's coming. Do you see the glasses are gonna-- Do you think the glasses are gonna work? Is it more ambient agents? Like any thoughts?

108. 17:10

     Yeah. I, first of all, I think how we interact with AI and agents is gonna look super different from today. Today, it's like mushed into what looks like AOL Instant Messenger, right?

109. 17:20

     A chat, a chat interface, uh, or it's a voice call. I think agents are gonna look like shape-shifters that can summon text, voice, video, imagery, user interface, a- and more, and, and you're gonna interact with, uh, every, uh, sense and mechanism that, that you have.

110. 17:39

     As for the hardware, uh, look, I spent 10 years of my life, uh, building, uh, in AR and VR. My strong view is that, uh, glasses and wearables will be the ultimate vehicle for the trusted personal AI that is with you, something that can see what you see, hear what you hear, uh, that can whisper in your

111. 17:58

     ear or, you know, nudge you that way, uh, visually. I, I just think we're on this path to every one of us having an omnipresent, uh, omni-capable, uh, AI assistant that can help us navigate the world, uh, lead better and healthier lives, uh, be smarter than we are on our own.

112. 18:18

     And I think, you know, going into your pocket or purse or bag to retrieve the, you know, rectangle of glass and metal and, you know, swipe up and, uh, whatever it is, I, I just-- I think for such an important capability that will feel in time like an extension of ourselves, you, you want that to be with

113. 18:37

     you throughout the day. And so I think wearables, I think glasses will be a, a central part of that. And, uh, it's something I'm, I'm super excited to see emerge.

114. 18:45

     Awesome. Thank you, Clay, for joining us.

115. 18:47

     Alessio, thank you so much. Thank you. Thanks, everyone. [upbeat music]