Why Agent Engineering

Talk transcript

1. 0:00

   [on-hold music] Hi.

2. 0:17

   Good morning, everyone. [cheering] [clapping] Love that. Love that. Um, I'm gonna get right into it. One of the challenges we have with Summit is that we actually ask our spe- speakers to do very short talks.

3. 0:27

   So I, as the leader of Summit, I have to do even shorter talks. [laughing] So let's go. Uh, you can see a lot of these. There'll be a lot of show notes and homework.

4. 0:34

   You can see it on the live stream. How is AI engineering doing? Uh, it's pretty good. We have an O'Reilly book. That's pretty cool. Yeah. [laughing] Um, uh, Chip is actually a good friend, and she's actually speaking at, uh, she's giving our keynote for the workshop session, uh, tomorrow, which is pretty cool.

5. 0:48

   Uh, Gartner hates us. Gartner thinks we've, we've hit the peak. [laughing] So it's only downhill from here, guys. I'm sorry to inform you that AI engineering is over. Uh, there's no, there's nowhere else, else, uh, else to go but down.

6. 0:58

   Um, a lot of, uh, what I try to do with these eight, with these, uh, talks that I do at, at each conference is to try to landmark the se- the, the state of the art or the state of the industry.

7. 1:10

   Um, so with Latent.Space, I, I did the rise of the AI engineer. With the first AI Engineer Summit, we talked about the three types of AI engineer. And with last year's AI Engineer World's Fair, we talked about how the discipline of AI engineering was maturing and spreading across different disciplines.

8. 1:26

   Um, uh, I think this is starting to get a little stale. Uh, by now a few million people have seen this and, like, you know, uh, used this to form their teams, and I think that was the intended effect.

9. 1:33

   What I a- am encountering these days is the two resistance from two sides of the AI engineer spectrum. Uh, if you come from an MLE point of view, you think that the AI engineer is just, like, mostly an MLE plus a few prompts.

10. 1:46

    If you come from the software engineering point of view, you think that it's mostly software engineering and, uh, calling a few LLM APIs. Um, I think over time, it- the AI engineering is gonna basically emerge as its own discipline, and it's still not there yet.

11. 1:59

    It's still very, very early. I still say things like, "Oh yeah, AIE is ninety percent software engineering and ten percent AI." I think that will grow over time, and I think this is the year when it starts to spread out, and that's, that's what I'm here to talk about a little bit today.

12. 2:13

    Um, so for example, uh, I, I think, like, what I try to do with AIE is also, like, it's a, it's a work in anthropology, like how people describe themselves, form groups, form identities, and form industries.

13. 2:24

    Uh, so MLE, you know, it leaks out in your language. Um, they say test-time compute because the only reason to run inference is to test it. Uh, AIE will maybe say inference-time compute because we actually really care about inference.

14. 2:34

    Um, software engineers may be reas- reasoning. Um, a- a- and I think you see these differences, and I'm trying to articulate them over time. Um, part of what I wanna do here to set context is to explain why we've kind of pivoted AI Engineer Summit to be the Agent Engineering Conference.

15. 2:48

    Um, [coughs] it's not a decision that we made lightly because, uh, we're saying no to all these things. We're saying no to RAG. We're saying no to open models, uh, GPUs, and we're just saying, uh, you know, th- this is the only thing that we're gonna do today.

16. 3:03

    Um, and, uh, but, like, closing all those doors actually opens up others. So when we put out the call for speakers, we, uh, made up all this list of, uh, you know, other a- agent engineering disciplines.

17. 3:13

    Uh, and I soon realized we didn't have to. I'll talk about this in a bit. Um, I also looked at last year's top-performing talks on YouTube, and you guys told us, uh, that, you know, you really wanted all the, all the agentic things.

18. 3:25

    Now, the only problem with this is that we only got speakers who basically made agent frameworks for a living, uh, and everyone's asking the, the, the real question, "Who's putting this in production?"

19. 3:35

    So we had a new rule this year of, all right, no more vendor pitches. Um, you know, you, you complain about-- Yeah. Let's-- [clapping] Oh, thank you. [laughs] [clapping] Uh, as a, as a curator, it makes it so mu- infinitely harder because, uh, basically the people that you're about to see have no incentive to come on stage and share what

20. 3:53

    they're sharing, uh, but somehow we talked them into it. So, uh, I hope you're looking forward to that.

21. 3:58

    Uh, the, the other thing also I realized that, like, everything plus agent works basically. So agent plus RAG works. Agent plus code gen works. Agent plus search works. Um, and this is kind of like the simple formula for, like, making money in twenty twenty-five.

22. 4:10

    Uh- [laughing] Most of these, most of these names you'll see in the talks that, uh, that will follow, uh, in, in the sessions.

23. 4:18

    Um, some of you have heard this one before, twenty twenty-five is the year of agents, right? If you say it often enough, it might be true. Uh, I think that when people make predictions, oftentimes they confuse what they want to happen for what will actually happen.

24. 4:31

    Um, so maybe you believe Satya Nadella, maybe you believe Roman, maybe you believe Greg Brockman, maybe you believe Sam Altman. All of them want you to believe that twenty twenty-five is the year of agents.

25. 4:41

    Uh, and I'll be very honest, uh, me and my co-host, Alessio, I think I saw you over there. Hey. [clapping] Um, uh, [laughs] we were pretty skeptical as well. We were on the record being skeptical.

26. 4:51

    Actually, actually, all of you are being o- on the record because last-- yesterday, uh, Barr played, uh, Family Feud with, with our, with the audience, and the number two, uh, buzzword that everyone is tired of hearing is agents.

27. 5:02

    Um, but fortunately, you guys are not tired enough 'cause you came to today. I have you for one more day of, uh, of agents talk. Uh, but we're on record March twenty twenty-four with David Luan, uh, the former VP of Eng of OpenAI, uh, saying that we, we tell people to take agents off of their branding.

28. 5:18

    Uh, now we tell them to put it back on. [laughing]

29. 5:20

    So okay. The, um, I, I, I think, I mean, I'm doing this as a public service. To start any agents conference, we have to define the word agents. Are you guys ready?

30. 5:29

    Yeah. [laughs] All right. I actually have one. I, I-- it's a monumental task. I could do it in one slide. Um, so if you talk-- Again, this, this is a very POV, sort of anthropological point of view.

31. 5:41

    The machine learning people will talk about some kind of reinforcement learning environments. They wanna talk about actions, achieving goals, and all that. Um, AIE, we don't know what they, what they want yet.

32. 5:49

    Uh, the, the software engineers are very reductive. They're just, you know, put it in a for loop.[audience applauding] [laughs]

33. 5:56

    Okay, you-- It seems like you agree. Um, so, uh, fortunately, you know, I think every AIE conference needs to invoke the name of Simon Willison. Uh, he is our, uh, patron saint.

34. 6:05

    Um, he's actually gone and crowdsourced three hundred, uh, definitions of what an agent is. So I didn't have to survey all of you. I, I was thinking about asking every single speaker to start with what is your definition?

35. 6:14

    Uh, it doesn't matter. Um, there's-- here's six of them, right? You-- It's either about goals, it's about tools, it's about control flow, it's about long-running processes, it's about delegated authority, uh, it's about multi-step task completion.

36. 6:26

    Yeah, I see all the phones coming out. Don't worry, it's on the live stream, right? There's, like, twenty thousand people, uh, watching along. Um, and then there's a, there's a bunch of other things.

37. 6:33

    Uh, I think, I think the last one on the bottom left, bottom right is, uh, is an interesting one. Like, just have some things that everyone defines, uh, agrees is an agent, and make sure that they're sort of-- your agent definition is passing those things.

38. 6:46

    Um, except, so that was my one slide. That was my slide, uh, of, like, what, what defining an agent. And then yesterday, OpenAI went and dropped a new agent's definition, uh, on the live stream, uh, that you can w- well, watch yesterday as well.

39. 6:58

    Um, so this is something that they're obviously going to work with. Um, and, uh, I, I think you should, you should definitely pay attention to, to this because they're, they're building on top of this, uh, new definition as well. [lips smack]

40. 7:08

    So that's defining agents. Why now? Why is, why are agents working now when they did not work a year ago, two years ago? Um, I have a rough idea.

41. 7:18

    So the people are ta-ta-talking about capabilities, and so, uh, you can see that capabilities, even, even on a trajectory of twenty twenty-three, twenty twenty-five, um, have been, have been really growing, and they start to ru- around to hit human baselines, uh, right about now. [lips smack]

42. 7:32

    Um, and I also have a map of other cap-- [phone beeping] uh, reasons as well. So I'll just bring you through each of them. Most people will say, "Oh yeah, we have better reasoning now, we have better tool use now, we have better tools," um, including MCP, which, which you're doing a workshop on, uh, tomorrow. [lips smack]

43. 7:47

    Uh, but I think there are some other less appreciated things which I'm gonna bring up to you right now. Model diversity, right? Uh, the OpenAI market share has gone from, like, let's say ninety-five percent two years ago, now down to fifty percent.

44. 7:56

    It's a much more diverse, uh, uh, landscape, including, like, this, this, this past week, um, two frontier model labs that are possible challenges to OpenAI have emerged, and which I think, which I think is, um, really exciting for twenty twenty-five.

45. 8:09

    We, we don't actually know what is gonna shake out to it by the end of the year. Uh, the second thing is, uh, that the cost of intelligence is super Moore's law, is what I call it.

46. 8:17

    Um, it's, it's gone, uh... The cost of GPT-4 level intelligence has gone down one thousand times in the last eighteen months. Um, and you can see the same curve starting for the o1 level intelligence. [lips smack]

47. 8:26

    Um, uh, and also we now start to have RL, RL fine-tuning options. Um, I have zero experience in this area. [laughs]

48. 8:33

    But fortunately, one of our speakers, Will, uh, is gonna tell us, talk to us, uh, later today about this, about this. Um, so we have all these reasons. We have-- Uh, I have a few more.

49. 8:42

    Uh, you know, in our conversation with Brett Taylor, um, he talked about, uh, co- charging for outcomes instead of c-- for, uh, instead of costs. Um, there's a lot of work on multi-agents as well as, uh, faster inference as well that's coming out from the, the better hardware that we have.

50. 8:53

    Um, there's more homework there if you want. Uh, this is all sourced and, uh, you know, co-- has, has, has some backing in our, in our Latent.Space conversations, uh, but I don't really have time for that.

51. 9:02

    Okay, so one last thing for you guys on agent use cases. So, uh, I think most people agree w-with, like, Barri-um, Barry's, uh, building effective agents talk. Um, he's, he's gonna talk about how coding agents and support agents have product-market fit.

52. 9:14

    I think now it's fair to say deep research has PMF. Um, but also I will say up and coming are some of these use cases, uh, some of which you'll, you're gonna see in the con-- the talks later.

53. 9:23

    But I also wanna s-- offer anti-use cases. Can we please stop demoing agents that book flights? Yeah?

54. 9:28

    Yeah.

55. 9:29

    No more flight booking agents. [audience applauding] Uh, [laughs] I wanna book my own flights, thank you very much. I wanna, I wanna book my own Instacart orders, and also please don't astroturf for it.

56. 9:38

    Right. Uh, okay. So, uh, one, uh, yeah, and I think the reason that the tell that, uh, you know... This is, this is a headline that I saw yesterday.

57. 9:45

    I had to put this in. Um, OpenAI reported four hundred million users, uh, which is a thirty-three percent growth from three months ago. Um, and then you can ask deep research to research OpenAI and draw this chart of ChatGPT growth, uh, going from, uh, zero to, uh, four hundred million users in two years-- in two and a

58. 10:01

    half years. Um, so, uh, I, I, I remember this chart very well because Open-- Uh, ChatGPT spent a year not growing. And why did it spend a year not growing?

59. 10:09

    Because they didn't ship any, any agentic models. Um, and if you actually just look at the, uh, the sort of weekly active user chart and stretch it out, you actually get this chart, uh, which is actually super interesting because it basically shows that one, one, um, the sort of o1 models have doubled ChatGPT usage.

60. 10:27

    And if you stretch it out, um, ChatGPT is gonna hit a billion users by the end of this year. This year. Uh, it's basically going to quintuple the number of users it had, uh, as of September of last year.

61. 10:37

    Um, and so, like, the, the, the, the, the growth of, of ChatGPT and the growth of any AI product is gonna be very, very tied to reasoning capabilities and the amount of agents that you can ship for your users.

62. 10:46

    Um, it is, it is real. It is, it is, uh, huge, huge numbers. This is one-eighth of the world population that's gonna be using ChatGPT by the end of this year, and I think there's a lot of money left on the table for everyone else.

63. 10:56

    So, um, I hope you enjoy doing that. Um, I'm well past time, so I'm gonna skip all this. But basically, I, I think that the job of AI is now evolving towards building agents in the same way that MLEs build models, software engineers build software.

64. 11:07

    Um, so, uh, I'm gonna skip all that. You can see all, you can see all that on, on the, on the live stream. Uh, but we're actually, uh, you know, just here to welcome you to the show.

65. 11:15

    Um, and, uh, I'm really excited to introduce you to everyone. So, um, thank you, and I hope you enjoy. [upbeat music] [audience applauding]

66. 11:25

    Thank you.