Conquering Agent Chaos

Talk transcript

1. 0:00

   [upbeat music] My name's Rick Blalock.

2. 0:16

   I'm from Agentuity, and, uh, that you're all wearing the, the logo around your necks. So stop by our booth, please. But, uh, today I wanna talk about, um, conquering agent chaos.

3. 0:25

   But really, I'm talking about deployment, running and deploying agents. Um, and just to give you an example, I was, uh, I was at University of Florida. I'm from South Florida, and I was at University of Florida about a month and a half ago, and I was talking to a bunch of professors and their students on projects that

4. 0:41

   they were deploying. Almost all of them were working on some type of agents in their class projects. And, uh, I asked all of them, "What was the number one problem?"

5. 0:49

   They all said, including the professors, "Deploying agents." And, um, they... Sim- similar problems that I, I had before we, we started Agentuity, which was like you, you build this thing, you deploy it on serverless.

6. 1:00

   In their case, they were using AWS Lambda, and then they, you know, time outs, right? [laughs] "Uh-oh, my agent runs for fifteen minutes," or, "My agent runs for thirty minutes."

7. 1:09

   We have an internal agent at Agentuity that runs for forty minutes a day, something like that. So they're running into that, and then, uh, they weren't allowed to use VMs or EC2 for their projects, so, uh, for obvious reasons. [laughs]

8. 1:20

   You don't want a s- bunch of students running EC2s. And, um, a bunch of other things are running into the gateway issues, you know, wiring up agents, talking to agents and things like that.

9. 1:30

   So it's very, very common thing. And the other one is, you know, the whole web is built, like, for a stateless type concept, right? Agents are not stateless necessarily.

10. 1:39

    They're stateful in a lot of cases. Um, and so those are the deployment headaches that I was hearing over and over with the students, and that mapped to a problem I had about a year ago building a qualitative research agent, set of agents.

11. 1:51

    Same thing. I built it on serverless not thinking, and then I realized, "Ah, crap, I gotta rearchitect this whole thing," 'cause it's doing synthesis and it takes a while.

12. 1:59

    Very common he- um, headaches. Has, has anybody experienced that headache before? I'm curious. Yeah? Deploying agents? So that led us to talk about, like, well, what do, what do agents need to be successful?

13. 2:12

    So they need to li- they need to run as long as they need to run, um, and they need to be able to pause and stop and resume. They also have, de- decoupled from your code, they need to have different inputs and outputs and, um, actually multiple, various ones, right?

14. 2:28

    We... I, I think we understand that, hopefully. And then I put this up here: introspection, self-observability and self-reflection. They need those two. Um, and we have that in the product.

15. 2:38

    We have, like, OTel tracing things. Um, hopefully you've all seen that, some of those types of things where you're doing introspection and, and traces across. But I put self-observability and self-reflection here 'cause this is important.

16. 2:50

    So the, the, the observability things that we're used to are human-based, where we see, like, these traces and spans, and, um, the agents need to understand that as well.

17. 3:00

    There's a, there's a, there's a difference there. There's a nuanced difference there. And of course, it needs memory. It needs to get better, evolution, code execution. So that's what agents need to be successful.

18. 3:09

    And so when I was, again, talking to those UF students, personal experience, um, we need those things, and that's what we started to build at Agentuity. And so I just wanna give you a demo.

19. 3:20

    I'm trying to decide, should I do a live demo or should I do the, the slides just in case? I don't know. Uh- Live. Live? Everybody says live? Yeah.

20. 3:27

    Y'all just wanna watch the train wreck. [laughs] All right. We'll, we'll try it live. So I'll... Wow. It's all live. Live. Wow. Yeah, it is. All right. We'll do that.

21. 3:38

    So, um, I'm gonna... Let's do this. Let's do this. Let's blow this baby up. Can you see this okay? Yeah. Yeah. All right. Good. I was told I had to do it in light mode because that's what you do, so.

22. 3:52

    Um, all right, so we have a CLI. So if you're gonna start, you run Agentuity create to create a project. Um, when you run it, drum roll... Okay, in this case, I'm logged into a bunch of organizations, so I'm gonna do our Agentuity, uh, demo one.

23. 4:08

    We have three runtimes that we support: Bun, Python with uv. You notice we, we put uv. We're trying to bias people towards uv. Please don't use pip. Um, [laughs] and then Node.js, of course.

24. 4:18

    So those are our three runtimes. I'm gonna pick Bun.

25. 4:22

    And then, uh, you can pick several templates, um, th- as a starting place. Now we're, we're, uh, framework agnostic. You notice there's Mastra here. Mastra's pretty good if you're going down the TypeScript route.

26. 4:33

    Um, I'm, I'm gonna pick Vercel AI SDK and Groq. I, a lot of our stuff internally, we have 50 agents or so, and they're... We don't use a framework.

27. 4:41

    Um, but we're framework agnostic, so you can bring CrewAI and then deploy another one in LangChain, another one in Pydantic, and then Vercel, a- and then they can all talk to each other.

28. 4:51

    Um, internal networking, talk to each other and that kind of thing. We'll show you that here in a minute.

29. 4:55

    I'm trying to leave enough time so we can have questions and answers. So I'm gonna pick Vercel AI SDK with Groq. Honestly, that's my favorite combo with Bun 'cause of the speed.

30. 5:03

    It's amazing. Um, and then I'm gonna name a project. Let's just call that project Hello. A project is a grouping of agents. It's also, um... You can think of it as like a one-to-one to a repo, 'cause you can hook it up to GitHub.

31. 5:17

    An agent, again, multiple agents in a project. You get your default one here, so let's just call this, uh, agent-1-hello. Uh, very creative name. Now, one of the things we give you...

32. 5:27

    Agents are a first-class citizen. Again, back to deployment problems. Agents are a first-class citizen, infras- infrastructure citizen, um, in Agentuity. So that means we handle all the routing and all that to that agent, and you can decide how you want to protect it.

33. 5:41

    So you can protect it via the whole project. Each agent in your project can have an API key. I'm gonna do none right now 'cause it's a demo.

34. 5:50

    There's more to that, but, um, I'll skip over that right now. Then you can hook it up to GitHub, uh, so then if you merge to main, it'll automatically deploy your project.

35. 5:59

    And that's it. So then we create this. I'll show you the code here in a second. I'm gonna show you dev mode real quick, and then I'll, we'll, we'll do the code.

36. 6:05

    All right. So- Rock and roll. So let's do Project Hello. And before I show you the code,

37. 6:13

    we have now this Vercel AI SDK Bun, you know, Groq template, right? So we have this command that you can run any of your agents with, Agentuity dev. If you have multiple projects, you can run them and then they'll have, they'll have, uh, ports assigned to them.

38. 6:27

    And so now we're running this locally, as you would expect. Very... What you're n- used to, I'm sure. But, um, one of the things I wanna, I wanna show you is...

39. 6:36

    So this is the, is routing to the agent. We only have one agent, so that's the, that's the agent ID. But we have this public routing, so we do tunneling for you.

40. 6:44

    So if you wanted to hit it through another service or something while you're deving, you can do this, uh, publicly too. All right. But let me just show you the dev mode real quick.

41. 6:52

    So, um, click this. This... Oh, of course I got logged out. Ugh. [laughs]

42. 7:04

    Who expired the token? Um, okay. So, um, we give you, like, this kinda simulator for your agent. Um, some agents don't take just text, right? So you can, you can pick JSON, HTML, you can...

43. 7:17

    PDFs. Um, actually, we do emails and things like that too. That's not in this dev mode. But this is where you can interact with your agent, test it out, and then when you do that...

44. 7:26

    Like, I'm just gonna run it a few times. What? I'm sorry?

45. 7:31

    Somebody say something? Oh, okay. Um, when you do that, you can get your logs. Of course, you can see it in terminal too. Um, you can in- inspect the logs, get more information.

46. 7:41

    And then the sessions are back to the... Remember the observability thing? So we have the human version of this here, where you can look and you can see, like, oh, this hit the, this hit our AI gateway, which we have an AI gateway so you don't have to set up keys for any of these models or any

47. 7:53

    of these services. And you can get the cost for that call, um, what the prompt was, what the response was and all that. And it... This works in production too.

48. 8:00

    It's the, it's a mirror image of production. So that's the dev mode. I don't wanna spend too much time on it 'cause we don't have much time, but that's...

49. 8:07

    So put a pin in that. Now I'm gonna kill that, and you're like, "Well, Rick, what does the code look like?" And we're trying not to be too opinionated here, but we do have a few conventions.

50. 8:17

    Um, let's do that. Let's do that. Um, so really the conventions we have in a project are you have a YAML file. Surprise. And, um, that's to help configure your agent when it's deployed.

51. 8:29

    And then, um, there's an agents folder. Python, it, it works the same way. And, um, you have an entry point, and so this is, this is the simple, simple template that we, we created here.

52. 8:42

    And, uh, the entry point in, in this case, in, in JavaScript would be, um, you export a default function, and that's your request handler. If you guys have used Next.js or have used Knatives or whatever, everybody should be familiar with that hopefully.

53. 8:54

    But what we do is we handle all that routing that comes into your agent, and we put it on that request object. And so that, that could be an email.

54. 9:01

    You can add a email to your agent. You can add a phone number to your agent. You can, uh, send a JSON, a PDF, whatever. We have all those things that we route to it.

55. 9:08

    You can stream it. Um, and then also you have, you have things that are, like, first kinda... first class citizens like ctx.get_agent, and then that... And then you can pass in an ID or a name.

56. 9:19

    So we have another agent. Maybe it's a Pydantic agent over here, and we can actually get it. We'll get an ephemeral token so it's allowed to talk to it for as long as it runs, and then you can execute it, send it back.

57. 9:30

    You can stream it back. There's a bunch of different things you can do here. Um, so that's on this context object. So there's... I'm not gonna spend a lot of time on this right now, but there's a, there's a lot of, like, infrastructure we give you in this really simple handler, um, uh, which is pretty cool.

58. 9:45

    All right. And that's it. That's the only convention. So again, like you can imagine if you're using Mastra, and we have a bunch of examples and templates that you can go look at Mastra.

59. 9:52

    You're, you create Mastra template... or, uh, models the way you would, you, you would normally do it, and you can drop them in right here. Same thing in Python.

60. 10:00

    Um, there's a run function in Python, and that's what you have to declare. And then you can drop in a crew or a couple crews, and then have it call Pydantic two and a never- another agent.

61. 10:09

    Um, and that's it. That's, that's, that's the code in a nutshell. Now let's deploy it really quick. YOLO. [laughs]

62. 10:19

    All right. And it should... It- Internet's fast, it should upload pretty quick. There, boom. All right. So we took, we took that agent, we wrapped it in a container, a specialized container that's...

63. 10:29

    In this case, it's a Bun runtime, and now it's live. So if I, if I click on that and jump to it,

64. 10:37

    you can see, um, this is when it was deployed. There's our little happy Bun. And, um, let me show you something really quick that you haven't seen yet. So in a project, again, you have multiple agents.

65. 10:48

    I only have one agent that you saw me create. And then this is where you can get, you can get cost breakdown by project, by agent, by run. You saw the session spans.

66. 10:58

    So we, we try to surface that up in all different ways to help understand how much an agent is costing. But this is where... Remember when I said you can decouple inputs and outputs from your agent if you want to?

67. 11:10

    And this is where I can hit plus here, and I can say, "Hey, I wanna give my agent an email." I hit save, and now I can email my agent.

68. 11:18

    So there's the email address right there. And I can do that with SMS. Um, I can do that with APIs. I can do a cron job. Wakes up, does a bunch of things.

69. 11:28

    Um, and that's inputs and outputs. So it's a nice way to build an agent, and then if somebody wants to use it, they can hit deploy now from your GitHub repo, drop it in, and like, "Okay, let me wire this email up.

70. 11:38

    Let me wire this up. Let me wire..." Later we're gonna get Slack and Discord and all that stuff too. Um, so that's inputs and outputs. Um, actually, let me just go ahead and try to hit this right here.

71. 11:47

    So this is our... Remember, we didn't secure it, so it's public. So actually, if you took a picture and hit this, it probably would work. No, don't do that.

72. 11:54

    Um, let's just... I don't know if this is gonna work. Hello.

73. 12:01

    All right. So I, I hit the webhook. It created this session. There's the session ID. Now it's... The agent's off running, and I should start getting logs here. Got three minutes. [laughs]

74. 12:14

    There it is. Oh, it's an error because I probably... Oh, I- okay, so this is in your YAML file, you can change the, uh, the memory and I d- I need to change the memory footprint for this, so it didn't run.

75. 12:24

    It's like, nope. That's something we're working on too. So in a nutshell, that is, that's what you get. Of course, you can wire up some integrations and, and, uh...

76. 12:34

    Actually, I think I just lost internet. Okay, there we go. There we go. You can wire up some integrations, GitHub, and a bunch of other stuff that's coming. So, um, to k- to kinda wrap up, this is, this was, like, the thing that we wanted to build, not just for everybody, but for us- [laughs] ...

77. 12:49

    'cause we wanted b- we have now 50, 60 agents internally built on this, and it's like finally we're getting really good velocity, and I'm really excited about. So this is our first product in our agent native cloud suite that we plan on working on.

78. 13:02

    Um, this summer we're working on, um, infrastructure agents that watch our logs and, uh, surface things that are coming up for, for, um, for developers and us. And, uh, w- we kinda...

79. 13:12

    We were joking last night, we should call it Agentuty, not PagerDuty. Like, the agent's gonna do that stuff [laughs]. [laughs] Uh, right? Exactly.

80. 13:19

    It's a thing right there.

81. 13:19

    So yeah. So there's a bunch of stuff coming, but that's where we're at right now. Stop by the booth. Um, we got one minute and 50 seconds, so if there's any questions, we got about a minute and 50 seconds.

82. 13:29

    Any, any questions? Jokes, stories, anecdotes? Try to leave just enough room. Yes.

83. 13:37

    One thing that I've been hearing about recently is the idea of adding tool calls into reasoning. Um, like into the, the reasoning tokens. Um, is that something you've been thinking about at all recently?

84. 13:50

    Yeah. Uh-huh. And, and I mean, yeah, for sure [laughs]. Yeah [laughs]. Um,

85. 13:58

    the, the... Now there's a, uh, for us, like I said, from a product perspective, there's a thing we always ask ourselves, which is what does an agent need to be successful?

86. 14:06

    And, and some of that is what... Like, we wanna offer that as a services layer from an infrastructure services perspective, where a lot of that stuff you just get out of the box, and then you can hook it up to a Vercel AI SDK tool thing, or you can hook it up to something else.

87. 14:20

    Like, we, we wanna do some of those kinda service level things.

88. 14:24

    Gotcha.

89. 14:24

    But I mean, like, on a tactical level, yeah, I, I think it's... I mean, I need them actually for a CRM agent that we're working on, so yeah.

90. 14:32

    Cool.

91. 14:32

    All right. Thanks everybody for coming. [outro music]