Agents vs Workflows: Why Not Both?

Talk transcript

1. 0:00

   [upbeat music] Okay, agents or workflows?

2. 0:16

   Why not both? Uh, thank you Alex, for the nice intro. Um, uh, like, like he said, I used to be the, uh, founder of, uh, co-founder of Gatsby. Um, I wrote a book called Principles of AI Agents, which is floating around.

3. 0:28

   Hopefully many of you have gotten a copy. We, we, we have more around the conference. Uh, there was a big debate, uh, a couple of months ago, which determinally on Twitter people may have noticed, um, which I just referenced.

4. 0:43

   Um, and I think, like, this is a big reason why I've ha- why we're having this talk, why we're having this track. Um, this is gonna be kind of like a reverse mullet talk or something like that.

5. 0:53

   It's like party in the front, business in the back or something. So we're gonna start with the party or the debate or, or whatever this is. Um, this was referenced in the last talk as well.

6. 1:02

   Anthropic wrote a great, uh, blog post in December. Um, it was called Building effective agents. It had great diagrams. It showed what an agent was. Can we close the door, please?

7. 1:12

   Um, it, it showed, uh, like some workflow examples. Um, like different sort of types of routing and orchestration. It was a great blog post. Um, in April, OpenAI also released a paper.

8. 1:27

   I think there was some controversy about that on Twitter because some of the points people made was like, "Look, this isn't a lot of new material." Um, other p-people pointed out this kind of call-out at the end, which is basically like an anti-workflow blast.

9. 1:41

   It's like, "Hey," like ... And, and, and I think people were like, "Hey, yo, what's going on here? This is just like not accurate and it's coming from a big model provider, so it's kind of muddying the water."

10. 1:50

    So there was, there was a lot of, uh, controversy around that. That was the, uh, Swix blog post I was referencing. Uh, it was an emergency blog post, um, went out on Latence Space.

11. 2:00

    Uh, I have a couple hot takes on that, and then I have a takeaway. I promise both the hot takes and the takeaway are relevant to the systems you're building.

12. 2:08

    Um, we care a lot about this. It's the reason we wrote a, we wrote a book. Um, first hot take is like, l-look, just, this ... I'm gonna at OpenAI here, but it's like just don't be that guy.

13. 2:19

    Um, and I'll, I'll explain like the context of the like ... I mean, I think we know this meme, but like the, what I mean by that guy in this context.

14. 2:26

    So th-that guy just thinks that like they and they alone like know the only right way to do development. Um, sometimes, and like we ... I actually ran into Laurie Voss in the hallway yesterday, and we started talking about that guy that we sort of had known from the last decade.

15. 2:43

    Um, but sometimes that guy works for this, uh, for like a FAANG-style company, and they're in like a public-facing role, and then the rest of us are just really in for it.

16. 2:52

    Um, because like if you sort of look at the last decade, a lot of web devs got these like lectures by not all Googlers, but like certain Googlers about like the right way to use the platform.

17. 3:03

    Um, and it was just, you know, again, I'm gonna go r-really deep. I don't know how deep into web dev like folks are, but like it was just sort of a, this code, anti-React code word and it was kind of ...

18. 3:13

    Like, uh, they were sort of pushing these technologies that were not very easy to use in-instead. Um, I'm just kind of hoping like the, the, the model providers like kind of have this like elevated position in the ecosystem, so whatever they say carries a lot of weight, um, similar to, to like the, the FAANG companies in, in

19. 3:30

    like, you know, uh, web dev and, and, and, and general. So like let's just ... Here's, here's to hoping for a good quality of a discourse this time around.

20. 3:39

    Um, here's a hot, uh, take number two, and I'm gonna at LangChain here. Um, we should consider a f- like graph node and edge termin- APIs within frameworks harmful.

21. 3:50

    Um, and, and like I say this as someone, uh, who used to be a co-founder of a React metaframework that famously used GraphQL as a default way of fetching data.

22. 4:05

    We wrote da- we, we wrote our data fetching queries like this. Um, it was really cool in 2017. Um, we ... GraphQL is still cool, right? GraphQL is a great technology.

23. 4:15

    But we indexed on, on this pattern and it became kind of the default way of fetching data in Gatsby. Um, some of our users loved this, but many of them didn't.

24. 4:26

    Many of them just wanted a React metaframework. Okay, why is he talking about like the last decade of web development? I'll, you'll see why. They ended up using other frameworks instead.

25. 4:35

    Um, and when I see APIs that look something like this, it gives me flashbacks.

26. 4:43

    I do not think you should need to learn graph theory to write workflows, to build production applications. More problematically, you should also probably not need your, like all of your team to, to, to grok graph theory.

27. 4:59

    A more grokable pattern like looks something like this, and I've used, like I've used Masterworkflows, um, here. Um, but it, it's sort of like a fluent syntax. You can like clearly see the control flow.

28. 5:12

    Um, but I mean, like l- this is like the ingest workflow syntax. Um, you can, you can kind of clearly see the, the flow of, of, of the code.

29. 5:21

    You can see what happens, and then what happens after that, and what happens after that. You can just see it by like when you sort of like step ...

30. 5:28

    Y-when you're reading the code, your, your eyes can go from the top to the bottom and, okay, I, I see what's going on here. Great, I get it. Right?

31. 5:35

    It's readable code. It's, it's, it's like a readable way of doing things. Um, I think when, if we have to use nodes and edges and connect things, we lose that readability of code, which is really important when we're building ...

32. 5:48

    We all build software in teams, right? Generally. Um, uh, uh, so, so i-I mentioned this earlier, right? Like you and your colleagues should be able to use a workflow framework or whatever without learning graph theory.

33. 6:03

    Um- Again, like I said, it's a reverse, reverse mullet, like party in the front, hot ticks in the front, like business in the back. Um, okay, so, so like now that we've kind of like talked about like we- we've sort of like opined on the discourse of the day, um, let's get down to business, okay?

34. 6:20

    Um, [laughs] design patterns for agents and workflows. And when I say design patterns, like this phrase has kind of a storied history. So this is a book which came out I think like late '70s by this guy named Christopher Alexander.

35. 6:33

    Um, it was very famous. It spawned a bunch of like, uh, not... He... So, okay, Christopher Alexander was a professor at Berkeley. Um, he was a architect. He sort of, um, cataloged in both, uh, sort of, uh, like urban planning as well as like internal sort of like in- in building architecture.

36. 6:52

    Like these are a couple hundred of the patterns of what we see are the right ways of building. Um, and, um, just wrote them all up in a book.

37. 7:02

    Um, and so, um, other architects were not very fond of this, but like software engineers loved it, and sort of it became all the rage in like the, and this predates me, but like the l- late '80s, early '90s, sometime around then.

38. 7:13

    Um, uh, and so, so I think there's like... I think what we do not yet have, um, we have sort of like steps towards this, but what we do not yet have is a commonly accepted verbiage and, and like language and glossary of, of what are agentic patterns, right?

39. 7:32

    Um, what are agentic workflow patterns? Um, and so, um, okay, let's just start with like what are agents and workflows. Um, maybe I'm not gonna spend a lot of time on this slide because I think the pre- previous speaker talked about this.

40. 7:45

    I was honestly, because people have covered this ground, these guys did a, a workshop yesterday, and they did a great job, so I was just like, took their slides and put them in there, put them in here.

41. 7:53

    Props to Nick and Zach if they're in the room somewhere. Uh, but, uh, their, um, their, they did a workshop on Mastra yesterday, which was amazing. Um, okay, um, so but like, okay, like, let's just like w- how would we explain it to a friend?

42. 8:07

    The, okay, I, I think about agents like a turn-based game, right? Like I take a turn, then the agent takes a turn, then I take a turn, and then the agent takes a turn, and then the agent takes another turn, maybe makes like a tool call or something, right?

43. 8:20

    It's like back and forth. Um, and then I think about like workflows are like this rules engine for your, uh, for your tech tree, right? Okay, we gotta... I, I played Civ a lot when I was a kid.

44. 8:30

    Um, I, I, I, I... You gotta discover bronze working before you can research iron working, right? You've gotta get me- metallurgy before you can, uh, research gunpowder, right? Like there's, there's some sort of dependency chain here.

45. 8:41

    Um, and it's important to kinda track the dependencies 'cause you can't do step B until you do step A. And a lot of workflows are these like data pipelines, step A, step B, step C, step D, step E, execute them all in order, go, right?

46. 8:54

    Um, you know, um, conversations have threads. You can have memory. Like these are all the emergent properties that happen when you think about, uh, when you think about like lots and lots and lots of messages.

47. 9:07

    Um, similarly, if you think about, uh, these sort of like, um, dependencies, you, you can think about branching and parallelism and conditions and loops and suspending and resuming and replaying and all this fun stuff.

48. 9:19

    Like those are sort of the emergent properties of, of workflows. Uh, and I mean, just kind of recapping, like workflows have been around for a while, obviously. They're becoming more popular now for a variety of reasons, but one of them just...

49. 9:33

    And I wanna bring this back here 'cause it's important, right? Like you can always just write, um, you know, code that says, "Do A and then do B and do C and do D."

50. 9:41

    Um, but the reason why they, like d- they're just more popular in AI engineering than sort of like normal engineering is because like nondeterminism is, is sort of core, [laughs] core to what we're doing here. [laughs]

51. 9:52

    And, and we, and, and being able to kinda trace it and figure out what happened is like, if it's important in, in, in software engineering, it's 10X as important as in, in AI engineering.

52. 10:02

    Um, so, um, let's see. Uh, look, at the end of the day, it's just a trade-off, right? Um, you, you can have power or you can have control. You can decide which parts you want power on, which parts you want control on.

53. 10:13

    You can start with power, and then anything that like goes off the rails, you can add control. Um, at the end of the day, like many things we do, it's just a trade-off.

54. 10:20

    Um, uh, this slide was d- uh, was not able to drop in, but, uh, the photo I wanted to drop in, but, um, uh, we, we, you know, we, we, we've done a lot of whiteboarding sessions with like, "Hey, I'm starting to build an agent, and, uh, I, I wanna think, trying to figure out how to think

55. 10:37

    about this," or, "My agent is, I'm feeding in this giant PDF of medical documentation, and I'm not, I'm trying to diagnose [REDACTED:age] symptoms, and I'm, they're not, it's not accurately pulling out the right information."

56. 10:49

    Um, okay, have you considered breaking that one LLM call into [REDACTED:age] LLM calls, right? A lot of what you do in these kinds of sessions is you sort of think about, you kind of ask, "Hey, what part of your application is performing not very well in, in terms of reliability?"

57. 11:07

    And then like how could you add some structure to the process here so you could, you, you can get additional reliability? Um, a- and I encourage that sort of like practice.

58. 11:18

    We kind of encourage that practice. Like obviously we're happy to do that with whoever, but like also just like do it with each other and like just try explaining your architecture to your friend or your colleague, right?

59. 11:27

    And then like diagram it out on a board because you can mag- When you're doing these things, like magically, like you realize that, uh, actually there's a better way of doing a certain thing and m- maybe a more creative way of using the primitives together.

60. 11:43

    Um, coming to that, right? So here's just some thoughts, right? W- Agents and workflow composition. So agents have tools, and, like, you know, they c- they can call tools.

61. 11:54

    You know, workflows have steps. An agent can be a step. A workflow can be a tool. An agent can be a tool. A workflow can be a step.

62. 12:03

    A- and like most primitives, the magic happens when you combine these things together. Um-

63. 12:10

    The agent supervisor model. You have an agent that is calling other agents as tools, right? So, um, let's see. We have... Th-this one was a research agent and a summary agent, and then, like, an orchestrator agent.

64. 12:21

    Like, these are like, these, these are all, like, Mastra, um, sort of like Mastra code as just more of, like, an example of, like... You know, but, but I think, like, it's illustrative not the particular lines of code and what they are, but, like, these examples are sort of simple enough to fit in the, you know, slightly

65. 12:37

    smaller version of the right panel of my slide, right? And that, and that's sort of the interesting thing. We can use these terms and the implementation is not too long.

66. 12:47

    It's grok-able in a slide. Um, and, and so, again, like, that's kind of what gives us power, is that, like, the primitives are simple, but the combinations are also ...

67. 12:56

    Like, once we get a hang around, uh, once we get the hang of them, we can, you know, run pretty fast. You know, we, you could have workflows as tools.

68. 13:05

    Um, uh, so, um, I think it, it, you know, it's like, hey, like, you wanna plan location, you wanna, like, check the weather, then you wanna plan the trip.

69. 13:13

    Maybe these are, like, more, more complex workflows. Pass that to an agent. Let it sort of, like, iterate and decide. Um, workflows doing agent handoffs. Uh, I'm looking at time here.

70. 13:23

    Dynamic tool injection. This is interesting, too. I think, like, you know, agents, um, can start failing if you give them, let's say, double digit numbers of tools, and you may wanna th- be thoughtful about which tools you're handing in- handing to a particular agent at a particular time when it's performing its particular task.

71. 13:41

    You can also, um, you know, nested workflows. Again, a workflow is a step. But again, like, the real... And, and just I'm gonna reemphasize this. The real alpha comes from sort of, like, using these patterns together in the right sort of way.

72. 13:55

    Um, reality has a surprising amount of detail, and so do agentic workflows that, um, sort of, like, by the time they enter production. Um,

73. 14:05

    I, I think I, I also have a couple of minutes for questions. So, uh, anybody?

74. 14:10

    So do you think then basically it would be even better if you combined the deep research agent plus the workflow versus you do one to replace another? Is that correct?

75. 14:20

    Uh, so the question- Can you repeat the question? Yeah. The, the question is, uh, would, would it be better to combine the deep research agent-

76. 14:25

    So I, I have a workflow that I know for a fact that it works very well right now-

77. 14:30

    Uh, yeah

78. 14:31

    ... with 20 tools for the purpose. But then I also hear arguments that there's no need for me to orchestrate this workflow and just-

79. 14:37

    So the, the, the question is your agent works great with 20 tools. I, I would say, like,

80. 14:43

    we are a community of practice more than we are a community of theory. If your agent is working according to what you would need, like, like, do it. If it's not theoretically correct, that probably means the theory is wrong, not, not the, uh, not the practice.

81. 14:57

    This is a young field and the, the, the, the practice is evolving faster than the theory, right? I think that's just my general, um, comment. Uh, one more question.

82. 15:10

    Where can we find you after the talk? Uh, you can find me around the conference. You can @, uh, I'm, I'm [REDACTED:username]. That's C-A-L-C, like calculator, and S-A-M, like my name, which was my handle when I was [REDACTED:age], 'cause I was, uh, uh, you know, anyway.

83. 15:24

    Uh, thanks everyone for coming. Really appreciate it. Please grab a copy of the book around the conference. [clapping] [outro music]