How to build Enterprise-aware agents

Talk transcript

1. 0:00

   [upbeat music] Uh, thanks Alex for the introduction.

2. 0:16

   That was a very impressive LLM-generated summary of me. Uh, I've never heard it [chuckles] before, but, uh, nice. Um, so, um, today I'm gonna talk to you about something that has been keeping me up at night, uh, probably some of you too.

3. 0:32

   So how to build enterprise-aware agents. How to bring the brilliance of AI into the messy, complex realities of, uh, how your business op-operated. So let's jump straight to the hottest question of the month for AI builders: Uh, should I build workflows or should I build agents?

4. 0:52

   So what are workflows? Workflows are a system where LLMs and tools are orchestrated through predefined code paths. So there are two main ways where you can, um, represent the workflows.

5. 1:06

   The first way is through, uh, imperative code base. So these are the workflows where you, you know, write a program that calls LLMs, uh, read the response, and then call tools and sort of like, uh, do this in a traditional programming flow.

6. 1:22

   And then here you get-- have like direct control of the execution of, um, all the steps. The second way to represent workflow is through, uh, declarative graphs. So in this way, you sort of, um, represent your workflow as like a graph of-- where nodes are sort of like steps, where you can call tools or call LLMs, and

7. 1:44

   then there's sort of edge between nodes. Um, so you kind of define the structure but not the execution, and the execution of this is usually handled by some framework, um, workflow frameworks.

8. 1:56

   So I'm, I'm not gonna go into the details of pros and cons for these two approaches, but, um, the main point here is, like, for workflows, you get structure and predictability.

9. 2:07

   So if you run a workflow today, it will mostly behave the same way, uh, if you run it tomorrow.

10. 2:15

    On the other hand, um, we have agents, which are systems where LLM sort of dynamically direct their own processes of, like, decide how to achieve a task. Like decide what tools to call, uh, what step to take, depends on the task itself.

11. 2:33

    Um, so the core agent loop is pretty simple. So it receive a task or, like, a goal from a human, and then it, uh, sort of enter this iterative loop, where it, uh, plan what to do and then execute the action, and then read the results from the environment, and sort of iterate until, uh, it, uh, get

12. 2:53

    all the result it wants and then, uh, respond to the user.

13. 2:58

    So what are the trade-offs between workflows and agents? Um, workflows are sort of like the Toyota of AI systems. Uh, it's very predictable. Um, it's good for when you wanna automate, uh, repetitive tasks, uh, or, like, encode existing best practice or, or, like, know-how in your bu- in your business.

14. 3:20

    It's usually lower cost and lower latency 'cause you don't have to spend time on this, all these LLM calls to decide what to do. And they're also, also easier to debug 'cause, like, you have this code or this graph that you can manually pinpoint, uh, at which step is going wrong in, in the execution.

15. 3:38

    And in building workflows, uh, humans are sort of in control. Like, you can control your destiny, like, given, even given, uh, imperfect LLMs. Uh, you can sort of do tweaks and engineering so that your tasks work right now.

16. 3:55

    On the other hand, agents are sort of like the Tesla of AI systems. Like, it's more, uh, you know, open-ended. It's good for, like, researching unsolved problems. Uh, it's also usually good at taking advantage of, um, better and better LLM capabilities because here the AI is in control.

17. 4:13

    Um, generally, it's higher cost and latency because you need LLM to, like, figure out what to do and then... But the, uh, upside is, like, there's less logic to maintain, the call loop is very simple, and, um, sometimes you get, like, these, uh, hints of brilliance that always feels like, you know, everything is gonna be automated in

18. 4:32

    a few months. Um, the problem is, like, your, your Tesla, like, it's-- works very well most of the time, but sometimes it still take the wrong exit on the highway, and that's when you kinda miss your Toyota.

19. 4:45

    So, and the decision to build workflows or agent is a pretty tricky one because it depends highly on the state of the LLM. Um, so some workflows that doesn't work in an agentic loop now might start to work later in a few months when a new model has come out.

20. 5:07

    So it's, uh, it's a really huge dilemma. Um, but recently, one thought, um, that's sort of really changed how I think about it is, what if you don't really have to choose, right?

21. 5:21

    So if you think of agent, what they do is when you give the agent a task, it will figure out the steps that needs to be done to achieve that task, right?

22. 5:35

    So, um, you give it a task, it figure out the, the one step, take the action, figure out the next step, and then at the end when the agent finish the execution and then you look at the trace of what happened, all those se- series of steps is a workflow.

23. 5:51

    So if I represent this in, like, a, um,

24. 5:55

    a programming kinda way, then agent takes a sta- a task and then generate a workflow to achieve that task. Um-

25. 6:04

    So if we think of th-it this way, agent take a task and generate a workflow, then you can sort of see, like, there are really good synergies between workflows and agent.

26. 6:17

    So the first thing is you can actually use workflows as, uh, evaluation for your agents, right? So, uh, let's say in, in your company, you can collect a huge amount of, um, golden workflows, like given a task, this is the steps that, uh, needs to be done to solve that task, and you, you have a huge list

27. 6:38

    of, uh, of those, uh, sort of handbook on, on how, um, to do things in your company. Then you can actually evaluate your agents by, uh, you know, like give it a task, see what it did, and compare it to the, the golden workflow.

28. 6:53

    Like, did it actually figure out the right steps? So this is a little bit different from evaluating end-to-end. You are not judging agent by, uh, the end response, but, like, by, uh, whether it actually did the right step to get to that end response.

29. 7:10

    Um, the second and, uh, even better way, uh, for workflows to help, help agents is, you know, given that same golden, uh, workflows library, you can also use it to train your agents.

30. 7:23

    Um, so here you truly get the best of both worlds, where, you know, with the data feeding, you can, uh, your agents will be able to execute the exact workflow that you have in your library for the known task.

31. 7:39

    Um, but then it co-or co- um, it can also rely on its own, uh, internal reasoning capabilities to sort of compose different workflows together to, uh, achieve new tasks, and even use its own reasoning to kind of extend, uh, what you teach it, but, like, make it better.

32. 8:00

    Um, and then agents can also help workflows as well. Uh, one way to do that is, um, for workflow building platforms, uh, you can use an agent to generate the workflows.

33. 8:14

    Um, so this is sort of how, uh, Glean agents work under the hood, where, uh, the user can give, uh, the workflow builder like a sort of natural language, uh, a description of the task it, it is trying to achieve, and then we run an agent implementation to figure out the steps that are needed to achieve that

34. 8:31

    workflow. Then the user can sort of like, uh, make edit or like add change, uh, the workflow that, that the g- the agent was, uh, proposing.

35. 8:44

    Um, and lastly, and, and I think is like, uh, the most powerful, um, synergy is you can use agents as a workflow discovery engine, right? So you ship an agent, uh, users try to accomplish new tasks with your agent, and then when they find that the agent did a good job, then you can sort of save that

36. 9:09

    workflow as like, "Okay, this is how you do this task in my company." And then over time, you can use this, um, as like training data to help agents get better.

37. 9:20

    Cool. Um, so that was the main points of my talk. Um, I guess maybe some of you are thinking, do we st-still need this kind of stuff in a world where we have AGI?

38. 9:34

    Um, so here's, here's my thought experiment and, uh, why I think this may be still needed after AGI. So AGI is gonna be a super intelligent employee, right? Um, but if they, if AGI doesn't know about, uh, how your company works, it's sort of like, uh, a really good employee who just joined and doesn't know about all

39. 9:57

    the business practices and still needs onboarding, needs to know, like, who to talk to, to get unblocked and, like, uh, all the very nuanced ways of doing things in the enterprise.

40. 10:09

    Um, so what is enterprise-aware AGI? So enterprise a-aware AGI is fully on-onboarded, very intelligent, knows the ways your company do things. And, um,

41. 10:23

    one, one key kind of insight I, um, I think is like, so there are many acceptable ways to achieve a task, um, but there's a gap between an acceptable output versus a great output.

42. 10:37

    Um, one example is like, you know, competitor analysis. Like, sure, it can do some basic Google search and, like, uh, read some, uh, notes out-outside to, like, do some competitor analysis, but does it actually follow, uh, the protocols or the processes that your company define?

43. 10:57

    And does it actually address all the key metrics that your executive, uh, really care about?

44. 11:03

    So, um, given all this data, you know, like tasks and golden workflows, how do you actually train your agents, um, using those data? So this is, uh, the second part of my talk.

45. 11:18

    Um, so there are two main ways we have, um, experimented with. The first one is through fine-tuning. Um, there are sort of two main flavor of fine-tuning here. One is, uh, you know, supervised fine-tuning, where you give, uh, give an input and an expected output, and you train your model to just, um, mimic that, uh, behavior.

46. 11:43

    The second way is through RL-RLHF, where you don't have a golden label, but you sort of have a, a rating or a reward when you know, like, this task, this workflow, is it a good one or is it a, a bad one?

47. 11:55

    So then you can sort of run f-your, uh, favorite optimization algorithms to fine-tune the LLM.

48. 12:02

    So the pros of this method is that it can learn really well when you have a lot of data, um- Um, if you have a huge amount of, uh, tasks and workflows, it can really learn, um, like sort of generalize across different tasks and like combine workflows.

49. 12:20

    Um, the problem here is, one, uh, you kinda have to create a fork from the, from the frontier LLM, right? So you start with some LLM, you do some fine-tuning, and then by the time the fine-tuning finishes, maybe there's a be- new and better model already come out, and you have to like redo this whole process again.

50. 12:40

    And the second is like any change to your training data, uh, like you need to do retraining, right? So if you have a new tool, then maybe some of the existing workflow is outdated, then you have to retrain.

51. 12:52

    Uh, if you do s- change some business priorities or business processes, then you have to like redo the training a- again. And it also not super flexible for personalization.

52. 13:04

    Um, so given the same task, maybe different teams or different employees might actually have a different optimal workflows to, to do those tasks, and fine-tuning is not super well-suited for, for those use cases.

53. 13:18

    Um, then comes the second option, uh, which is dynamic prompting through search. So, um, given the same label data, uh, from task to a golden workflow, you build a really good search engine for tasks, um, so that you can find similar tasks given a new task.

54. 13:36

    So then at runtime, uh, to accomplish a new task, we'll find the most similar task in the training data, and then you feed the representation of those workflows to the LLM as the examples, right?

55. 13:50

    So here you really have a spectrum of, uh, determinism and creativity. So when there's no workflow that sort of match your input task, then the LLM are in control.

56. 14:03

    Like it can, uh, use its creativity to generate a new workflow. But when there's a high confidence match of something that, that you have done before, then the LLM will sort of give you a work- workflow that's very similar to what was in the training data.

57. 14:20

    Um, so one very concrete example, uh, come back to the competitor analysis be-- uh, example before. So you collected this huge list of task workflow, um, and then when a new task, like say what, what competitors have we been running into recently, then it will retrieve, you know, how to analyze each competitor, and then you will find

58. 14:45

    a workflow on how to find, uh, your recent customer calls. And then the LLM will take those example and then sort of generate a composed workflow where it read customer calls, read, uh, internal messages, extract competitors, and then run analysis for each of them.

59. 15:04

    Um, okay, so comparison time. Um, fine-tuning RLHF is very strong, uh, when you have a lot of data that you wanna generalize. Dynamic prompting with search is more flexible, uh, also gives you better interpretively, uh, that you can sort of look into the exact e- examples that was affecting your outputs.

60. 15:27

    And, um, fine-tuning is good for learning generalized behaviors, uh, where the ground truth labels don't change over time or like across different users. Um, dynamic prompting with search is better for learning customized behaviors or like the last mile quality gap where, you know, uh, requirements are changing quickly.

61. 15:47

    Um, one, one sort of analogy I think about fine-tuning versus dynamic prompting is, um, fine-tuning is very similar to like building customized hardware. So when you know-- When you have a sort of task that you really want to optimize for and the requirements don't change over time, like you can really build custom hardware that do it very

62. 16:09

    well. Uh, but it's sort of costly when you, uh, change the requirements, compared to dynamic prompting is more like writing software. Um, not as, uh, optimized, but like you can just change them very quickly.

63. 16:24

    Um, last point, uh, so how do we actually build this workflow search, right? So how do you give it a task, like find similar tasks? Uh, I would say it's very similar to building document search, right?

64. 16:38

    Um, and there are two main components to this. The first one, uh, is what everyone usually think of when they think of search, which is a textual similarity, right?

65. 16:49

    Um, given this task, what are some of the similar sounding tasks that are in the training data? Um, and here the sort of, uh,

66. 17:00

    golden recipe is like le- like hybrid search between lexical vector embeddings, uh, re-ranking, late interaction, all that.

67. 17:07

    But, uh, what I found is in, in the enterprise settings, uh, pure textual similarity is not enough. When, uh, when you give users the choice to create workflows and write documents,

68. 17:21

    when you s- wanna search for something, there will be like hundreds or thousands of similar looking documents or workflows, and, uh, the problem becomes how do you choose the right one, uh, right?

69. 17:32

    So, uh, which is what I call as, uh, authoritativeness here. And to solve this problem, uh, then you kinda have to go into a knowledge graph, right? So if this workflow is created by someone who I work closely with, uh, it has high success rate, and like people post about it, um, on Slack, then it's more likely

70. 17:54

    to be the right one. So all the tricks in, uh, the recommended system,

71. 17:59

    uh, um, world also applies here for, for workflow search. And, um, this kind of authoritativeness signals are very hard to encode directly into an LLM, which is why we sort of have to have the, like a separate system that does the, the search for workflows.

72. 18:18

    Cool. Um, so key takeaways, uh, workflows, good for determinism, human are in control. Agents, more open-ended, AI is in control. And, um, the synergy between a- agents and workflows is workflows can be used for agents evaluation, uh, workflows used for agents training, and agents is used for workflows discovery.

73. 18:43

    Um, fine-tuning is good for generalized behaviors. Dynamic prompting with search is good for personalized behaviors. All right, uh, I still have one minute and, uh, thirty seconds, uh, maybe time for one question.

74. 18:58

    Exactly one question.

75. 19:01

    Yeah.

76. 19:02

    Um, I'm kind of curious on the fine-tuning, and I think earlier you mentioned that we may need to offer data, right? How about with the new, like RLVR? Can we set up a reward model, um, so we don't need to provide as much data for the fine-tuning process?

77. 19:16

    So the question was, uh, wait, I'll try to re- re-interpret it. Let me know if it's wrong. Uh, how much data do we need to do fine-tuning given the new-

78. 19:27

    RLVR

79. 19:29

    ... RLVR? That's a very difficult question to answer because, uh, it really depends on

80. 19:36

    how out- out of distribution your task is compared to the internal, uh, knowledge of the LLM. Um, but I'll catch you after and we can talk more.

81. 19:45

    Yes.

82. 19:45

    Thank you.

83. 19:45

    Too difficult of a question- [outro music]