Agents Building Agents

Talk transcript

1. 0:01

   Hi, everyone. Today, we will talk about agents, building agents, so how we are leveraging coding assistance and spec-driven development to build reliable and secure agents. First of all, just a couple of notes about myself.

2. 0:15

   I'm Alfonso. I'm a tech lead at Nearform. We are a services company. Um, I am currently working on some of our AI agentic projects. I'm supporting multiple teams adopting AI-native engineering, and I'm also a, uh, O'Reilly author of the book Learning AI-Native Software Engineering.

3. 0:36

   So we are seeing in the, in the industry that right now everyone wants AI agents for workflow automation, search, you know, every use case that you might have in mind.

4. 0:48

   Um, but AI agent come with a, sometimes a very high cost. Uh, they have hallucinations. They have, uh, an entire new set of problems. Of course, they are non-deterministic.

5. 1:00

   So today, we will see, um, how we are building and improving iteratively AI agents. And of course, as you may guess,

6. 1:10

   how can we do that, right? Uh, AI is very powerful and very good at building any type of software, and given that AI agents is just one type of software, as you may guess, we are using AI to build AI.

7. 1:26

   Um, as I was mentioning, there are a lot of problems, a lot of different problem classes with building AI agents. Uh, non-determinism is one of those, latency, cost, uh, hallucinations, uh, uh, a, a lot of stuff really.

8. 1:43

   Uh, but today, we will see how to solve them at least partially by leveraging agents and the repeatable process that we developed over time on our projects.

9. 1:54

   Just as a very, very quick refresher, um, we can say that an AI agent is basically just a, an LL- an LLM, which is like the brain of the agent.

10. 2:06

    Then the LLM is connected to a bunch of tools. It has access to a bunch of context and basically lives into an agentic loop. So that's the, that, that's the gist of it.

11. 2:18

    Of course, there is way more into AI agents. There is, um, everything around observability. How do, how do we ensure, uh, that, you know, it's doing what's, uh, what's necessary, so everything around evals that we will discuss in a minute.

12. 2:33

    But, like, just from the first principles, we can say that an agent is an LLM inside a, an agentic loop, which is connected to tools and can retrieve context.

13. 2:46

    That's it. Now, there are a couple of classes of problems that we will analyze today. So the first one is bad performances on the evals, and we will see what the evals are in a minute.

14. 2:59

    And the second one is bad performances on live data, which sometimes is similar to what we have on the evals. Uh, but you know, live data coming from real users, uh, with, you know, real expectations from the system can be a lot wider and sometimes a lot messier as well.

15. 3:20

    So let's analyze the first, um, the first failure mode. So we have bad performances on evals. Before we analyze, uh, this failure mode, I just want to introduce you to co- to the context of a Golden dataset.

16. 3:35

    Basically, a Golden dataset is a file or a set of files that we develop together with the subject matter experts when we are building our AI agents. And basically, this file defines what is the input that the system should retrieve, um, and should get, sorry, and what is the expected output that we should get.

17. 3:57

    Now, in the naive case, the expected output, as we can see here on the right, can just be like impossible or like a number or a text, so it can be a lot of things.

18. 4:10

    In real-world scenarios, the expected output can be, for example, I do expect the system to call this tool, uh, or call this tool with this parameter or, uh, call this tool in this chain, right?

19. 4:23

    Because maybe I want to, uh, do a retrieval and then do an update, you know. So how we define the output is very different in every, in every agent, and the idea is that we want a way basically to ensure that our agent is working correctly, so that's why we are building the Golden dataset.

20. 4:45

    You can see the Golden dataset as a test suite, but in a non-deterministic scenario. So when we build the Golden dataset, uh, we also have to build a scorer or a set of scorers that can basically, um, go through the Golden dataset together with the LLM, uh, and then can give us basically a, a number in the

21. 5:05

    end, which is saying what is the current accuracy of the, of the system so that, you know, we have a baseline, we can look for regressions, and we can further improve the system over time.

22. 5:17

    Now, let's, let's assume that we have a very simple Hello World agent. In this case, I'm using Maestra. It's a very, very simple agent, as we can see. I've, I've called it math agent, but in reality, in the Golden dataset, we have, uh, multiple types of questions, right?

23. 5:33

    Uh, but let's assume that this is our very simple agent, right? So it has a name. It has just couple of instructions, uh, has, has a model, of course.

24. 5:41

    As you can see, it doesn't have any tools yet, so it's like th- really the bare minimum. Um, we also built a naive evaluator. So if you remember from the Golden dataset, we only have inputs and outputs, and what this very naive evaluator is doing is checking whether the, um, LLM output contains the ground

25. 6:06

    truth output that we are expecting. Of course, once we run the eval suite here, the results are very poor. So as we can see, um, we have a pass rate of 18%, right?

26. 6:19

    Uh, just because a, a lot of questions are simple enough, like additions, multiplications, right, are simple enough so that the actual, um, LLM has this knowledge in its training data, and it doesn't need access to any external context or, like, any particular tool to answer these questions.

27. 6:41

    So 18% of the questions can be answered by the weights of the LLM, the rest can't. So we need a way to actually improve this pass rate.

28. 6:53

    Some failure modes that we can expect are, well, first of all, the agent is missing the right tools, right? So let's assume that I expect from my agent to be able to surf the web, fetch web pages.

29. 7:06

    If I'm not giving those specific tools to my agent, of course, every eval that is expecting that, it's gonna fail miserably. So first of all, we are maybe just missing some tools, or, uh, we don't have the right system prompt, because the system prompt basically contains all the instructions, uh, all the behaviors that we should expect and

30. 7:27

    that we shouldn't expect from our agent. So in a lot of cases, um, a lot of the optimizations would be just to tweak the system prompt and update it so that, um, you know, our agent has all the information it needs to work on our domain.

31. 7:45

    And then, um, of course, context retrieval, which is maybe our agent doesn't know where to look for information. Of course, usually context retrieval is implemented as tools, right? Um, maybe it's, I don't know, an average search in a RAG system, maybe it's a fetch on the web, you know, all these kind of things.

32. 8:06

    So we need to give enough, um, let's say, tools, uh, to our agent so that it can retrieve all the context it needs, of course, in a secure way.

33. 8:18

    Now, the question is, can an AI agent improve another AI agent autonomously or semi-autonomously? And, um, Karpathy, Andrej Karpathy, which is one of the most well-known AI scientists on the planet at the moment, implemented something which is called auto-research.

34. 8:39

    Basically, auto-research is a, uh, loop which, um, updates the code, updates the hyperparameters, updates the, the actual Python code of machine learning algorithms. And it actually shows that a coding agent tweaking the code of a machine learning, um, of a deep learning algorithm can actually improve the results,

35. 9:04

    right? So that's the, um, that's, that's the loss that we can see. And basically, every, uh, dot here in this chart is an example, is, is a test that the system is doing, um, by changing some of the parameters.

36. 9:21

    And we can see that the baseline has a specific accuracy, and while we go, um, while we go with the number of experiments, accuracy is improving, and therefore, of course, the loss, um, the, the loss is decreasing.

37. 9:36

    Now, the question is, can we do the same with AI agents and not just, like, machine learning models?

38. 9:43

    The answer is yes. So I built something which is called AutoAgent, which is basically the same idea, but applied to, um, actually, like, AI agents, right? So basically, this is a loop which is able to run the evals and then, um, update the code of the system, try new system prompts, create new tools, doing

39. 10:08

    everything autonomously, and then check again whether things improved or not.

40. 10:13

    And it actually works quite well. So here we have a few iterations, and this is the AI agent, this is the naive AI agent that, agent that we have seen earlier.

41. 10:25

    And in that agent, as you can see here on the bottom left, um, we have the baseline accuracy, which was 18%, and we managed to reach up to 83%, um, in, like, something around 10, uh, 10 iterations.

42. 10:42

    Now, of course, that's a, um, that, that's a very naive case, right? Because we started with something which didn't have any tools and nothing else. So it's relatively easy for the system to improve this agent.

43. 10:58

    But, uh, we also improved, um, some evals by 10% on a production agent that was already humanly optimized. So actually, the, the machine, uh, the,

44. 11:12

    the, the agent, the, the coding agent found new ways that humans didn't find, um, to improve the agent, and we got plus 10% on some of our internal benchmarks.

45. 11:24

    Now, how does it works? Actually, I ju- I gave already, like, a few insights of how this works, but let's see it in details. So basically, the core idea is that we have a coding agent, in this case, Cloud Code, but it can work with multiple, uh, coding agents.

46. 11:42

    It builds the agent, so it writes the code of our target agent, and then the target agent is giving feedback to our Cloud Code or whatever, uh, coding tool we are using, um, as evals, right?

47. 11:58

    So it's giving some information as, "Okay, here I got some regressions," or, "These evals are now passing." Um, also, Cloud Code can read the traces, the thinking traces, or, like, the, the full trace of the target agent to see whether something broke or something is not working as expected, so it can do, like, self-improvement.

48. 12:21

    Of course, there is the human in the loop. Uh, and the human in the loop is usually in structuring the initial agent and giving as much context as we can to, um, our coding agent about what it can do, what are the relevant files, uh, what it cannot do.

49. 12:38

    Like, just to give an example, updating the Golden dataset or the scorers, uh, just to let the eval pass is not a good idea, [chuckles] so we want to enforce, we want to tell the, um, we want to tell the AI agent to not do that, right?

50. 12:52

    So as humans, we can, um, we can steer a little bit our, our coding agent so that it can optimize the target agent.

51. 13:02

    Now, there are a few steps, right? The first step is to create an optimization job. Of course, it's very easy. Everything is a markdown. Uh, we all love markdown now.

52. 13:12

    So we have... We can define the objective, we can define the target repository, the metrics, and literally everything, right? So in this file, we can give as much context as we want to our coding agent so that it can actually, uh, run and improve the target agent.

53. 13:31

    And then the second step is to run the loop. So basically, we run the evals once, so we, we, we generate our baseline data, so we understand what's working, what's not working.

54. 13:41

    You know, the system will generate the first report, and then we run the loop, the optimization loop. So we can see that in the first case, uh, we get a regression.

55. 13:51

    So basically, the system works by creating a hypothesis, so it's tackling one class of problems at a time. It's updating the, the agent, and it's running the evals again.

56. 14:02

    And then it's seeing whether we got a regression, an improvement, or something else, uh, uh, happened. And so the agent can decide whether to roll back the change or move on.

57. 14:15

    So in this case, in the first iteration, we got a regression, so we want to roll back. Um, then we see a 5% improvement, which is great. So in this case, we are getting, uh, we are creating a new hypothesis.

58. 14:28

    We get a 0% improvement, unfortunately. But of course, the agent is keeping track of all the hypothesis. Uh, and so in this case, can go and create a new one, and we can see a 12% hypothesis and, and so on and so forth, right?

59. 14:43

    Of course, we define the number of iterations that we wanna run.

60. 14:48

    Um, of course, the way we create the baseline data is, you know, we just, uh, run the evals once, and we generate basically a document. And this is super helpful because it gives the coding agent the informations about the current status of the system.

61. 15:04

    Then this generates a baseline report, which contains, you know, all these informations. Uh, it contains what are the cases, it contains a summary of what's working, what's not working.

62. 15:16

    So basically, um, our coding agent, it's gonna have a clear picture of what's happening right now. And so it will try to understand, "Okay, how do I improve this system from this moment?"

63. 15:29

    And so from now on, we can start with the first iteration. So every iteration starts with a new branch. We create a hypothesis. Uh, the system changes the agent to implement that hypothesis.

64. 15:42

    We run the evals, we run our eval suite, and we generate a report.md file which contains everything that happened after we run the evals. We update the memory file, uh, which is like a global memory file across all the runs.

65. 15:58

    And if the metrics improved, then we continue from this branch. Um, if the metrics didn't improve or we have a strong regression or something bad happened, uh, then we roll back to the previous branch.

66. 16:10

    Of course, uh, the generated hypothesis are based on what the agent reads when it starts the investigation, which is the memory file, other reports file, so it has access to pretty much everything.

67. 16:24

    And that's what happens when we run multiple iterations, right? So let's assume that in the first case, I create a branch, I create an hypothesis, uh, I run the evals, everything goes well, so I get an improvement, so I am fixing one failure modes.

68. 16:38

    Um, therefore, I continue from that branch. I create a new one so that I can keep track of every change. Um, I create a new... that new branch. I try a new hypothesis.

69. 16:50

    This time it didn't really improve, so maybe I get a regression, maybe things didn't improve really. Uh, so what I do is I roll back to the previous branch, so the first branch where we had a real improvement in the system.

70. 17:04

    And then from here, I create a new branch, and I try a new hypothesis, and so on and so forth.

71. 17:11

    At the end, what we are building is a full change log of all the changes we have done, um, every improvement in... or improvement or regression, as we can see here on the right.

72. 17:25

    Um, we are seeing all the hypothesis that we are doing. And of course, if we want, we can also see all the hypothesis because every hypothesis is building a report.

73. 17:35

    So let's assume that something didn't really go well in terms of evals. Maybe the evals didn't improve after an hypothesis, but maybe that hypothesis was promising, right? Maybe the agent was onto something, but it just didn't implement the system, um, the, the change in a correct way, right?

74. 17:55

    What we can do is, as humans, we can just go back into the hypothesis, read, understand what the agent was trying to do, and then maybe steer it in the right direction next time.

75. 18:07

    Of course, um, this is the real task on a, on a real agent that we did have. Um, and as we can see here, the baseline accuracy was 67%.

76. 18:21

    Um- But then in something around 10 iterations, we managed to reach 86% in our evals without actually cheating because it found edge cases, it improved the system prompt, uh, it improved the tool descriptions to catch more edge cases, and it also fixed some tools' logic.

77. 18:44

    So that's a real agent that is now running in production. Uh, and actually, this system found ways to improve it more and more.

78. 18:54

    Now, the second class of problems that we are gonna tackle today is not the evals data, but bad performances on live data. By live data, we mean data that we collected by our real users, beta testers, subject-matter experts, you know, all the people that are actively testing and using the system, and they're giving us a feedback.

79. 19:19

    Just as an example, you can see here on the right, "Was this response helpful?" And you get yes or no, and then you get a note. And basically, we leverage this data to, uh, further optimize our agents.

80. 19:33

    Now, how do we do that? Uh, this is the full flow that we will see in details. So first of all, of course, we will need our users to actually use the system.

81. 19:44

    Uh, then we collect all the traces, right? So we collect all the informations from our users. Our users can either give us a feedback that we have just seen, so thumbs up, thumbs down with a, with a comment, or, uh, our subject-matter experts can annotate the trace.

82. 20:02

    So basically, they can go on the platform that we are using. They can look at the trace, look at the input, look at the output, uh, look at basically, like, all the tools that have been called, uh, look at the agent behavior, and then annotate the trace by giving us a feedback about how the agent performed and

83. 20:21

    how it should have performed. Once we collect a, a, a relatively good number of traces, o- once we have, um, enough information, we basically run an agent workflow that we built.

84. 20:34

    So basically, we analyze all the traces with both the negative and positive feedback, but we are more interested about the negative feedback here. So we analyze all those traces.

85. 20:46

    We do clustering of the failure modes, right? So bas- based on all the feedback that we collected, we try to understand, okay, given all this feedback, what are, like, the five, six, seven failure modes that we have here?

86. 21:00

    And then we analyze these, um... We analyze these clusters of failure modes with our subject-matter experts. So we say, "Okay, uh, based on those 10 traces, we can note that the agent in this specific case performed poorly because of reason X," for example.

87. 21:20

    We validate all this with our subject-matter experts, and then we do have a fixed, uh, proposal which gets generated and implemented by our coding agent. Then we implement this, um, and then we ship it to production to see whether it actually improved on real data.

88. 21:37

    And of course, we use the traces that we do have as regressions, so we test our fix against our traces, first of all. But let, let's see that in detail.

89. 21:49

    So first of all, the user tests the system. So in this case, we have a very simple example, uh, how can I optimize a React component? Uh, here we have the answer.

90. 22:00

    Here we have, you know, the information about the component, whatever, right? So that's the first step. Second step is, uh, we are gonna collect all the tracing, um, information.

91. 22:12

    So we are collecting everything from user interaction, um, the tool usage, the, uh, how much time the AI took to respond, you know, how many tokens we burned, everything.

92. 22:25

    The second step then is, uh, the user, once we collected the, the user behavior, we do expect the user to give us some feedback. Um, as I was mentioning, thumbs up, thumbs down, uh, and then a, a comment saying what went well, what didn't, um, and what's the expected behavior of the system.

93. 22:47

    And then, of course, uh, the other option that we were discussing is that the actual subject-matter experts can annotate the trace. So in case I don't have a feedback from the real user, I can ask the subject-matter experts to take a look at the trace, invest their time, and give us feedback as they were users.

94. 23:05

    The third step is now to collect all the traces with feedback and download them locally in a JSON or whatever format you're using. As you can see here, in this case, we have 114 traces, and we're using those traces to then do the cluster analysis.

95. 23:21

    How do we run the analysis? Well, this is actually a skill that we built. Basically, the skill is gonna instruct a coding agent properly to go in the, go in the JSON file, look at all the traces, run some clustering.

96. 23:38

    Um, then we do an adversarial review of that. So basically, um, once we have all the clusters, we do also a little bit of root cause analysis because, of course, our coding agent has access to the actual code of the AI agent, so it can do some root cause analysis.

97. 23:54

    It can try to understand what happened, uh, especially because it has also access to the details of every trace, so it can go within the trace, understand why the agent answered that way, you know, trace it down to the root cause.

98. 24:08

    Um, and then what's gonna do, it's gonna, um, propose possible improvements. And what we get at the end of this process is a full markdown report, which is very helpful for us.

99. 24:21

    Uh, it's gonna give us, like, the positive feedback, the negative feedback, the negative rate. You know, we are just trying to collect as many data as possible And so we get this very complete, um, executive summary and this very complete report, which tells us what are, like, all the clusters which are not working as the user was

100. 24:43

     expected. So in this case, uh, we have that... We have a small markdown formatting failure, which is saying URLs are not hyperlinked, and there are some formatting inconsistencies. So it's linking some of the traces with the trace ID.

101. 24:58

     Uh, it's mentioning the user feedback. It's showing, like, some root-- possible root causes, and it's telling us, okay, how do we fix it, right? Once we have this full report, uh, our work is not done yet because this report will contain all the possible failure clusters reported by users.

102. 25:18

     But now what we have to do is to triage everything and validate with subject-matter experts. Once we are doing-- Once we do the triage and the pri- and the validation, then we prioritize, and we understand what we wanna fix now, fix later, don't fix.

103. 25:35

     Um, and so either we, uh, fix it with the agent, so we give to the agent, you know, all the context. We give it the failure mode. We give it the, uh, traces, and we ask our agent to fix it, or we discard it because sometimes it can be the, the failure cluster can be a false positive

104. 25:55

     or can be an intended behavior, and maybe, uh, the user gave us, um, a feedback which is not really useful for us at this moment. So, you know, th-there is a little bit of human judgment here.

105. 26:07

     As I was mentioning earlier, all the failure modes that we are finding during this investigation step, they will become part of the Golden dataset that we mentioned earlier, and the eval suite is updated to spot those regressions.

106. 26:21

     So in case this failure mode, for whatever reason, is introduced, uh, later on in the code, we will spot it very fast because now it lives into our Golden dataset and our scorers.

107. 26:33

     In terms of how often the report gets generated, it really depends on your use case. Uh, we found out that once per sprint is good enough, but it really depends on how much data you have, right?

108. 26:44

     So if you have 100 traces, it's, uh, you might want to generate that at 100 traces. But if you have, like, 10,000 traces with feedback, might be too much, so you might want to, you know, generate multiple reports.

109. 26:58

     So it's really on your use case. We found out that once per sprint is actually reasonable. Uh, in multiple use cases, what we found is that a coding agent, when instructed, uh, has been able to fix an entire suite, uh, of, of issues like the one that we have seen earlier, uh, with just one prompt.

110. 27:18

     So once we give enough context, and once we give the coding agent the ability to, uh, test its changes against some regression tests, uh, then everything becomes way, way easier for the, for the coding agent.

111. 27:33

     And then, uh, in some complex cases, uh, we have paired this approach with AutoAgent. Uh, and so we basically pointed out AutoAgent to some of the failure clusters, um, and we let a AutoAgent build some draft PRs for us.

112. 27:51

     Now, of course, both AutoAgent and this pipeline to improve, um, agents based on live data, they're possible thanks to harness engineering. So basically, the core idea is that we are leveraging a set of things so that our coding agent has a powerful enough harness so that it can change the code, validate its own changes, which is, like,

113. 28:16

     super important, uh, and then propose new changes if the updates that it did didn't lead to the, to, to the outcome that we were looking for, right? So just very, very quickly, harness engineering is the idea of building the environment around our coding agents so that they can work reliably.

114. 28:36

     Uh, so we are giving all the constraints, uh, all, all the tasks, the feedback loop. We are giving the governance to our, um, coding agents, and we are doing that with a few things, right, which we already mentioned.

115. 28:48

     The first one is spec-driven development. So for example, when we are trying to fix some, um, failure modes, every failure mode becomes a spec. Um, we are creating a spec for the expected behavior of the agent, and then we are implementing that.

116. 29:04

     Then we have quality gates, uh, like linting, uh, our unit tests. We have our, our evals. Um, we have LLM, uh, code review, so a bunch of stuff. Uh, everything related to context engineering, so giving the right context to our agents.

117. 29:21

     And then, of course, observability, um, because we need to know what's happening, right? Um, if we don't know what's happening when we ship in production, we are basically blind, right?

118. 29:32

     So we want to ensure that we know what's happening so that our coding agent can fix any bugs or anything once, once it's found. So, um, I hope that this has been interesting or helpful.

119. 29:46

     I hope you're gonna implement some of those techniques in your agents. And in case you have any questions about this talk, uh, please feel free to reach out on LinkedIn.

120. 29:55

     I would love to have a chat with other people which are building agents. I know how hard it is. I know how interesting it is. So in case you have, like, anything that you would like to talk, uh, I would love to get a message on LinkedIn.

121. 30:08

     So I hope you enjoyed this talk, and wish you a great day. Cheers.