# Let's Build an Agent from Scratch — Kam Lasater

Kam Lasater, Manufactured

AI Engineer Summit 2025

[Talk page](https://ai.engineer/talks/xzXdLRUyjUg-lets-build-agent-from-scratch) · [Video](https://www.youtube.com/watch?v=xzXdLRUyjUg)

## Let's Build an Agent from Scratch

Kam Lasater builds an agent one capability at a time, showing how tool execution, completion checks, and mutable task state turn a simple model call into a planning workflow.

Familiarity with TypeScript, asynchronous functions, JSON, and a basic LLM API call will help you follow the implementation.

### What has to surround a model call?

[0:01](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1s)

What is the smallest agent you can build well enough to run, break, and understand? That is the starting problem in Kam Lasater’s tutorial. The aim is to expose the machinery beneath an agent framework, so that using a framework becomes an informed choice. The [companion code](https://github.com/mfdtrade/agent-talk-2025) provides successive versions to experiment with as the behavior moves beyond a single request and response.

Start with an LLM, memory, planning, tools, and a while loop. Then split memory into **read and write operations**, and split the while loop into a condition and iteration. This decomposition gives the build its sequence: call the model, judge an answer, execute tools, support repeated calls, and finally add a plan that the agent can read and update. None of these components requires a framework to make its role visible.

![Black slide with three equations expanding memory into read and write, and a while loop into condition and loop.](https://ai.engineer/api/talk-essay-media/b9134b8fd458d3accebe5bf3826b2b49be791bce80c3611c4a0b8d223c2bee78)

Breaking an agent into an LLM, condition, tools, read, write, planning, and loop.

### Answer a question, then judge the answer

[1:41](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=101s)

Step zero is an ordinary OpenAI Chat Completions call. Step one adds a second model call: a strict critic receives the original question and the proposed answer, then decides whether the answer fully addresses the question. Its response must follow a strict JSON schema containing a Boolean `done` field. This article retains the recording’s Chat Completions approach; current API documentation may use different request shapes.

The question is the average wing speed of a swallow. After the first model answers, the judge returns a thumbs-up. The application now has an answer and a machine-readable completion decision, but those are separate outputs with separate failure modes. **A schema-valid Boolean is not proof that the answer is correct.** Lasater describes this stage as deterministic and mechanistic: the workflow is fixed—generate, judge, respond—even though the model’s content is not guaranteed to be identical on each run.

### Let the model request a search

[3:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=199s)

Step two changes the task to finding a fur-lined hoodie near Times Square in New York. To answer with outside information, the agent gets a `searchGoogle` tool backed by [SerpApi](https://serpapi.com/search-api). Its `getJson` call supplies the Google engine, an API key, a query, and a location. The implementation defaults the search location to Philadelphia, which is distinct from the location requested in the shopping task.

The tool has two interfaces. The local function performs the search and prints results. A handwritten tool configuration tells the model the function’s name, description, strictness, and parameters. It describes `query` and `location`, identifies required fields, and specifies whether additional properties are permitted. That schema is how the model learns what it may request; it does not execute the search.

**The application executes the tool; the model requests it.** The initial implementation handles one requested call at a time:

1. Call the model with the conversation and tool configuration.
2. Inspect the returned tool call for its function name and arguments.
3. Dispatch the request to the local `searchGoogle` function through the tools collection.
4. Append both the assistant’s tool-call message and the tool’s response to the conversation.
5. Call `completeWithTools` again with the expanded conversation.

The OpenAI SDK does not automatically run these local functions. The recursive call gives the model the search result and another opportunity to choose what happens next.

![Code editor showing completeWithTools parsing tool arguments, invoking a local tool, appending messages, and calling itself again.](https://ai.engineer/api/talk-essay-media/c1d07e67e63105c2abfcf683e16ba028d42c30251ced22ae19dc58edffcb49a5)

Local code executes a requested tool and appends its result to the conversation.

Once the model stops requesting tools and produces an answer, `completeWithTools` returns it to the main flow. The external critic then checks that answer, just as it did for the swallow question. There are now two distinct decisions: whether another tool call is needed, and whether the resulting answer satisfies the user’s request.

### A valid-looking argument can still fail

[6:31](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=391s)

The live search run produces an unhandled rejection. Lasater attributes it to a validation constraint on `location`: the value must come from a particular set of accepted strings, rather than being any arbitrary string. A parameter can therefore look reasonable to the model and still fail the service’s requirements.

Lasater also reports another judge thumbs-up, without explaining how that approval relates to the failed request. The approval cannot establish that the search succeeded. At this stage, the same LLM chooses to search and evaluates whether the returned text reasonably answers the hoodie question; the surrounding workflow still feels straightforward and mechanistic to him.

### Execute multiple calls without losing their identities

[7:30](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=450s)

Step three keeps the hoodie request but moves `completeWithTools` into a utility file. The helper now accepts an array of requested tool calls and executes them with `Promise.all`, passing each call’s arguments to the corresponding local function. Each result carries a **tool call ID** so the model can associate it with the request that produced it.

Function names alone are insufficient: the model may request `searchGoogle` several times with different parameters. In TypeScript, the dispatch portion can keep each request’s identity attached to its result like this:

```typescript
type ToolCall = {
  id: string;
  function: { name: string; arguments: string };
};

type Tool = (args: Record<string, unknown>) => Promise<string>;

type ToolResult = {
  role: "tool";
  tool_call_id: string;
  content: string;
};

async function executeToolCalls(
  calls: ToolCall[],
  tools: Record<string, Tool>,
): Promise<ToolResult[]> {
  return Promise.all(
    calls.map(async (call): Promise<ToolResult> => {
      const tool = tools[call.function.name];
      if (!tool) {
        throw new Error(`Unknown tool: ${call.function.name}`);
      }
      const args = JSON.parse(call.function.arguments);
      const content = await tool(args);
      return {
        role: "tool",
        tool_call_id: call.id,
        content,
      };
    }),
  );
}
```

The surrounding helper appends these results after the assistant message that requested the calls, then asks the model to continue. The ID preserves the relationship even when the same function appears more than once.

Lasater’s preferred mental model for a tool is a **text transformation**. `searchGoogle` accepts a query object and returns a string; even when inputs have several fields, the model-facing interaction is largely text in and text out. The refactored run returns sites and another judge approval. Parallel dispatch improves the machinery, but it has not yet produced the behavioral change he is looking for.

### A to-do list supplies planning and memory

[10:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=619s)

Step four adds a to-do list, the point Lasater identifies as the behavioral inflection. The same mechanism now supports planning, reading state, and writing state. Iteration still comes from the exchange between model and application: the model requests another tool, application code executes it and sends back the result, and a subsequent model call decides how to proceed.

That exchange does not guarantee convergence. A model can keep requesting tools indefinitely, especially if context pruning prevents it from reaching a context boundary or error. Lasater describes client-enforced iteration limits as a possible guardrail, but the demonstration does not implement one. Moving the next-action decision into the model does not remove the application’s responsibility for bounding execution.

The storage is deliberately ordinary: one array holds pending to-dos, and another holds completed items. The add operation accepts an array of tasks, pushes them into pending work, prints them, and returns the tasks that were added. It also has a tool configuration so the model can request the operation.

![Code editor showing todos and done arrays, addTodos, its Zod tool configuration, and the beginning of markTodoDone.](https://ai.engineer/api/talk-essay-media/79145f92062d2058d8ab4ba23b87b972ad6c7edce613c526448a34613d598175)

To-do list storage, the add function, and its tool configuration.

The remaining operations make that state usable across successive model calls:

| Operation | Effect |
| --- | --- |
| Mark a to-do done | Remove an existing task from pending work and record it as done |
| Check done | Inspect completed work; report when no tasks have been marked done |
| Check to-dos | Retrieve the pending list |

Each operation has a corresponding tool configuration. Removing completed tasks from the pending list helps prevent repeated work. Inspecting the completed list is also distinct from deciding whether the user’s overall goal has been satisfied.

### Give the tools a working procedure

[12:56](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=776s)

The available tools now include the judge, search, the to-do operations, and `browseWeb`. The browsing tool takes a URL, retrieves its content, and uses [Cheerio](https://cheerio.js.org/docs/intro/) and [Turndown](https://github.com/mixmark-io/turndown) to return Markdown after an HTTP 200 response. This is page retrieval and conversion, not a full browser that renders a page and executes its JavaScript. The result is text the model can use in its next decision.

The prompt then supplies a procedure for using those capabilities. It casts the model as an assistant to a busy executive, with friendly, short, clear, direct writing. More consequentially, it tells the model how to organize the work:

1. Make a plan in the to-do list before doing any work.
2. Use available tools to carry out the task.
3. Mark tasks complete after doing them.
4. Check the list and create a report of the actions taken.
5. Ask the assistant to check whether the goal is done.
6. Send the report if approved; otherwise, add the feedback to the to-do list.

The list stores the work; the prompt establishes when and why to update it.

Lasater also includes the current date to orient the model in time. The judge has moved from a fixed call after the main workflow into the available tool set, so the completion condition itself becomes something the model is instructed to invoke. The main function can now make one call to `completeWithTools` and return its final answer. That compact entry point still wraps multiple model requests and local tool executions.

### Watch the agent research how to build agents

[14:48](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=888s)

The default step-four request is to learn about building agents without a framework. Lasater runs the step through npm, and the model first calls the add-to-dos tool. Its generated plan has three parts: search for information, summarize the key points, and ask the assistant whether the explanation is sufficient. The model is now choosing intermediate work and recording it before producing the final answer.

The run checks the to-do list, calls `searchGoogle`, and browses multiple pages. Markdown comes back into the conversation. The agent then marks work done, checks its to-dos, and requests the goal check. The judge’s thumbs-up precedes the final report. This sequence makes the planning cycle visible through ordinary tool calls and state changes.

The resulting report describes an agent as a language model with tool use and conversational context. It identifies tools such as database queries and web search, then outlines input processing, tool decisions, execution, response formulation, prompt engineering, and direct API calls. Lasater particularly endorses the suggestion to **store state directly and access it in code** instead of passing everything through the LLM. The to-do arrays already embody that choice: the model requests changes, while local code owns the stored state.

### Try a broader task, then extend the same machinery

[16:35](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=995s)

For a broader test, Lasater asks for help planning a Saturday date with his wife, finding local activities and dinner between 6 PM and 11 PM. The generated plan separates finding local activities from finding a dinner option. It searches Google for restaurants, browses pages, receives Markdown, marks work done, and checks the goal.

The final response offers activity and dinner options. Lasater does not present the agent as perfect; what interests him is how the components now work together. A model call and a condition established the initial workflow. Tools connected it to outside information. Planning and the to-do list’s read/write operations gave it a way to organize and track intermediate work on a less narrowly specified request.

A possible next layer would retain the browsed pages in a vector database—perhaps an in-memory Chroma database—or use them in a retrieval-augmented system. These are proposed extensions, not capabilities demonstrated in the run. They would build on an existing path from URL to page text, adding a way to retrieve that information later.

The exercise ends with an invitation to run the code, break it, and suggest improvements. Lasater points readers to the slides on his personal site, the code on GitHub, and LinkedIn for contact. Having watched each layer appear, the useful experiment is to change one of them and observe how the agent’s behavior changes: the implementation is small enough that the relationship between a tool, a prompt instruction, and the resulting work remains visible.

![Thanks slide listing Kam Lasater's LinkedIn profile, slides at kamlasater.com/talks/agents-2025, and code at github.com/mfdtrade/agent-talk-2025.](https://ai.engineer/api/talk-essay-media/fe0755904af0dfedc42d60fd2f88c38077ecfc4f46a69efa7fbcefcbe4e235da)

Closing resources: LinkedIn, slides, and example code.

## Resources

- [Let's Build an Agent — companion code](https://github.com/mfdtrade/agent-talk-2025) — The February 2025 tutorial repository, progressing from an LLM call to tools, to-do planning, memory, and iteration.
- [OpenAI function calling](https://developers.openai.com/api/docs/guides/function-calling) — Current documentation for defining functions, executing model-requested calls, and returning results to the model.
- [OpenAI structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs) — Current guidance for schema-constrained responses, including their remaining correctness limitations.
- [SerpApi Google Search API](https://serpapi.com/search-api) — Google search results through a JSON interface, with query and geographic-location parameters.
- [Cheerio](https://cheerio.js.org/docs/intro/) — Parse and extract HTML using a jQuery-like interface without running a browser.
- [Turndown](https://github.com/mixmark-io/turndown) — A JavaScript library for converting HTML strings and DOM nodes into Markdown.

## Slides

### Speaker and resources

Kam Lasater shares his slides and code and encourages you to run and break the example to see when deterministic outcomes begin to feel like an agent.

[0:47](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=47s) [Slide listing the speaker’s profile, presentation slides, and code repository.](https://assets.ai.engineer/sites/aiecode2025/sha256/5b22f706694ee434e64262b172c3e64308f5825c91d0efa5f8e70fb724e256a6.webp)

Kam Lasater
https://www.linkedin.com/in/kamlasater/

Slides
https://kamlasater.com/talks/agents-2025

Code
https://github.com/mfdtrade/agent-talk-2025

### What is an agent?

Lilian Weng defines an agent as an LLM with memory, planning skills, and tool use. swyx adds a while loop to that definition.

[1:08](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=68s) [Slide showing two definitions of an agent in terms of a language model, memory, planning, tools, and a loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/dd9ce4f5517bd5fcaa61cef027a9a56ec470c0c6a2e7cc2c8a4c2c868b843666.webp)

What is an agent?

swyx ➡ @aidotEngineer NYC
SOON
@swyx

agent = llm + memory + planning + tools + while loop

Lilian Weng @lilianweng · Jun 26, 2023
Agent = LLM + memory + planning skills + tool use

This is probably just a start of a new era :)

lilianweng.github.io/posts/2023-06-...

### Breaking down an agent

Lasater expands memory into read and write operations and the while loop into a condition and a loop, then reorders the pieces into a plan for the code.

[1:17](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=77s) [Three equations expand memory into reading and writing and a while loop into a condition and loop, then reorder the components.](https://assets.ai.engineer/sites/aiecode2025/sha256/2aa696e1c2b8d3ba7b4a7608390eec57d8c8cce58f5a63b38c9ad9ff958b909c.webp)

agent = llm + memory + planning + tools + while loop
agent = llm + (read + write) + planning + tools + (condition + loop)
agent = llm + condition + tools + read + write + planning + loop

https://kamlasater.com/talks/agents-2025

### Calling a language model

The first step calls GPT-4o with a developer instruction to say “Hello World” when asked its name, then logs the first completion choice.

[1:40](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=100s) [Code editor showing a JavaScript chat completion with developer and user messages.](https://assets.ai.engineer/sites/aiecode2025/sha256/8e5af33c23dbeed4ad6671def7d6a79a30ed96944c817e1cc83fa32e19ca9c69.webp)

// From OpenAI docs:
// https://platform.openai.com/docs/api-reference/chat/create
import OpenAI from "openai";

const openai = new OpenAI();

async function main() {
  const completion = await openai.chat.completions.create({
    messages: [
      { role: "developer", content: "You are a helpful assistant, if asked your name say Hello World."},
      { role: "user", content: "What is your name?" }
    ],
    model: "gpt-4o",
  });

  console.log(completion.choices[0]);
}

main();

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js
AGENT-TALK-2025
node_modules
src
_steps
step0-llm.js
step1-condition.js
step2-tool.js
step3-refactor.js
step4-planning.js
tools
utils
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
README.md
Ln 19, Col 8  Spaces: 2  UTF-8  LF  JavaScript

### Language-model response

Running step zero returns an assistant greeting and a completion with finish_reason set to stop.

[2:02](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=122s) [Terminal showing the initial script’s assistant response and completion status.](https://assets.ai.engineer/sites/aiecode2025/sha256/e1f468e199ec37e905768fbb38d09b037c7eaab22124aacee25121f5af4f664c.webp)

npm run step0-llm

> agent-talk-2025@1.0.0 step0-llm
> node -r dotenv/config src/_steps/step0-llm.js

{
  index: 0,
  message: {
    role: 'assistant',
    content: 'Hello World. How can I assist you today?',
    refusal: null
  },
  logprobs: null,
  finish_reason: 'stop'
}

Interface text:
agent-talk-2025 git:(main) [21:16:41]
agent-talk-2025 git:(main) [21:37:58]

### Condition demo output

The demo answers the swallow question, then an LLM judge gives the answer a thumbs-up.

[3:04](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=184s) [Terminal output showing an answer to the swallow question and a thumbs-up judge result.](https://assets.ai.engineer/sites/aiecode2025/sha256/b8d8f6425c25855ddf3df44ee29f3c7facc71fd214fd349e12d6c4f78a25e62b.webp)

> agent-talk-2025@1.0.0 step1-condition
> node -r dotenv/config src/_steps/step1-condition.js

########################################
Question: What is the average wing speed of a swallow?

########################################
Answer: The question about the average wing speed of a swallow is a famous and humorous reference from the movie "Monty Python and the Holy Grail." In the film, the specific question is, "What is the airspeed velocity of an unladen swallow?"

From a more scientific perspective, for the European swallow (Hirundo rustica), the airspeed velocity is typically estimated to be around 11 meters per second, or approximately 24 miles per hour. However, this can vary based on conditions like wind and the specific subspecies of the swallow.

It's important to note that without specifying whether the swallow in question is an African or European swallow, the answer, much like in the movie, remains a humorous source of debate.

########################################
LLM as judge: 👍

Interface text:
agent-talk-2025 — ckl@flywheel — ...ent-talk-2025 — -zsh — 122×34
➜ agent-talk-2025 git:(main) [21:38:57]
➜ agent-talk-2025 git:(main) [21:38:58]
➜ agent-talk-2025 git:(main) [21:38:58]
➜ [illegible] git:(main) [21:38:58]
➜ [illegible] git:(main) [21:38:58]
➜ [illegible] git:(main) [21:38:59]
➜ [illegible] git:(main) [21:39:00]
➜ [illegible] git:(main) [21:39:00]
➜ [illegible] git:(main) [21:39:00]
➜ agent-talk-2025 git:(main) [21:39:00]

### Search function and tool configuration

The search function sends a query and location to SerpApi, then formats the first five organic results. Its tool configuration defines the query and location parameters and requires both fields.

[3:50](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=230s) [Code editor showing a search function and the beginning of its tool schema.](https://assets.ai.engineer/sites/aiecode2025/sha256/623f4b6fd7fee6f02bdeab8d4f3b55a81abc7a9893e014d183e7766449d0c161.webp)

const doneResponseSchema = z.object({
  done: z.boolean(),
});

async function searchGoogle({query, location="Philadelphia, PA"}) {
  console.log("\n\n"+"#".repeat(40))
  console.log(`Searching Google[${location}]: ${query}\n\n`)
  const resp = await serp.getJson({
    engine: "google",
    api_key: process.env.SERP_API_KEY, // Get your API_KEY from https://serpapi.com/manage-api-key
    q: query,
    location
  })
  const stringResult = resp.organic_results.slice(0,5).map(el => `${el.title}: ${el.snippet}\n${el.link}`).join('\n\n')
  console.log(stringResult)
  return stringResult
}

const searchGoogleToolConfig = {
  "type": "function",
  "function": {
    "name": "searchGoogle",
    "description": "Run a search query against google for information from the web.",
    "strict": true,
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "The search query to send to gogole."
        },
        "location": {
          "type": ["string","null"],
          "description": "What city to send the query from. This will effect the localization and provide better information fo[illegible]
        }

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:40 PM
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js
src > _steps > step2-tool.js > searchGoogle > resp
AGENT-TALK-2025
node_modules
src
_steps
step0-llm.js
step1-condition.js
step2-tool.js
step3-refactor.js
step4-planning.js
tools
utils
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
README.md
main  Ln 19, Col 22  Spaces: 2  UTF-8  LF  JavaScript  Cursor Tab

[4:15](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=255s) [Code defines a search tool with query and location parameters.](https://assets.ai.engineer/sites/aiecode2025/sha256/1a3671195f18846af081588c6d5e1f46b5cfa0c8878e0254fb947e7495e955b2.webp)

async function searchGoogle({query, location="Philadelphia, PA"}) {
  [lines 16–25 not visible]
  return stringResult
}

const searchGoogleToolConfig = {
  "type": "function",
  "function": {
    "name": "searchGoogle",
    "description": "Run a search query against google for information from the web.",
    "strict": true,
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "The search query to send to gogole."
        },
        "location": {
          "type": ["string","null"],
          "description": "What city to send the query from. This will effect the localization and provide better information fo[illegible]
        }
      },
      "required": [
        "query",
        "location"
      ],
      "additionalProperties": false
    }
  }
};

const tools = {
  "searchGoogle": searchGoogle
}

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:40 PM
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js
src > _steps > step2-tool.js > searchGoogleToolConfig
AGENT-TALK-2025
node_modules
src
_steps
step0-llm.js
step1-condition.js
step2-tool.js
step3-refactor.js
step4-planning.js
tools
utils
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
README.md
main  Ln 29, Col 33  Spaces: 2  UTF-8  LF  JavaScript  Cursor Tab

### Tool call loop

The local code executes the first requested tool, appends the call and its result to the conversation, then calls the model again. It repeats until the model returns an answer without a tool call.

[4:50](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=290s) [Code executes a requested tool, appends its result, and calls the completion function again.](https://assets.ai.engineer/sites/aiecode2025/sha256/ec13cbc633a62d752c2c004f5823d3c8110da9ccd72405b7bc03efd3c8adf214.webp)

async function completeWithTools(args) {
  const completion = await openai.chat.completions.create(args)

  if (completion.choices[0].message.tool_calls) {
    const toolCall = completion.choices[0].message.tool_calls[0];
    const toolArgs = JSON.parse(toolCall.function.arguments);

    console.log("\n\n"+"#".repeat(40));
    console.log(`tool_calling: ${toolCall.function.name}(${JSON.stringify(toolArgs)})`)
    const result = await tools[toolCall.function.name](toolArgs);

    args.messages.push(completion.choices[0].message); // append model's function call message
    args.messages.push({                              // append result message
      role: "tool",
      tool_call_id: toolCall.id,
      content: result
    });
    return completeWithTools(args)
  }
  console.log("\n\n"+"#".repeat(40));
  console.log(completion.choices[0].message.content)
  return completion
}

async function main() {
  const completion = await completeWithTools({
    messages: [{ role: "developer", content: prompt }],
    model: "gpt-4o",
    tool_choice: "auto",
    tools: [searchGoogleToolConfig],
    store: false
  });

  const answer = completion.choices[0].message.content

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:41 PM
agent-talk-2025
Tabs: step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js
src > _steps > step2-tool.js > completeWithTools
AGENT-TALK-2025
node_modules
src
_steps
step0-llm.js
step1-condition.js
step2-tool.js
step3-refactor.js
step4-planning.js
tools
utils
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
README.md
Chat ⌘L  Edit ⌘K
main  Ln 77, Col 29 (17 selected)  Spaces: 2  UTF-8  LF  JavaScript  Cursor Tab

### Tool helper and parallel calls

The refactored helper uses Promise.all to run requested tool calls in parallel, appends each result with its tool call ID, and calls itself again to continue the conversation.

[7:41](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=461s) [Code editor showing a helper that runs requested tool calls in parallel and appends their results.](https://assets.ai.engineer/sites/aiecode2025/sha256/311f3ba1d4d9f09ef478165bb8c2382cb03ac25e07aabdbc37fdf946f5188c3b.webp)

src/utils/ai.js

import OpenAI from "openai";
import tools from '../tools/index.js';

export const openai = new OpenAI();

export async function completeWithTools(args) {
  // console.log("\n\n"+"#".repeat(40));
  console.log(`Calling llm with: ${JSON.stringify(args.messages[args.messages.length-1]).substring(0,500)}`)

  const completion = await openai.chat.completions.create(args)

  // console.log(`Got back: ${JSON.stringify(completion.choices[0].message)}`)

  if (completion.choices[0].message.tool_calls) {
    const toolCalls = completion.choices[0].message.tool_calls;

    args.messages.push(completion.choices[0].message);

    const toolResps = await Promise.all(toolCalls.map(async (toolCall) => {
      const toolArgs = JSON.parse(toolCall.function.arguments);

      console.log("\n\n"+"#".repeat(40));
      console.log(`tool_calling: ${toolCall.function.name}(${JSON.stringify(toolArgs)})`)
      const result = await tools.functions[toolCall.function.name](toolArgs);

      args.messages.push({                    // append result message
        role: "tool",
        tool_call_id: toolCall.id,
        content: result
      });
    }))

    return completeWithTools(args)
  }

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:44 PM
agent-talk-2025
Tabs: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, ai.js, step4-planning.js
Explorer: AGENT-TALK-2025, node_modules, src, _steps, step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js, tools, utils, ai.js, .env, .env.sample, .gitignore, .nvmrc, LICENSE, package-lock.json, package.json, README.md
Breadcrumb: src > utils > ai.js > completeWithTools > toolResps
main  Ln 19, Col 76  Spaces: 2  UTF-8  LF  JavaScript

### Handling tool calls

The helper runs requested tools in parallel, adds each result to the conversation with its tool call ID, then calls the model again. If there are no tool calls, it returns the completion.

[8:14](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=494s) [Code editor showing tool responses added to the conversation, recursive continuation, and the final return.](https://assets.ai.engineer/sites/aiecode2025/sha256/4ba18d08852b5b3f7cc5c4b4bda0b0c472af3c925275f17c153e0ce75a9125ad.webp)

export async function completeWithTools(args) {
  // console.log("\n\n"+"#".repeat(40));
  console.log(`Calling llm with: ${JSON.stringify(args.messages[args.messages.length-1]).substring(0,500)}`)

  const completion = await openai.chat.completions.create(args)

  // console.log(`Got back: ${JSON.stringify(completion.choices[0].message)}`)

  if (completion.choices[0].message.tool_calls) {
    const toolCalls = completion.choices[0].message.tool_calls;

    args.messages.push(completion.choices[0].message);

    const toolResps = await Promise.all(toolCalls.map(async (toolCall) => {
      const toolArgs = JSON.parse(toolCall.function.arguments);

      console.log("\n\n"+"#".repeat(40));
      console.log(`tool_calling: ${toolCall.function.name}(${JSON.stringify(toolArgs)})`)
      const result = await tools.functions[toolCall.function.name](toolArgs);

      args.messages.push({                         // append result message
        role: "tool",
        tool_call_id: toolCall.id,
        content: result
      });
    }))

    return completeWithTools(args)
  }
  console.log("\n\n"+"#".repeat(40));
  console.log(completion.choices[0].message.content.substring(0,500))
  return completion
}

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:44 PM
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  ai.js  step4-planning.js
src > utils > ai.js > completeWithTools > toolResps > toolCalls.map() callback
AGENT-...  node_modules  src  _steps  step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js  tools  browseWeb.js  index.js  llmJudge.js  searchGoogle.js  todoList.js  utils  ai.js  .env  .env.sample  .gitignore  .nvmrc  LICENSE  package-lock.json  package.json
main  Ln 28, Col 35  Spaces: 2  UTF-8  LF  JavaScript  Cursor Tab

[8:58](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=538s) [Code executes requested tools, appends their results, and calls the completion function again.](https://assets.ai.engineer/sites/aiecode2025/sha256/885eca68af35806fab6a26302b63d8521a9c8bda56e940bf16180945eda985a6.webp)

export async function completeWithTools(args) {
  // console.log("\n\n"+"#".repeat(40));
  console.log(`Calling llm with: ${JSON.stringify(args.messages[args.messages.length-1]).substring(0,500)}`)

  const completion = await openai.chat.completions.create(args)

  // console.log(`Got back: ${JSON.stringify(completion.choices[0].message)}`)

  if (completion.choices[0].message.tool_calls) {
    const toolCalls = completion.choices[0].message.tool_calls;

    args.messages.push(completion.choices[0].message);

    const toolResps = await Promise.all(toolCalls.map(async (toolCall) => {
      const toolArgs = JSON.parse(toolCall.function.arguments);

      console.log("\n\n"+"#".repeat(40));
      console.log(`tool_calling: ${toolCall.function.name}(${JSON.stringify(toolArgs)})`)
      const result = await tools.functions[toolCall.function.name](toolArgs);

      args.messages.push({                         // append result message
        role: "tool",
        tool_call_id: toolCall.id,
        content: result
      });
    }))

    return completeWithTools(args)
  }
  console.log("\n\n"+"#".repeat(40));
  console.log(completion.choices[0].message.content.substring(0,500))
  return completion
}

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:44 PM
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  ai.js  step4-planning.js
src > utils > ai.js > completeWithTools > toolResps > toolCalls.map() callback
AGENT-...  node_modules  src  _steps  step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js  tools  browseWeb.js  index.js  llmJudge.js  searchGoogle.js  todoList.js  utils  ai.js  .env  .env.sample  .gitignore  .nvmrc  LICENSE  package-lock.json  package.json
~/git/mfd/agent-talk-2025/src/tools/llmJudge.js
main  Ln 28, Col 35  Spaces: 2  UTF-8  LF  JavaScript  Cursor Tab

### Search tool configuration

searchGoogle accepts a query object, searches Google through SerpApi, and returns the first five result titles and URLs as a JSON string. A Zod schema defines the query parameter for the tool.

[8:59](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=539s) [Code defines a search function and its query schema and tool configuration.](https://assets.ai.engineer/sites/aiecode2025/sha256/a6810fe6cf541d34ae34748544901ae3f03bb0fcb116b7311f20379d08dcca05.webp)

import * as serp from "serpapi";
import { z } from "zod";
import { zodFunction } from "openai/helpers/zod";

export async function searchGoogle({query}) {
    // console.log("\n\n"+"#".repeat(40))
    // console.log(`Searching Google: ${query}`)
    const resp = await serp.getJson({
        engine: "google",
        api_key: process.env.SERP_API_KEY, // Get your API_KEY from https://serpapi.com/manage-api-key
        q: query
    })
    // const stringResult = resp.organic_results.slice(0,5).map(el => `${el.title}: ${el.snippet}\n${el.link}`).join('\n\n')
    const stringResult = JSON.stringify(resp.organic_results.slice(0,5).map(el => {return {title: el.title, url: el.link}}))
    console.log(stringResult)
    return stringResult
}

// Define a Zod schema
const searchGoogleSchema = z.object({
    query: z.string().describe("The search query to send to gogole.")
});

export const searchGoogleToolConfig = zodFunction({
    name: "searchGoogle",
    description: "Run a search query against google for information from the web.",
    parameters: searchGoogleSchema
})

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:45 PM
agent-talk-2025
Tabs: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, searchGoogle.js, step4-planning.js
src > tools > searchGoogle.js > searchGoogle
AGENT-TALK-2025
node_modules
src
_steps: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js
tools: browseWeb.js, index.js, llmJudge.js, searchGoogle.js, todoList.js
utils: ai.js
.env, .env.sample, .gitignore, .nvmrc, LICENSE, package-lock.json, package.json
function searchGoogle({ query }: { query: any; }): Promise<string>
Chat ⌘L  Edit ⌘K
main  Ln 5, Col 45 (9 selected)  Spaces: 4  UTF-8  LF  JavaScript

### Adding to-dos

The addTodos tool pushes new tasks onto the to-do list, prints the list, and returns the number added and the new total. A separate array holds completed tasks.

[10:23](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=623s) [Editor shows arrays for pending and completed tasks and a tool for adding to-dos.](https://assets.ai.engineer/sites/aiecode2025/sha256/267b0da34eff48f679e1ec9e864913a4febae154d21cd0a3d71ebe29886d2fab.webp)

import { z } from "zod";
import { zodFunction } from "openai/helpers/zod";
import { todo } from "node:test";

let todos = []
const done = []

// [illegible divider]
// CREATE - Add New Todos
export function addTodos({newTodos}) {
    todos.push(...newTodos)
    const delim = '\n  - '
    console.log(`Todo list:${delim}${todos.join(delim)}`)
    return `Added ${newTodos.length} to todo list. Now have ${todos.length} todos.`
}

export const addTodosToolConfig = zodFunction({
    name: "addTodos",
    description: "Add an array of todos to my todo list.",
    parameters: z.object({
        newTodos: z.array(z.string()).describe("The array of new todos to add to my todo list.")
    })
})

// [illegible divider]
// Mark Todos Done
export function markTodoDone({todo}) {
    if (todos.includes(todo)) {
        todos = todos.filter(item => item !== todo)
        done.push(todo)
        return `Marked the following todo as done:\n  ${todo}`
    } else {
        return `Todo list doesn't include todo:\n  ${todo}`

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:46 PM
agent-talk-2025
step0-llm.js  step1-condition.js  step2-tool.js  step3-refactor.js  step4-planning.js  todoList.js
src > tools > todoList.js > addTodos
node_modules
src
_steps
step0-llm.js
step1-condition.js
step2-tool.js
step3-refactor.js
step4-planning.js
tools
browseWeb.js
index.js
llmJudge.js
searchGoogle.js
todoList.js
utils
ai.js
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
main
Ln 14, Col 16 (11 selected)  Spaces: 4  UTF-8  LF  JavaScript  Cursor Tab

### Marking to-dos done

The demo moves a matching to-do from the pending list to the done list. If it isn't found, the function returns a message; a separate function reads completed tasks.

[12:09](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=729s) [Editor shows code that moves a task to the done list, followed by its tool configuration.](https://assets.ai.engineer/sites/aiecode2025/sha256/a38f5ca6676184ccfb4bf375671eefcfb2618987baf0797642b68950cc73289c.webp)

// [illegible]
// Mark Todos Done
export function markTodoDone({todo}) {
    if (todos.includes(todo)) {
        todos = todos.filter(item => item !== todo)
        done.push(todo)
        return `Marked the following todo as done:\n  ${todo}`
    } else {
        return `Todo list doesn't include todo:\n  ${todo}`
    }
}
export const markTodoDoneToolConfig = zodFunction({
    name: "markTodoDone",
    description: "Mark an individual item on my todo list as done.",
    parameters: z.object({
        todo: z.string().describe("The array of new todos to add to my todo list.")
    })
})

// [illegible]
// Read the done list
export function checkDoneTodos({}) {
    if (done.length > 0) {
        return JSON.stringify(done)
    } else {
        return "No tasks have been marked done."
    }
}

export const checkDoneTasksToolConfig = zodFunction({
    name: "checkDoneTask",
    description: "Read every thing on the todo list that has been marked done.",
    parameters: z.object(z.tuple([])).describe("No arguments are needed for this function.")
})

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:48 PM
agent-talk-2025
Tabs: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js, todoList.js
src > tools > todoList.js > ...
AGENT-TALK-2025
node_modules
src
  _steps
    step0-llm.js
    step1-condition.js
    step2-tool.js
    step3-refactor.js
    step4-planning.js
  tools
    browseWeb.js
    index.js
    llmJudge.js
    searchGoogle.js
    todoList.js
  utils
    ai.js
.env
.env.sample
.gitignore
.nvmrc
LICENSE
package-lock.json
package.json
main  Ln 9, Col 37  Spaces: 4  UTF-8  LF  JavaScript  Cursor Tab

### Reading to-do lists

The demo defines separate tools to read completed and pending tasks. Each returns the list as JSON when it has items and a message when it is empty.

[12:31](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=751s) [Editor shows functions and tool configurations for reading completed and pending tasks.](https://assets.ai.engineer/sites/aiecode2025/sha256/d7994c4681ca74c230e8691460660239fd2c4e7e1cb8242bac742ebb23bef6e3.webp)

// [hash separator]
// Read the done list
export function checkDoneTodos({}) {
    if (done.length > 0) {
        return JSON.stringify(done)
    } else {
        return "No tasks have been marked done."
    }
}

export const checkDoneTasksToolConfig = zodFunction({
    name: "checkDoneTask",
    description: "Read every thing on the todo list that has been marked done.",
    parameters: z.object(z.tuple([])).describe("No arguments are needed for this function.")
})

// [hash separator]
// Read the todo list
export function checkTodos({}) {
    if (todos.length > 0) {
        return JSON.stringify(todos)
    } else {
        return "The todo list is empty."
    }
}

export const checkTodosToolConfig = zodFunction({
    name: "checkTodos",
    description: "Read everything on the todo list.",
    parameters: z.object(z.tuple([])).describe("No arguments are needed for this function.")
})

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:48 PM
agent-talk-2025
Tabs: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js, todoList.js
src > tools > todoList.js > ...
AGENT-TALK-2025
node_modules
src
_steps: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js
tools: browseWeb.js, index.js, llmJudge.js, searchGoogle.js, todoList.js
utils: ai.js
.env, .env.sample, .gitignore, .nvmrc, LICENSE, package-lock.json, package.json
main  Ln 9, Col 37  Spaces: 4  UTF-8  LF  JavaScript  Cursor Tab

[12:44](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=764s) [Code for reading the to-do list and configuring the corresponding tool.](https://assets.ai.engineer/sites/aiecode2025/sha256/6439d6e6dedc875584c2556fda6c04b42e6e60f0b27b8a6334692f681ef7ec0e.webp)

}

export const checkDoneTasksToolConfig = zodFunction({
    name: "checkDoneTask",
    description: "Read every thing on the todo list that has been marked done.",
    parameters: z.object(z.tuple([])).describe("No arguments are needed for this function.")
})

// [hash separator]
// Read the todo list
export function checkTodos({}) {
    if (todos.length > 0) {
        return JSON.stringify(todos)
    } else {
        return "The todo list is empty."
    }
}

export const checkTodosToolConfig = zodFunction({
    name: "checkTodos",
    description: "Read everything on the todo list.",
    parameters: z.object(z.tuple([])).describe("No arguments are needed for this function.")
})

Interface text:
Cursor  File  Edit  Selection  View  Go  Run  Terminal  Window  Help
Fri Feb 7 9:48 PM
agent-talk-2025
Tabs: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js, todoList.js
src > tools > todoList.js > ...
AGENT-TALK-2025
node_modules
src
_steps: step0-llm.js, step1-condition.js, step2-tool.js, step3-refactor.js, step4-planning.js
tools: browseWeb.js, index.js, llmJudge.js, searchGoogle.js, todoList.js
utils: ai.js
.env, .env.sample, .gitignore, .nvmrc, LICENSE, package-lock.json, package.json
main  Ln 9, Col 37  Spaces: 4  UTF-8  LF  JavaScript  Cursor Tab

## Full transcript

[0:01](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1s) Hi, I'm Kam, and I welcome you to my talk on how to build an agent. So I have three goals for this talk. I want you to experience the simplest version of what an agent could be.

[0:15](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=15s) Um, and I want you to feel comfortable in running and breaking the included code. Um, I think the, the running and breaking is a very, very important part to the learning.

[0:26](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=26s) Um, and I also want you to take away, uh, an intuition for how agents work or how other frameworks work. If you choose to go with some agent framework, that's great.

[0:38](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=38s) Um, I want you to have some understanding of maybe the underlying, the underpinning building blocks that they used to make it work. So with that, let's get to some slides.

[0:48](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=48s) Okay, great. This is where you can find me on LinkedIn. This is where you can find the slides for this talk and the code. I really encourage you to go and grab the code, try it, run it, break it, see where this code begins to turn from deterministic outcomes to a feeling that you're starting to, starting to

[1:09](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=69s) play with an agent. So what is an agent? Uh, I really like both of these definitions. Uh, agent, LLM, memory, planning, tools, and a while loop. Um, so let's break that down a little bit more.

[1:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=79s) So mathematically, we could, uh, take memory and say, "Oh, actually, this is a, both a read and a write operation," and also on the while loop, um, it's really a conditional and a looping.

[1:30](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=90s) So we can, uh, break those apart, reorder them a little bit, and come up with the plan for what we're gonna do. And so with all of that out of the way, let's jump in and get to the code finally.

[1:41](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=101s) Alrighty, so let's jump right in. So s- the first step here is, uh, calling an LLM. This is straight from the OpenAI, uh, Hello World docs, right? Uh, standard chat completion, and we can run it here.

[1:58](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=118s) Step zero. Okay. Um, this should be pretty standard for most people so far. Okay, great. Um, then let's jump to the step one, the condition. So, um, here we're gonna have the same completion call, um, checking for the prompt, but then once we get the answer back, we're gonna make another

[2:23](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=143s) LLM call that's gonna work as an LLM as judge, right? We are gonna ask, "You're a strict critic. Given the following question, determine if the answer is a full answer to the question."

[2:35](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=155s) Question, answer. Okay. Um, and then we give it a forced, um, JSON strict response of, um, whether it's done or not. And you can see the object here is whether it's done, it's gonna be a Boolean.

[2:50](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=170s) And the original question is, what is the average wing speed of a swallow? So we can go in, and we can run step one to see how it works with a condition.

[3:03](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=183s) So, um, it gives me LLM as judge, gives me a thumbs up. That's great. [clears throat] Here's the answer coming back from the LLM. So, so far, very deterministic, very mechanistic in its, uh, outputs, and then the judgment, and then the response to the, to the user.

[3:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=199s) So, um, the next step here is tools. Um, and we are incorporating SerpApi. I think I'm pronouncing that correctly. This is a Google Search API, uh, service. Um, and they basically make Google Search API, uh, easy JSON, um, API versus dealing with whatever the, the Google API nonsense is. [clears throat]

[3:43](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=223s) So I'm asking about buying a hoodie in New York and, um, near Times Square and the like. Again, gonna have the same conditional. I'm gonna now have this searchGoogle tool, right?

[3:55](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=235s) And you can see the Serp, um, getJson call here. Engine is Google, passing in the API key with a query, um, and a location. I'm defaulting to Philadelphia from the location of where to start the query from. [clears throat]

[4:13](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=253s) And then I'm gonna print the results. Um, the tool config going into OpenAI looks like this, and so I'm, I'm hand writing out this JSON so that we can inspect it a little bit.

[4:25](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=265s) Um, we're defining the name of the function we're gonna call, giving it descriptions, um, strict, whether it, uh, is strictly adheres to the, the schema here, um, and then the parameters into the tool call.

[4:38](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=278s) So there's a query and a location, um, and which elements are required, and if there's any additional ones. So this is the JSON that gets generated and passed in, um, when we make our call.

[4:50](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=290s) Um, the other thing to note with tool calls

[4:54](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=294s) is that we call the LLM, the tool call kicks back and tells our local code what tool to call or which tools to call, and what, uh, parameters are included in there.

[5:06](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=306s) So we need to handle that. That's not handled by the OpenAI SDK, um, and we need to look into the tool call. If there are tool calls, in this case, we're only doing a single one.

[5:16](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=316s) We'll get to parallel, uh, in a few minutes. Um, and then we go, and we make that tool call, um, off of, uh, an array here on tools, which includes the searchGoogle, um, passing in the args.

[5:32](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=332s) And then we, uh, push those, both the tool call and the response from the tool call back to the conversation, and then we can, um, completeWithTools again, meaning in this case, we are recursing through and calling, uh, back to the LLM with those responses passed in as those args, which it'll then make another determination.

[5:54](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=354s) So, um, so main loop here. Um, completeWithTools, the prompt searchGoogle tool config, and then, um, it re-responds back, and then it'll see, it'll internally loop on tool calls until it is satisfied and the LLM is satisfied, and then result come back out.

[6:15](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=375s) And then here, the critic will come and perform, uh, condition again. So let us go here. Let us run step two on the tool.

[6:31](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=391s) Unhandled rejection. Uh-oh. Okay, it looks like we failed there due to, um, validation constraint on the location parameter.

[6:46](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=406s) It doesn't take just any string, it takes a specific set of strings. Um, in this case, the LLM-as-judge gave us a thumbs up again. Um, again, this feels very deterministic.

[6:57](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=417s) We are, um, asking an LLM to determine that it needs to call a search API, and then we're asking the same LLM to go and evaluate whether that search API returned text that was a reasonable answer to where I could buy a fur-lined hoodie near Times Square.

[7:16](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=436s) So, um, of course, still very mechanistic, very, um, deterministic, very, um, straightforward in its, in its outcomes. Um, okay, but then let's take the next step. Let's go to step three.

[7:30](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=450s) And in this case, um, a little refactoring is in order. Um, so same prompt around, um, wanting to buy a, a hoodie, fur-lined hoodie. Um, we've pushed the completeWithTools into a other, into a utils folder, so into another file, um, just to get it out of that main.

[7:52](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=472s) This still has that same looping, but in this case, we've written this so that it can do the parallel tool calling. Um, these tool calls could come back with multiple, right?

[8:03](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=483s) This could be an array of tool calls that the LLM is asking the, the, us to perform, the client to perform. And so we're, uh, Promise.all-ing over the tool calls, and we are, um, passing in the function arguments for each of the tool calls, um, into the tool functions, um, here.

[8:27](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=507s) And then we are then pushing the response from this local function that we have called back into, uh, the conversation. And this will have a tool call ID so that the LLM is able to trace, um, I asked for this tool to be called, and here's the ID, and then here's the response of that tool call.

[8:47](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=527s) So sometimes it asks for the same tool to be called multiple times with different input parameters, right? So that's how that works. Um, the tools, um, are configured. So for example, in the searchGoogle example,

[9:02](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=542s) we're passing in this query, um, as an object, and it's performing the query, and it's returning a string result. Um, in my mental model, uh, the tools are, are best handled by thinking of them as text transformations.

[9:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=559s) You might have a couple of different parameters going in, but usually string to string is sort of the core of what, uh, these tools are performing. Um, so that was the AI and the, the completions, uh, the, sorry, the completeWithTools on the refactor.

[9:38](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=578s) Um, and so that is here. And then we are outputting the result, um, and then we are running the same LLM as judge. So we can run that again.

[9:51](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=591s) That'll give us the step three. Um, that'll give us some additional looping and parallel tool calling.

[10:08](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=608s) All righty. LLM as judge, thumbs up again. Okay, great. Here are a couple of those sites. Again, though, still feels very deterministic. We just reshuffled things around. Shouldn't have been too big of a difference.

[10:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=619s) This is where we are gonna really feel an inflection here. So, um, the biggest change on step four for planning is creating a to-do list, and this now solves both the read, the write, and the planning aspect.

[10:36](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=636s) Um, the while looping aspect that we talked about is covered by the LLM itself, right? In making a tool call and then having us reply back with the result of that tool call, the LLM is able to keep iterating and keep, um, operating on the code that we are, or the prompt that we gave it, and keep

[11:01](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=661s) working through towards a solution. Um, there are cases when that LLM can just iterate forever, calling tools and never converging, as particularly if we keep pruning the context window and we never hit some error or boundary condition.

[11:18](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=678s) Um, some guardrails I've seen get put on are the number of, uh, iterative loops that they're able to go through before, um, [chuckles] client code kinda cuts them off and says like, "Okay, LLM, uh, you're drunk.

[11:31](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=691s) Go home." Um, so in this case though, we don't have that set up, um, but we have added this to-do list, which has what you would expect from a standard hello world to-do list, right?

[11:45](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=705s) You can add things to your to-do list, and in this case, we have the, also the tool config for the to-do list of adding new to-dos. That's an array of to-dos, right?

[11:54](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=714s) There'll be an array of to-dos, and we'll have array of done. Um, so you can add new to-dos, and they get pushed onto the to-do list. And then we're just printing it out, and then we're returning the to-dos that were added to the to-do list.

[12:08](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=728s) Um. Then we can also mark to-dos as done. Um, and so if a to-do is included in this to-do list,

[12:18](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=738s) um, mark it as done and pull it out of the, um, to-dos that are there, um, so that we don't have to do it again. Um, as well as the config for that.

[12:31](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=751s) Um, and then the check done. We can see if, uh, we have completed all of the to-dos. Um, if the length is zero, then it'll say, "No tasks have been marked done."

[12:45](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=765s) Um, and then the config for that as well, and then check to-dos. So it's able to get all the to-dos that are on the to-do list. So again, sort of standard to-do list.

[12:56](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=776s) Um, and now with all of those tools, the, the LLM-as-judge, the searchGoogle, and I guess I should cover this, this browseWeb. Very similar to the searchGoogle, right?

[13:09](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=789s) It takes in a URL. Um, it's using Cheerio and Turndown, and we are, uh, requesting that URL here. Um, and if the response comes back, uh, 200, then it's just taking the text

[13:24](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=804s) using Turndown to turn it into markdown and returning that markdown, um, out of the tool. So very simple operation given that URL, right? So given that, the planning will then take the to-do list that is generated based on the prompt, right?

[13:42](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=822s) So this is where the programming of the agent begins to take over, right? You're a helpful assistant working for a busy executive. Your tone is friendly but direct. They prefer short, clear, and direct writing.

[13:53](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=833s) You try to accomplish the specific task you're given. You can use any of the tools available to you. Before you do any of your work, you always make a plan to use your to-do list, right?

[14:02](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=842s) So that's driving the planning towards the to-do list. Uh, you can mark to-dos off of your to-do list after they've been complete. You summarize the actions you took by checking the to-do list, then create a report.

[14:13](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=853s) You always ask your assistant to check goal done. That drives towards LLM-as-judge. If you say you are done, you send the report to the user. If your assistant has feedback, you add it to your to-do list.

[14:24](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=864s) And then I've added today's date because sometimes the LLM doesn't quite know when in time it is, [chuckles] which is quite amusing. And in this case, since we've pushed LLM-as-judge as a tool, right, the condition can be a tool itself.

[14:38](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=878s) The loop is handled by the LLM itself. Then this main function is just a single call to this complete with the tools, and then we're gonna respond with our answer.

[14:48](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=888s) So, um, oh, and the, the default here is I want to learn about building agents without a framework. So let us try that. We can npm run step four and see how it goes.

[15:03](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=903s) I want to learn about building agents without a framework. Okay, it's calling the add to-dos. So here it came up with a plan. These are the new to-dos, right?

[15:12](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=912s) Um, search for information about building agents without a framework. Summarize key points from the search results, and then he asks the assistant to check if the expl-explanation is sufficient.

[15:21](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=921s) Um, it's calling check to-dos, calling searchGoogle about this. It's browsing this blog post. It's browsing this Pond data, browsing the web, browsing the web. Here's a bunch of markdown from that website.

[15:39](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=939s) Um, it's marking done, summarizing key points from the search result. It's checking to-dos. It's checking goal done.

[15:50](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=950s) It's giving it, uh, LLM-as-judge gives it a thumbs up. Okay, great. Thumbs up. We're, we are done. Oh, so here is a summary of building an agent without using a framework.

[15:59](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=959s) To build an agent without a framework, follow these steps. Understand co-core components. Recognize that an AI agent is a language model capable of tool use and maintaining conversational context.

[16:08](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=968s) Define tools. Tools are functions for environmental interaction, um, like database queries, web search. Okay. That's tools. That's memory. Um, a loop, uh, input processing, tool decision, execution, response formulation, prompt engineering, direct API calls.

[16:27](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=987s) Uh, this is a good one instead of trying to pass through the LLM, but just directly store state and call it for yourself. Um, so I feel like this is pretty decent.

[16:35](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=995s) Um, you know, where I've started to see some, um, real interest in this LLM, uh, or this agent that I built is, uh, doing some things like, uh, I am planning a date with my wife this Saturday,

[16:57](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1017s) um, near [REDACTED:location]. Please help, help me find

[17:11](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1031s) some activities and dinner between 6 PM and 11 PM.

[17:23](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1043s) Um, let's see how it does. Um, so searching for local activities. Here's the plan again. Finding a good dinner option.

[17:35](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1055s) Um, searching Google. Best restaurants. Browsing the web.

[17:44](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1064s) Lots of markdown. Marking things as done. Checking the goal.

[17:54](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1074s) Okay. Here are some of the activities. Here are the, some of the dinner options, and hopes that I have a lovely evening. So it's not that this, uh, this agent is perfect by any means, but hopefully, you can see how adding each of these components from the LLM call, the conditional, the tool use,

[18:19](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1099s) um, adding some planning, some read/write in that to-do list, and how we can really leverage some of those things coming together and start to produce something that's, that's quite interesting.

[18:30](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1110s) You could see how next steps could be adding a vector database or injecting these, um, browsed, um, web pages into, um, you know, a, a Chroma DB in memory or into something, into a more RAG-like system.

[18:46](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1126s) So hopefully, this was really helpful, and, uh, hopefully, this gives you some interest or excitement in how to dive in and, uh,

[18:56](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1136s) do this yourself. Okay, great. Thanks for coming to my talk. Uh, again, I'm Kam Lasater. You can find me on LinkedIn here. The slides are up on my personal site, and the code is up on GitHub.

[19:09](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1149s) I encourage you to take it for a spin. Uh, let me know what you think. Let me know if, uh, there's some improvements you could make or if, uh, something like this is exciting.

[19:18](https://www.youtube.com/watch?v=xzXdLRUyjUg&t=1158s) We're always, uh, building and looking for people who are interested to build, uh, agents for, for our customers. So again, see you online. Cheers.
