Let's Build an Agent from Scratch — Kam Lasater

What is the smallest agent you can build well enough to run, break, and understand? That is the starting problem in Kam Lasater’s tutorial. The aim is to expose the machinery beneath an agent framework, so that using a framework becomes an informed choice. The companion code provides successive versions to experiment with as the behavior moves beyond a single request and response.

Start with an LLM, memory, planning, tools, and a while loop. Then split memory into read and write operations, and split the while loop into a condition and iteration. This decomposition gives the build its sequence: call the model, judge an answer, execute tools, support repeated calls, and finally add a plan that the agent can read and update. None of these components requires a framework to make its role visible.

Step zero is an ordinary OpenAI Chat Completions call. Step one adds a second model call: a strict critic receives the original question and the proposed answer, then decides whether the answer fully addresses the question. Its response must follow a strict JSON schema containing a Boolean done field. This article retains the recording’s Chat Completions approach; current API documentation may use different request shapes.

The question is the average wing speed of a swallow. After the first model answers, the judge returns a thumbs-up. The application now has an answer and a machine-readable completion decision, but those are separate outputs with separate failure modes. A schema-valid Boolean is not proof that the answer is correct. Lasater describes this stage as deterministic and mechanistic: the workflow is fixed—generate, judge, respond—even though the model’s content is not guaranteed to be identical on each run.

Step two changes the task to finding a fur-lined hoodie near Times Square in New York. To answer with outside information, the agent gets a searchGoogle tool backed by SerpApi. Its getJson call supplies the Google engine, an API key, a query, and a location. The implementation defaults the search location to Philadelphia, which is distinct from the location requested in the shopping task.

The tool has two interfaces. The local function performs the search and prints results. A handwritten tool configuration tells the model the function’s name, description, strictness, and parameters. It describes query and location, identifies required fields, and specifies whether additional properties are permitted. That schema is how the model learns what it may request; it does not execute the search.

The application executes the tool; the model requests it. The initial implementation handles one requested call at a time:

1. Call the model with the conversation and tool configuration.
2. Inspect the returned tool call for its function name and arguments.
3. Dispatch the request to the local searchGoogle function through the tools collection.
4. Append both the assistant’s tool-call message and the tool’s response to the conversation.
5. Call completeWithTools again with the expanded conversation.

The OpenAI SDK does not automatically run these local functions. The recursive call gives the model the search result and another opportunity to choose what happens next.

Once the model stops requesting tools and produces an answer, completeWithTools returns it to the main flow. The external critic then checks that answer, just as it did for the swallow question. There are now two distinct decisions: whether another tool call is needed, and whether the resulting answer satisfies the user’s request.

The live search run produces an unhandled rejection. Lasater attributes it to a validation constraint on location: the value must come from a particular set of accepted strings, rather than being any arbitrary string. A parameter can therefore look reasonable to the model and still fail the service’s requirements.

Lasater also reports another judge thumbs-up, without explaining how that approval relates to the failed request. The approval cannot establish that the search succeeded. At this stage, the same LLM chooses to search and evaluates whether the returned text reasonably answers the hoodie question; the surrounding workflow still feels straightforward and mechanistic to him.

Step three keeps the hoodie request but moves completeWithTools into a utility file. The helper now accepts an array of requested tool calls and executes them with Promise.all, passing each call’s arguments to the corresponding local function. Each result carries a tool call ID so the model can associate it with the request that produced it.

Function names alone are insufficient: the model may request searchGoogle several times with different parameters. In TypeScript, the dispatch portion can keep each request’s identity attached to its result like this:

type ToolCall = {
  id: string;
  function: { name: string; arguments: string };
};

type Tool = (args: Record<string, unknown>) => Promise<string>;

type ToolResult = {
  role: "tool";
  tool_call_id: string;
  content: string;
};

async function executeToolCalls(
  calls: ToolCall[],
  tools: Record<string, Tool>,
): Promise<ToolResult[]> {
  return Promise.all(
    calls.map(async (call): Promise<ToolResult> => {
      const tool = tools[call.function.name];
      if (!tool) {
        throw new Error(`Unknown tool: ${call.function.name}`);
      }
      const args = JSON.parse(call.function.arguments);
      const content = await tool(args);
      return {
        role: "tool",
        tool_call_id: call.id,
        content,
      };
    }),
  );
}

The surrounding helper appends these results after the assistant message that requested the calls, then asks the model to continue. The ID preserves the relationship even when the same function appears more than once.

Lasater’s preferred mental model for a tool is a text transformation. searchGoogle accepts a query object and returns a string; even when inputs have several fields, the model-facing interaction is largely text in and text out. The refactored run returns sites and another judge approval. Parallel dispatch improves the machinery, but it has not yet produced the behavioral change he is looking for.

Step four adds a to-do list, the point Lasater identifies as the behavioral inflection. The same mechanism now supports planning, reading state, and writing state. Iteration still comes from the exchange between model and application: the model requests another tool, application code executes it and sends back the result, and a subsequent model call decides how to proceed.

That exchange does not guarantee convergence. A model can keep requesting tools indefinitely, especially if context pruning prevents it from reaching a context boundary or error. Lasater describes client-enforced iteration limits as a possible guardrail, but the demonstration does not implement one. Moving the next-action decision into the model does not remove the application’s responsibility for bounding execution.

The storage is deliberately ordinary: one array holds pending to-dos, and another holds completed items. The add operation accepts an array of tasks, pushes them into pending work, prints them, and returns the tasks that were added. It also has a tool configuration so the model can request the operation.

The remaining operations make that state usable across successive model calls:

Each operation has a corresponding tool configuration. Removing completed tasks from the pending list helps prevent repeated work. Inspecting the completed list is also distinct from deciding whether the user’s overall goal has been satisfied.

The available tools now include the judge, search, the to-do operations, and browseWeb. The browsing tool takes a URL, retrieves its content, and uses Cheerio and Turndown to return Markdown after an HTTP 200 response. This is page retrieval and conversion, not a full browser that renders a page and executes its JavaScript. The result is text the model can use in its next decision.

The prompt then supplies a procedure for using those capabilities. It casts the model as an assistant to a busy executive, with friendly, short, clear, direct writing. More consequentially, it tells the model how to organize the work:

1. Make a plan in the to-do list before doing any work.
2. Use available tools to carry out the task.
3. Mark tasks complete after doing them.
4. Check the list and create a report of the actions taken.
5. Ask the assistant to check whether the goal is done.
6. Send the report if approved; otherwise, add the feedback to the to-do list.

The list stores the work; the prompt establishes when and why to update it.

Lasater also includes the current date to orient the model in time. The judge has moved from a fixed call after the main workflow into the available tool set, so the completion condition itself becomes something the model is instructed to invoke. The main function can now make one call to completeWithTools and return its final answer. That compact entry point still wraps multiple model requests and local tool executions.

The default step-four request is to learn about building agents without a framework. Lasater runs the step through npm, and the model first calls the add-to-dos tool. Its generated plan has three parts: search for information, summarize the key points, and ask the assistant whether the explanation is sufficient. The model is now choosing intermediate work and recording it before producing the final answer.

The run checks the to-do list, calls searchGoogle, and browses multiple pages. Markdown comes back into the conversation. The agent then marks work done, checks its to-dos, and requests the goal check. The judge’s thumbs-up precedes the final report. This sequence makes the planning cycle visible through ordinary tool calls and state changes.

The resulting report describes an agent as a language model with tool use and conversational context. It identifies tools such as database queries and web search, then outlines input processing, tool decisions, execution, response formulation, prompt engineering, and direct API calls. Lasater particularly endorses the suggestion to store state directly and access it in code instead of passing everything through the LLM. The to-do arrays already embody that choice: the model requests changes, while local code owns the stored state.

For a broader test, Lasater asks for help planning a Saturday date with his wife, finding local activities and dinner between 6 PM and 11 PM. The generated plan separates finding local activities from finding a dinner option. It searches Google for restaurants, browses pages, receives Markdown, marks work done, and checks the goal.

The final response offers activity and dinner options. Lasater does not present the agent as perfect; what interests him is how the components now work together. A model call and a condition established the initial workflow. Tools connected it to outside information. Planning and the to-do list’s read/write operations gave it a way to organize and track intermediate work on a less narrowly specified request.

A possible next layer would retain the browsed pages in a vector database—perhaps an in-memory Chroma database—or use them in a retrieval-augmented system. These are proposed extensions, not capabilities demonstrated in the run. They would build on an existing path from URL to page text, adding a way to retrieve that information later.

The exercise ends with an invitation to run the code, break it, and suggest improvements. Lasater points readers to the slides on his personal site, the code on GitHub, and LinkedIn for contact. Having watched each layer appear, the useful experiment is to change one of them and observe how the agent’s behavior changes: the implementation is small enough that the relationship between a tool, a prompt instruction, and the resulting work remains visible.