# The A to Z of Building AI Agents

Apoorva Joshi, MongoDB · Ben Perlmutter, MongoDB

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/IRp7lvBlbHs-z-building-ai-agents) · [Video](https://www.youtube.com/watch?v=IRp7lvBlbHs)

## The A to Z of Building AI Agents

Build a research assistant by choosing where an agent is useful, connecting paper-search tools, composing an execution loop, and persisting conversation history in MongoDB.

Basic Python familiarity, comfort with Jupyter notebooks, and an understanding of LLM prompts will help you follow the implementation sections.

### What does an agent add to an LLM?

[0:21](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=21s)

When does an LLM application need an agent, and what do you actually have to build? This workshop starts with that decision, then turns the components into a working research-assistant exercise. The format is a conceptual introduction followed by hands-on work, with Tom, Ben, and Fabian helping participants troubleshoot. Apoorva Joshi introduces herself as a MongoDB developer advocate, five months into the role after roughly six years applying data science to cybersecurity problems such as phishing, malware, and ransomware detection. Questions and group work are encouraged throughout.

The slides and [AI Agents Lab](https://github.com/mongodb-developer/ai-agents-lab) are the companion materials. In the room, participants access them through the displayed link and QR code, also printed on postcards.

![Lab Content slide displaying mdb.link/agents-workshop-aiewf above a large QR code, with the presenter inset at lower left.](https://ai.engineer/api/talk-essay-media/8f149d62a8838e92c532adacadfb0480392a3e68b839d9906d53139afb3e3250)

Workshop lab content link and QR code.

The end-to-end deliverable is an AI research agent. Its defining capability is straightforward: **an LLM reasons about a problem, creates a plan, and executes that plan through tools**. The workshop develops those pieces before the exercises; a separate Q&A depends on the time remaining.

### Choose prompting, retrieval, or an action loop

[3:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=202s)

Simple prompting asks the model to answer from its pretrained, parametric knowledge. In that setup, even a carefully worded request cannot give it missing information or a way to execute an external task. A single response also provides no persistent mechanism for revising an answer or remembering a user's preferences. Retrieval-augmented generation, or **RAG**, adds information from a knowledge base, but retrieval alone does not supply an iterative execution or personalization loop.

An agent adds tools, access to past interactions, and repeated opportunities to decide what to do next. A tool result becomes input to the next model call: the agent can repeat a step, choose a follow-up tool, or finish. Persisted interactions can inform later responses. These capabilities introduce costs, too. Long-term planning remains difficult, and repeated calls add latency and expense in pursuit of better accuracy. Joshi regards agents as a way to get more from the models available at the time, rather than a reason to turn every question into an agent workflow.

The first audience exercise asks who was the first president of the United States. Joshi recommends simple prompting because most LLMs likely already contain that information. The next asks for a company's travel reimbursement policy. An audience member points out that the application must first establish the employee's location, because that determines which policy applies. Joshi acknowledges the qualification and still recommends RAG once the relevant context and knowledge base are available. **A clarification step does not automatically require an autonomous agent.**

The next request combines several capabilities: determine how average daily calorie intake among adults has changed over a decade, consider its possible impact on obesity rates, and graph the trend. Answering it requires data aggregation, visualization, and interpretation of the results. That makes it a useful agent task. A personalized learning assistant offers a different reason to use an agent: it must adjust its language, examples, and teaching methods as it observes a student's responses over time.

| Requirement | Suitable starting point |
|---|---|
| Answer a stable general-knowledge question | Prompt the model |
| Answer from company documents | Retrieve the relevant information |
| Combine research, analysis, and execution | Use tools in an agent loop |
| Adapt across interactions | Add persistent, relevant memory |

The useful boundary is the work the system must perform: combining question answering, task execution, and analysis toward an outcome, or adapting its responses over time.

### Plan, act, observe, and revise

[8:53](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=533s)

Planning begins with prompting. The simplest approach asks the model to create a plan from its initial understanding and then follow it without revising the plan in response to new tool results. The workshop groups chain-of-thought and [Tree of Thoughts](https://arxiv.org/abs/2305.10601) under planning without feedback, and [ReAct](https://arxiv.org/abs/2210.03629) and reflection under planning with feedback. Here, the distinction concerns feedback from external actions: Tree of Thoughts still evaluates and revises its own search choices.

**Chain-of-thought** prompts the model to work through a problem step by step instead of immediately producing an answer. A zero-shot prompt requests that behavior directly; a few-shot prompt provides one or more worked examples. **Tree of Thoughts** expands the process into multiple candidate reasoning paths. The model generates and evaluates intermediate choices, while a search procedure supports lookahead and backtracking. That allows it to reconsider a branch rather than commit to the first path it generates.

**ReAct** brings external observations into the loop:

1. The model identifies the next step toward solving the problem.
2. It selects an action and supplies its inputs.
3. The action runs, producing information the model can observe.
4. The model uses that observation to choose another action or finish.

The loop ends when the model determines it has reached the answer, or a user intervenes. In the workshop example, the model selects a search tool and its arguments, examines the result, recognizes that the answer is still incomplete, and chooses a further step. The observation changes what happens next.

**Reflection** adds critique of past actions or generated answers before the next attempt. The same model can generate and critique, different models can take those roles, or separate agents can divide the work. In each arrangement, the generation–reflection cycle may repeat before producing a final answer. The tradeoff is additional computation for a chance at greater accuracy.

### Decide what the agent should remember

[12:43](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=763s)

Memory lets an agent store and recall interactions so that later responses can use earlier information. The workshop separates two scopes:

- **Short-term memory:** information within a single conversation.
- **Long-term memory:** information stored, updated, and retrieved across conversations, supporting personalization over time.

The distinction is about the scope of the remembered information, not simply whether it is stored in a database.

A short conversation can be retained as a message list. As it grows, passing the entire list to the model becomes less practical. Two options are to retrieve only the `n` most recent messages or summarize the conversation. Keeping recent messages drops older context; summarization condenses that context at the cost of some information loss.

Long-term memory demands more application design: which states matter, how should they be represented, and when should they change? Joshi describes this as a largely unexplored area at the time of the workshop and recommends narrowing the problem to an application-specific agent. A focused application has fewer relevant states to track and clearer rules for updating them.

### Give the model an explicit tool contract

[14:35](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=875s)

Tools are the agent's interfaces to the external world. They can expose search or weather APIs, vector stores, or specialized machine-learning models. They are commonly represented as functions: the model selects a function and supplies arguments, and application code performs the call. **Selecting an action and executing it are separate responsibilities.**

LangChain can handle the calling machinery, but the model still needs a useful contract. Give each tool a descriptive name, connect it to the function that performs the work, describe what it does, and specify its argument types. The slide also includes whether the result should return directly to the user. These details help the model distinguish available capabilities and produce an appropriate request.

![Tools slide showing a short Python function beside five items: tool name, description, argument schema, function to call, and whether to return the result directly to the user.](https://ai.engineer/api/talk-essay-media/6b56fefd994a57e467f290d2a371a8fa49edc1b0270f74e0d35b595510f4f491)

A tool function and the information that describes its contract.

### Assemble the research assistant

[15:46](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=946s)

The research assistant has three objectives: suggest papers to read, summarize papers, and answer questions about research topics. Its model is [**FireFunction v1**](https://fireworks.ai/blog/firefunction-v1-gpt-4-level-function-calling) from Fireworks. [**FireFunction v2**](https://fireworks.ai/blog/firefunction-v2-launch-post) had just appeared, but the workshop retains v1 because the materials were prepared before that release. The workshop's description of v1 as free and open source is historical: Fireworks released open weights and offered free hosted inference during a limited beta, not permanently free service. The exercises explore chain-of-thought and ReAct with this model.

| Component | Responsibility |
|---|---|
| Paper-summary tool | Summarize a research paper |
| Reading-list tool | Return relevant papers |
| Knowledge-base tool | Answer questions using MongoDB-backed retrieval |
| MongoDB chat storage | Persist short-term conversation memory |

These are distinct capabilities that the agent can combine as the request requires.

The notebook gives the hands-on work a consistent structure:

1. Complete the open-hands exercises first; attempt superhero exercises if time permits.
2. Fill in the Jupyter Notebook's `code_block` placeholders.
3. Consult the documentation indicated by the books emoji before each relevant cell.
4. Try the exercise before opening the supplied solution, then make sure you understand the solution's behavior.

The first hands-on interval allocates 15–20 minutes to prerequisites, beginning with the lab's MongoDB Atlas section and continuing through Dev Environment.

### Load papers and return the right representation

[18:56](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1136s)

The lab remains self-paced and accessible after the workshop. Its next stage begins with two ways to obtain paper data. Hugging Face's Datasets library supplies `load_dataset` for downloading the arXiv embeddings dataset used to populate the knowledge base. LangChain's `ArxivLoader` loads papers directly from arXiv into document objects. Each document separates its raw text in `page_content` from extracted fields in `metadata`, including publication date, title, authors, and summary.

The loader serves both the prebuilt reading-list tool and the paper-summary tool. The `@tool` decorator turns a Python function into a LangChain tool. For `ArxivLoader`, `query` accepts a topic or paper ID, `load_max_docs` limits how many documents to download, and `.load()` produces the document objects. A reading list returns their metadata rather than their full text:

```python
from langchain_community.document_loaders import ArxivLoader
from langchain_core.tools import tool

@tool
def get_relevant_papers(query: str, max_papers: int = 3) -> list[dict]:
    """Find arXiv papers by topic or paper ID and return their metadata."""
    documents = ArxivLoader(
        query=query,
        load_max_docs=max_papers,
    ).load()
    return [document.metadata for document in documents]
```

This keeps the tool's output aligned with its purpose: selecting papers to read. The summary tool can instead use the loaded paper content.

PyMongo handles direct database operations: connecting to databases and collections, deleting documents, and inserting documents to build the research knowledge base. The workshop then uses separate LangChain provider packages to keep integrations independently versioned and easier to manage and test.

| Integration | Role in the agent |
|---|---|
| LangChain MongoDB integration | Atlas vector storage and chat-history storage/retrieval |
| `langchain-huggingface` | Open-source embedding models |
| `langchain-fireworks` | Fireworks chat-completion models |

Retrieval, model inference, and conversation storage are separate responsibilities even when they participate in one agent workflow.

### Compose workflows with runnables

[21:45](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1305s)

**LangChain Expression Language**, or LCEL, connects prompts, processing steps, models, and tools declaratively. Each unit is a runnable. The pipe operator passes the output of the runnable on its left into the runnable on its right. A basic model chain therefore has the shape `prompt | llm | output_parser`; calling `.invoke(...)` runs it and returns the result. This is the same composition mechanism used to build and test the workshop's RAG and agent workflows.

`RunnableLambda` brings an ordinary Python function into that composition model. For example, the paper-metadata transformation can become one runnable, followed by a title-extraction step:

```python
from langchain_core.runnables import RunnableLambda


def extract_metadata(documents):
    return [document.metadata for document in documents]


def extract_titles(records):
    return [record["Title"] for record in records]


def paper_titles(documents):
    chain = (
        RunnableLambda(extract_metadata)
        | RunnableLambda(extract_titles)
    )
    return chain.invoke(documents)
```

The first function's returned list becomes the second function's input. The workshop allocates the next 20-minute exercise interval to Create Agent Tools.

### Separate agent decisions from execution

[23:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1381s)

With the tools in place, the next exercise builds the agent and revisits the reasoning patterns. The workshop uses LangChain's classic constructor-and-executor API. Start with `create_tool_calling_agent`, which assembles a runnable sequence containing a prompt template, a tool-aware LLM, and an output parser. The prompt includes an **agent scratchpad** placeholder for intermediate actions and observations. That scratchpad gives the next decision access to what has already happened during the task.

The alternative, `create_react_agent`, uses a ReAct prompt to guide successive reasoning and action steps. Its setup includes a stop sequence to control the model's generation and a parser for the ReAct-style structure: thought, action, action input, and observation. The two constructors share a broad structure, but differ in how the prompt and parser express the model's next action.

**`AgentExecutor` is the runtime that turns those decisions into work.** It calls the agent, executes the chosen action, returns the action's output to the agent, and repeats as necessary. The workshop's loop can be read as this procedure:

1. Ask the agent for its next action or final answer.
2. If it selects an action, execute the corresponding tool.
3. Feed the tool result back into the agent's intermediate state.
4. Continue until the agent finishes.

A constructor creates the decision-making sequence; the executor drives repeated decisions and tool calls. The next 20-minute exercise interval covers Create Agent and any unfinished earlier work.

### Persist the conversation by session

[25:33](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1533s)

The final addition is short-term memory through access to chat-message history. `RunnableWithMessageHistory` wraps another runnable and manages its conversation history. The workshop applies this pattern to the agent workflow built with the tool-calling or ReAct constructor.

The wrapper accepts a function that supplies persistent message history; MongoDB is the storage layer here. A **session ID**, passed alongside the input query or prompt, identifies which conversation's history to use. Reusing a session ID lets subsequent requests draw on that conversation instead of starting without its earlier exchanges. This is persistent short-term memory: the stored object is still the history of a particular conversation.

![RunnableWithMessageHistory slide with two bullets describing management of another Runnable's chat history and organization by session ID.](https://ai.engineer/api/talk-essay-media/0c07d4d995f21fb4eb821698ab6f80ee7acf80b12449e4ddebf686388fbdb905)

RunnableWithMessageHistory manages chat history by session ID.

The remaining exercise is to add that history layer and experiment with the research agent across turns, with help available for questions or blockers. The build ends with a model that can select research tools, a runtime that executes them and feeds back results, and a conversation history that survives beyond a single request.

## Resources

- [MongoDB AI Agents Lab](https://github.com/mongodb-developer/ai-agents-lab) — Source repository for the lab site, with agent-building exercises and instructions for running its documentation locally.
- [FireFunction v1 launch announcement](https://fireworks.ai/blog/firefunction-v1-gpt-4-level-function-calling) — Historical introduction to the workshop's Mixtral-based function-calling model and its limited-beta availability.
- [Firefunction-v2 launch announcement](https://fireworks.ai/blog/firefunction-v2-launch-post) — Introduces the successor released shortly before the workshop, including multi-turn conversation and parallel function calling.
- [MongoDB arXiv papers with embeddings](https://huggingface.co/datasets/MongoDB/subset_arxiv_papers_with_embeddings) — A public collection of arXiv paper metadata, abstracts, and embedding vectors for retrieval experiments.
- [Tree of Thoughts](https://arxiv.org/abs/2305.10601) — Explains deliberate reasoning through branching search, evaluation of intermediate thoughts, and backtracking.
- [ReAct](https://arxiv.org/abs/2210.03629) — Introduces a framework that interleaves reasoning with actions and incorporates observations from external environments.

## Slides

### Presenting Sponsor

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=5s) [Microsoft presenting sponsor slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/9f227e03d5413a3f145480f3998b2dc3d19988d231e026e1f23d34ccc7102bb5.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and Platinum Sponsors

AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=10s) [AWS innovation sponsor and MongoDB, Google Cloud, and Neo4j platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/97ca19d6f33fddbda91019b40e3ae319a4bdca0457055c7967ee5848a232dc8c.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### Hands-On Session

Apoorva sets ground rules for the hands-on session: ask questions, discuss actively, and be respectful. She also encourages participants to form groups for the exercises.

[1:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=85s) [Ground rules encourage questions, active discussion, and respect.](https://assets.ai.engineer/sites/aiecode2025/sha256/80ee1aba5af232a4d4fc7c3a69eb9fb8b921aec5bd1aecb23f5711ab2e29b61b.webp)

Hands-On Session

• There are no stupid questions
• Actively discuss
• Be respectful

AIE
aws

### Agenda

The workshop introduces AI agent concepts, use cases, and components before participants build a research agent together. Q&A depends on time.

[2:32](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=152s) [Agenda covering AI agents, use cases, components, building a research agent, and Q&A.](https://assets.ai.engineer/sites/aiecode2025/sha256/094673da75965a1f046d39fa18474b664a08ec746a0793f8e40985e2acec322a.webp)

Agenda

• What are AI agents?
• AI agent use cases
• Components of an AI agent
• Building an AI Research agent
• Q&A

AIE
aws

### What are AI Agents?

The talk begins by introducing AI agents.

[3:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=181s) [Section title: What are AI Agents?](https://assets.ai.engineer/sites/aiecode2025/sha256/8f7a11e9e8643c92258ce9539e83d2d417d7a4ff9a9c1fbcd8f024d43ce6496c.webp)

What are AI Agents?

AIE
aws

### What is an AI agent?

The speaker defines an AI agent as a system that uses an LLM to reason through a problem, make a plan, and execute it with tools.

[3:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=187s) [An AI agent uses an LLM to reason through a problem, create a plan, and execute it with tools.](https://assets.ai.engineer/sites/aiecode2025/sha256/867fef937d05b13b2c450ea3cd9cba06ae2199d7fe8936385591ac96ec46d1a3.webp)

What is an AI agent?

An AI agent is a system that uses an LLM to reason through a problem, create a plan to solve the problem, and execute the plan with the help of a set of tools

AIE
Microsoft
smol ai

### Simple prompting

A prompt produces an answer from the LLM’s pre-trained knowledge. The speaker notes that complex tasks may require information or tools the model lacks, and that this setup cannot revise responses or learn preferences over time.

[3:21](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=201s) [A prompt-to-LLM-to-answer diagram and a list of stated limitations of simple prompting.](https://assets.ai.engineer/sites/aiecode2025/sha256/cbcc4a2ce0412d65be52f940a6a47f36b2b54b9ab9b3598798387a709f4ca322.webp)

Simple prompting

Prompt → LLM → Answer

LLM answers based on its parametric knowledge

Lack of knowledge
Cannot handle complex, multi-step tasks
Cannot self-revise and refine responses
Lack of personalization

AIE
Microsoft
smol ai

[3:35](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=215s) [Diagram of a prompt passing through an LLM to an answer, alongside the slide’s stated limitations of simple prompting.](https://assets.ai.engineer/sites/aiecode2025/sha256/0d01c7f8b01437bd5b48106f008803b49d032fb824182e2e6f8d3f7097d6fc13.webp)

Simple prompting

Prompt → LLM → Answer

LLM answers based on its parametric knowledge

Lack of knowledge
Cannot handle complex, multi-step tasks
Cannot self-revise and refine responses
Lack of personalization

AIE
aws

### Retrieval Augmented Generation

RAG supplies retrieved knowledge-base context to the LLM. The slide says complex tasks, self-refinement and personalization remain limitations.

[4:15](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=255s) [Diagram of a knowledge base supplying context to an LLM, with listed benefits and limitations of retrieval-augmented generation.](https://assets.ai.engineer/sites/aiecode2025/sha256/24fbacf21d1480f7a87dc82bd4025b9fe0948a6e56430e9c0ccffb90191b28fe.webp)

Retrieval Augmented Generation

Knowledge base
Context
Prompt
LLM
Answer

LLM answers based on content retrieved from a knowledge base

LLM has access to the right info
Cannot handle complex, multi-step tasks
Cannot self-revise and refine responses
Lack of personalization

AIE
aws

### AI agents

Agents combine an LLM with tools and memory, iterating through reasoning and actions to tackle multi-step tasks. Long-term planning, cost and latency remain challenges.

[4:40](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=280s) [Agent diagram connecting an LLM with memory, tools, reasoning and action, alongside stated capabilities and challenges.](https://assets.ai.engineer/sites/aiecode2025/sha256/d4fb6de0b3baa57805cc0e3f487faafab33c4b298e4ed9793fcc5acad413000a.webp)

AI agents

LLM reasons through a problem, uses external tools and past interactions to refine its execution plan and arrive at an answer.

Can handle complex, multi-step queries
Can self-revise and refine responses
Can personalize responses over time
Long-term planning remains a challenge
Higher cost and latency

Diagram labels:
Memory
Tools
Prompt
LLM
Action
Answer
Reasoning

AIE
aws

### When to use agents: a factual question

The speaker says this question does not need an agent: the answer is likely already in most LLMs’ parametric knowledge.

[6:03](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=363s) [The slide asks who was the first US president and displays “NO” for whether an agent is needed.](https://assets.ai.engineer/sites/aiecode2025/sha256/52115a2ead96789e1f77bb865d53355430ca6c825858d0e4500a75619edc68e6.webp)

When to use agents?

Who was the first president of the United States?

NO

AIE
Microsoft
smol ai

### When to use agents: company policy

For a company travel policy, the speaker favors retrieval-augmented generation with access to the right knowledge base. An audience member notes that the answer may depend on the employee’s location.

[6:32](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=392s) [The slide asks for a company’s travel reimbursement policy without displaying an answer.](https://assets.ai.engineer/sites/aiecode2025/sha256/929f3efebeb6650636c7f0c8e80a62944159ab5cee4f7b099207801f7ee4c7b7.webp)

When to use agents?

What is the travel reimbursement policy for my company?

AIE

### When to use agents: calorie intake and obesity trends

The speaker considers this a good fit for an agent because answering the question involves data aggregation, visualization, and reasoning over the results.

[7:29](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=449s) [The slide asks about US adult calorie intake trends, their possible impact on obesity rates, and a graph of obesity trends.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a0d20e30ddf3310a26e9bed69f1b253be9009c2cc074e5dce19c581a34e9979.webp)

When to use agents?

How has the trend in the average daily calorie intake among adults changed over the last decade in the United States, and what impact might this have on obesity rates? Additionally, can you provide a graphical representation of the trend in obesity rates over this period?

AIE

[7:41](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=461s) [A question asks about US adult calorie intake, its possible impact on obesity, and a graph of obesity trends.](https://assets.ai.engineer/sites/aiecode2025/sha256/94f190a1e3e78f2be04e7671165a98c43b1985fe71972b2e051a8b3ab8a511ef.webp)

When to use agents?

How has the trend in the average daily calorie intake among adults changed over the last decade in the United States, and what impact might this have on obesity rates? Additionally, can you provide a graphical representation of the trend in obesity rates over this period?

AIE

### When to use agents: personalized learning

A learning assistant that adjusts its language, examples, and methods to student responses is a good fit for agents, the speaker argues, because it requires long-term personalization.

[8:06](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=486s) [A personalized learning assistant adapts its language, examples, and methods to student responses; the slide says yes.](https://assets.ai.engineer/sites/aiecode2025/sha256/44d71df4fa9a0975ad16264454fa4285bdbfe22364c27384b861d19c44d12a35.webp)

When to use agents?

Creating a personalized learning assistant that can adjust its language, examples, and methods based on the student’s responses.

YES

AIE
Microsoft
smol ai

### When to use agents: summary

Use agents for complex, multi-step tasks that combine capabilities such as question answering, analysis, and execution, or require personalized, adaptive responses.

[8:30](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=510s) [Agents are suggested for complex multi-step tasks, adaptive experiences, and tasks requiring multiple capabilities.](https://assets.ai.engineer/sites/aiecode2025/sha256/d5b5d812384f5a85629fb9a9fdc92ffe91bfa6d8a9c510d7228df0e635f2b872.webp)

When to use agents?

• Complex, multi-step tasks
• Personalized and adaptive experiences
• Multiple capabilities such as Q&A, analysis, task execution required

AIE
Microsoft
smol ai

### Planning and reasoning

Without feedback, an agent creates a plan and follows it. With feedback, it can refine the plan using tool outcomes or reflection on its responses.

[9:05](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=545s) [Planning without feedback includes chain of thought and tree of thoughts; planning with feedback includes ReAct and reflection.](https://assets.ai.engineer/sites/aiecode2025/sha256/a12253c8f53e4325a12af5d4da067eb4fea226c44548f9544b2888b83a65d5ed.webp)

Planning and reasoning

• Without feedback: Break down tasks into subtasks
  ◦ Chain of thought (CoT)
  ◦ Tree of thoughts (ToT)

• With feedback: Refine execution plan based on past actions
  ◦ ReAct
  ◦ Reflection

AIE
Microsoft
smol ai

### Chain of Thought

Chain-of-thought prompts work through a problem step by step, using either a worked example or “Let’s think step by step.” In the slide’s example, both approaches reach 4 blue golf balls; direct answers give 8.

[10:00](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=600s) [Comparison of few-shot and zero-shot prompting with and without chain-of-thought reasoning.](https://assets.ai.engineer/sites/aiecode2025/sha256/aef403b3aec12e3feba1eba9f3f5809ba0db80170b4b2337933b22b08c99f5a4.webp)

Chain of Thought

(a) Few-shot
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: The answer is 11.

Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
A:
(Output) The answer is 8. ✗

(b) Few-shot-CoT
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.

Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
A:
(Output) The juggler can juggle 16 balls. Half of the balls are golf balls. So there are 16 / 2 = 8 golf balls. Half of the golf balls are blue. So there are 8 / 2 = 4 blue golf balls. The answer is 4. ✓

(c) Zero-shot
Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
A: The answer (arabic numerals) is
(Output) 8 ✗

(d) Zero-shot-CoT (Ours)
Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
A: Let's think step by step.
(Output) There are 16 balls in total. Half of the balls are golf balls. That means that there are 8 golf balls. Half of the golf balls are blue. That means that there are 4 blue golf balls. ✓

Source: Kojima et al. (2022)

AIE
Microsoft
smol ai

### Tree of Thoughts

Tree of thoughts explores multiple reasoning paths, using the LLM to evaluate choices and search to look ahead or backtrack.

[10:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=625s) [A branching thought tree illustrates systematic exploration using generation, evaluation, and search.](https://assets.ai.engineer/sites/aiecode2025/sha256/6a379347876727a8165fbb46a05e3c65dd6e5314f661d9466ced91ffe8cd9e9c.webp)

Tree of thoughts

Input
thought
Output

LLM’s ability to generate and evaluate thoughts
+
Search algorithms (BFS, DFS)/Prompts
=
Systematic exploration of thoughts

[illegible] (2023)

AIE
Microsoft
smol ai

### ReAct (Reason + Act)

ReAct alternates reasoning, tool actions and observations. Here, a failed search for Front Row leads to a search for Front Row (software), ending with keyboard function keys.

[10:57](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=657s) [A ReAct example pairs reasoning and search actions with observations from a tool.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c4e358fb5e08f5c4a4571097b6de9ed3749c814fd3c12053b00d0b50e637de3.webp)

ReAct (Reason + Act)

(1d) ReAct (Reason + Act)

Thought 1: I need to search Apple Remote and find the program it was originally designed to interact with.
Act 1: Search[Apple Remote]
Obs 1: The Apple Remote is a remote control introduced in October 2005 by Apple … originally designed to control the Front Row media center program …

Thought 2: Apple Remote was originally designed to control the Front Row media center program. I need to search Front Row next and find what other device can control it.
Act 2: Search[Front Row]
Obs 2: Could not find [Front Row]. Similar: ['Front Row Seat to Earth', 'Front Row Motorsports', 'Front Row (software)', …]

Thought 3: Front Row is not found. I need to search Front Row (software).
Act 3: Search[Front Row (software)]
Obs 3: Front Row is a discontinued media center software …

Thought 4: Front Row (software) is controlled by an Apple Remote or the keyboard function keys. So the answer is keyboard function keys.
Act 4: Finish[keyboard function keys]

- Prompt LLMs to generate verbal reasoning traces and actions for a task
- Adjust plans using information obtained from external environments (tools)

[illegible] (2022)

AIE
aws

## Full transcript

[0:00](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=0s) [on hold music] Hello, everyone, and welcome to this workshop I like to call The A to Z of Building AI Agents.

[0:21](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=21s) So during the workshop today, uh, we'll spend about twenty to thirty minutes talking about the basic concepts of what AI agents are, when to use them, the different components of agents, and concepts that you'll find helpful during the hands-on portions of the workshop.

[0:37](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=37s) And then you will spend the rest of the time building an AI agent of your own with help and assistance from me. And I have my awesome team back there.

[0:46](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=46s) There's Tom, Ben, and Fabian. So if you run into issues, call upon one of us, and we'll figure it out.

[0:54](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=54s) Here's a little bit about me. I'm Apoorva, and I'll be your, uh, lead instructor for today. Five months ago, I stepped into my first ever developer advocacy role at MongoDB, and prior to that, I spent about six years as a data scientist in the cybersecurity space, applying machine learning to problems like phishing detection, malware and ransomware detection,

[1:15](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=75s) that kind of stuff. Outside of work, uh, I read a lot, try to yoga kinda regularly, and I'm always on a mission to hit as many local coffee shops as I can.

[1:27](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=87s) A few ground rules before we begin. No stupid questions here today. We are all here to learn, so ask as many questions as you'd like. We'll go over key concepts before getting into the hands-on labs.

[1:39](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=99s) So during these exercises, we definitely encourage you to, uh, form groups and work together, uh, where you can.

[1:47](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=107s) Here's a link to the slides and also the hands-on lab that you'll be working through today, and I'll leave this here for a few minutes for y'all to scan.

[1:56](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=116s) Um, the link and QR code should also be on these like postcards that were just handed out. And if you didn't receive one, then raise your hand, and we'll get you one.

[2:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=127s) Anyone need a postcard? Okay, I see some hands there, here.

[2:15](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=135s) Tom, right here.

[2:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=142s) Or is it just a collectible gauge? Yeah. They're not collectible. [laughs] [laughs]

[2:31](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=151s) All right. Moving on [laughs]. So the goal of the workshop is to introduce you to the basic concepts of AI agents and also get hands-on experience with building an agent end to end.

[2:43](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=163s) So yeah, I'm gonna start off by talking about what agents are, some... what are the AI agent use cases, components of an agent, and then we'll build an AI research agent together.

[2:54](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=174s) And depending on how long it takes us, we'll-- we may or may not have time for Q&A, but I'll be around to answer questions later.

[3:03](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=183s) So let's talk about-- start with talking about what are AI agents.

[3:09](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=189s) So an AI agent is a system that uses a large language model, or LLM, to reason through a problem, create a plan to solve the problem, and also execute the plan with the help of a set of tools.

[3:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=202s) So let's see how agents are different from other techniques for interacting with LLMs, because this will kind of help us build an intuition for when to use agents. So let's take the example of simple prompting, where you simply prompt an LLM to generate an answer based on its pre-trained parametric knowledge.

[3:40](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=220s) So as you can imagine, this is good for point-in-time general knowledge kinda questions, but probably not too much more, right? Because even if you manage to prompt, uh, the LLM to f-perform really complex tasks, then it might not have the means or information to execute on the task.

[3:56](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=236s) The LLM in this situation also can't self-revise and refine responses based on either previous or new information, and it definitely doesn't have a means to learn preferences and provide, uh, personalized responses over time, which sometimes is a requirement.

[4:13](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=253s) Moving on to retrieval-augmented generation, AKA RAG. Uh, with RAG, you can broaden the scope of the LLM by augmenting its knowledge with information retrieved from a knowledge base. So that way you can be somewhat confident that the LLM at least has information required to, uh, perform tasks that you want it to perform, but it doesn't quite solve

[4:34](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=274s) for some of these other requirements, such as handling complex tasks, self-refinement, or personalization.

[4:42](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=282s) Coming to agents. With agents, you can give the LLM access to external tools and past interactions which act as the memory of the agent, and then you can prompt it to go through multiple iterations of reasoning and action taking to finally arrive at the final answer.

[4:59](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=299s) So tools is how agents are able to execute on complex multi-step tasks, and LLMs can also be prompted to incorporate the feedback or output from tools into the reasoning process to, say, repeat steps if necessary or call additional tools as follow-up tasks.

[5:16](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=316s) Coming to past interactions. Past interactions can be persisted and updated, which means the LLM can now learn from these to provide personalized responses over a period of time. So as you can imagine, tools, memory, and iterative prompts can solve a lot of problems, but there's obviously some known challenges at the moment, such as long-term planning, where the

[5:37](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=337s) agent i-is expected to, uh, execute complex tasks based on information, a lot of information or information it's learnt over, uh, a long-ish period of time. There's also a high, uh, cost and latency associated with agents because they typically trade these for a shot at higher accuracy.

[5:55](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=355s) But despite all of these challenges, I think we can agree that agents is how we get the most out of large language models as of today.

[6:04](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=364s) So let's take some example tasks or questions and try to answer whether or not, um, the task really requires an AI agent. So this one, for example, like who was the first president of United States?

[6:17](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=377s) Does it require an AI agent to complete this task?

[6:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=382s) I see some people nodding yes [laughs] mostly nos. But I would say no, because the information required to answer this question is very likely present in the parametric knowledge of most LLMs that we know today.

[6:35](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=395s) So I don't think it requires, um, an AI agent.

[6:40](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=400s) How about this one? What's the travel reimbursement policy for my company, MongoDB, or your company? Uh, do you think this task requires an AI agent?

[6:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=411s) Two-step process.

[6:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=411s) What's that?

[6:53](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=413s) Two-step process.

[6:53](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=413s) Two-step process?

[6:54](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=414s) Yes, it-

[6:54](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=414s) What, what's the two steps?

[6:56](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=416s) It first you need to disambiguate, uh, for this ask because the policy will depend on the location of the employee.

[7:05](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=425s) That's a good point. Do-

[7:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=427s) I, I did it, so that's how it works.

[7:09](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=429s) Okay. [laughs] All right. All right. Yeah. So I would say it's a pretty straightforward task, provided the LLM has the r- access to the right information. So to me, it sounds like a better fit for retrieval-augmented generation where the LLM has knowledge, um, access to the right knowledge base than, uh, something complex like an AI agent.

[7:31](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=451s) How about this guy? Like, "How has the trend in the average daily calorie intake," it's already too long, but, "among adults changed over the last decade, and what impact might it have on obesity rates?

[7:41](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=461s) Additionally, can you provide a graphical representation of the trend?" Do we think this requires an AI agent?

[7:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=471s) I would think so. Like, I think this task looks like it involves multiple sub-tasks such as at least data aggregation, visualization, and also reasoning through, uh, the results that it's obtained from these various tasks.

[8:04](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=484s) So I think it sounds like a good fit for agents.

[8:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=487s) How about this one? Uh, "Creating a personalized learning assistant that can adjust its language, examples, and methods based on the student's responses."

[8:19](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=499s) I see some nods, and I agree. I think this is another example of a complex task which requires also long-term personalization. So again, I think it's a good use case for agents.

[8:31](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=511s) So the TLDR is use agents for complex multi-step tasks that require integration of multiple capabilities such as question answering, task execution, analysis, that kind of thing, uh, and using all of these to arrive at a final answer or outcome, and also if there is a need for personalization or adapted responses.

[8:53](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=533s) So as we saw, memory, tools, and being able to reason is what really makes AI agents so powerful. So let's dig a little bit deeper into each of these components, starting with planning and reasoning.

[9:06](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=546s) So the simplest way to imbue planning and reasoning capabilities into agents is via, uh, believe it or not, user prompts. You can start super simple by prompting the agent to create a plan of action based on its initial understanding of the problem, and this is what we call planning without feedback, since the agent does not modify its

[9:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=565s) execution plan based on any new in- information that it's gathering from tools that it's executing. It's just in the beginning, it creates an execution plan and runs with it.

[9:36](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=576s) So common design patterns for this kind of planning are chain-of-thought and tree-of-thoughts. Then there's planning with feedback, where you can prompt the agent to adjust and refine its responses based on tool outcomes or even asking it to, uh, critique and reflect upon its own responses.

[9:53](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=593s) And common design patterns in this regard are ReAct and reflection, and we'll experiment with some of these in today's workshop.

[10:02](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=602s) So let's first understand chain-of-thought. So chain-of-thought is as simple as prompting an LLM to think through a problem step by step instead of directly providing an answer. Uh, you can do this either in a zero-shot manner by literally saying, "Hey, let's think step by step," or in a few-shot manner where you show it how to work through

[10:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=622s) a complex problem using one or more examples.

[10:27](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=627s) Then we have tree-of-thoughts, which takes the idea of chain-of-thought up a notch. So tree-of-thoughts allows LLMs to perform like deliberate decision-making by considering multiple different reasoning paths and having it self-evaluate choices to decide the next course of action.

[10:44](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=644s) And so it kind of combines this LM, LLM's ability to generate and evaluate thoughts with search algorithms because it can also look ahead and backtrack when necessary to make kind of global choices.

[10:59](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=659s) Then we have patterns for reasoning with feedback, starting with ReAct. Uh, so what we do here is we prompt LLMs to generate verbal reasoning traces and also co- tell us the actions that it'll take to solve a particular problem.

[11:12](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=672s) So after each action, we ask the LLM to make an observation based on information or feedback obtained from the previous action and plan what action to take next. And then this kind of, uh, process continues until the LLM or you can intervene and say that you've reached the final answer, so exit the loop.

[11:32](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=692s) So in this example here, uh, as you can see, the first thing that the LLM does is generates a thought saying like, "Okay, this is how, uh, I need to solve this problem."

[11:40](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=700s) Then the second is an action step where, in this case, it's determined that it needs to, um, call the search tool with arguments that it's determined, and then it makes an observation saying, "Okay," like, "I don't think I, I have an answer yet.

[11:54](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=714s) This is what I'm going to do next," and does that till it, uh, reaches the final answer.

[12:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=721s) Another technique for incorporating feedback into the planning process is via reflection. And this involves prompting LLMs to reflect on and critique past actions to decide what, uh, action to take next.

[12:14](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=734s) And you can either, uh, prompt the same LLM to generate and critique, you can use different LLMs or even use multiple agents where one agent generates responses and the other critiques them.

[12:27](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=747s) But yeah, whatever the architecture, the goal is to, uh, run the generation reflection loop several times before, um, the LLM arrives at a final answer. So essentially trading compute for a better shot at accuracy.

[12:43](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=763s) The next component we want to talk about is memory.

[12:47](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=767s) And this component allows AI agents to store and recall past conversations and enables them to learn from these interactions. And as you can imagine, memory is a pretty, um, complex and nebulous concept, but-- And you could break it down into several categories.

[13:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=781s) But broadly, uh, when I think of memory, it's two main, uh, types of memory, much like us humans, right? Short-term and long-term memory. So short-term memory in, uh, the case of agents deals with storing and retrieving information from a single conversation.

[13:17](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=797s) And long-term memory deals with storing, updating, and retrieving information from multiple conversations had over a period of time. And this is what really helps agents personalize their responses over a long-ish period of time.

[13:32](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=812s) So short-term memory is relatively easy to implement. Like, how hard can it be to store a single conversation, right? Like, in most cases, not that hard. But unless the conversation gets too long, in which case you need to now start considering, uh, how to condense that list so you aren't overwhelming the LLM with too much information.

[13:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=831s) And some solutions to th- to that are things like retrieving the n most recent messages or summarizing the conversation at the cost of some information loss.

[14:03](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=843s) Long-term memory, on the other hand, is a largely unexplored area so far since it's non-trivial to decide and implement, uh, what states to track, uh, and how to track them and when to update them.

[14:16](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=856s) So but I think some patterns are emerging in the sense that the best way to go about implementing long-term memory is to design application-specific agents. That way you're able to narrow down the number of states you want to track and just focus on those, uh, and figure out how to update them.

[14:35](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=875s) And finally, we have tools. So tools are interfaces for agents to interact with the external world in order to achieve their objectives. And these can range from simple APIs such as search, weather APIs to, uh, complex things like vector stores or even specialized machine learning or deep learning models.

[14:57](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=897s) So tools for LLMs are typically defined as functions, and most recent LLMs have been trained to identify when a function should be called, and they'll respond with a function signature that you can then use to call a particular function in your code.

[15:12](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=912s) And tools like LangChain handle the function calling for you, but the basic concept still remains. And to help the LLM identify which function to use, you typically use a descriptive tool name, uh, specify which function to call, provide a pretty detailed description of what exactly the function does, and also, uh, the types of arguments would also be

[15:33](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=933s) helpful. So finally, the fun part. Uh, you're not here to listen to me ramble on about agents. So, uh, in today's workshop, we'll be building an AI research agent.

[15:46](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=946s) And the agent's primary objective is to provide research assistance by supplying a list of papers to read, uh, summarizing research papers, and answering questions about research topics.

[15:59](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=959s) And this is kind of how the workflow of our agent is going to look like. We will use a free and open source model from Fireworks called FireFunction v1.

[16:08](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=968s) They had just released a v2, but I had prepared my workshop until then. So today we'll use v1, uh, as the brain of our agent. We'll also try out some of the reasoning design patterns that we were just talking about, like chain-of-thought and ReAct.

[16:23](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=983s) We'll also give the agent access to three tools, one for sub-- uh, getting paper summaries, uh, one for getting, um, a list of papers to read, and the third one being answering questions using, uh, a MongoDB knowledge base.

[16:38](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=998s) And finally, we'll also explore adding short-term memory to the agent and persisting it to, uh, a database in MongoDB.

[16:48](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1008s) But yeah, uh, very qu- soon we are going to break for our first hands-on portion, but just some things to keep in mind. Uh, each time we, each time we break for a hands-on section, you'll navigate to the hands-on lab at the QR code that you have at your tables or you just scanned, and you'll work through

[17:05](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1025s) one or more sections at a time. And you'll see these emojis sprinkled all over the place. So this like open hands emoji and the superhero emoji indicate hands-on sections, except, uh, I would highly advise do the open hands ones first and only if your time-- uh, if you have time, go to the super emoji sections.

[17:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1045s) Uh, you'll also be filling code into a Jupyter Notebook, uh, and the places where you need to fill in code are indicated by these code_block placeholders. So those are the ones you need to fill in with your code.

[17:37](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1057s) And before any cell in the notebook that requires you to fill in code, you'll also see this books emoji indicating documentation that you need to reference for that particular piece of code.

[17:48](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1068s) And finally, you'll find, uh, solutions to all the hands-on pieces in the-- at the QR code link. But I highly encourage you to try working through stuff on your own before you look at the solutions.

[18:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1081s) And even if you do, then try to understand what's really going on.

[18:06](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1086s) With that, let's go ahead and break for our first hands-on section, uh, which is just setting up the development environment and prerequisites for the workshop. So yeah, let's take about fifteen to twenty minutes to work through this section.

[18:20](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1100s) Um, so if you go to that link, you want to start at the section titled MongoDB Atlas and work all the way through to the Dev Environment section. Let's go.

[18:33](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1113s) How are we feeling? Um, are we mostly done? Not done? Done? Like, show of hands, how many folks are done with this part?

[18:44](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1124s) Okay. Uh, mostly done? Okay. Uh, five more minutes? Yeah, let's do five more minutes.

[18:56](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1136s) All right. I think I'm gonna move on just in the interest of time, but it's a self-paced lab and you have access to all the material after the fact, so feel free to move at your own pace.

[19:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1147s) Um, cool. So let's move on to some libraries, tools, and general concepts that you'll come across in the next hands-on portion. So the first thing you'll run into is, um, this library called Datasets, which are go- we are going to use to download a dataset of arXiv papers from Hugging Face.

[19:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1165s) Uh, we are going to use the load_dataset method to download the arXiv embeddings dataset from, uh, the MongoDB educational AI Hugging Face org.

[19:36](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1176s) And then you'll run into something called ArxivLoader, which is a document loader class in LangChain. Uh, we are going to be using this to load research papers from arxiv.org as LangChain document objects.

[19:48](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1188s) And an example of what a document in LangChain looks like is shown here. So essentially has the raw text under the page_content attribute and some automatically extracted metadata, in this case, the publish date, title, authors, and summary under the metadata attribute.

[20:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1207s) So we're gonna be using ArxivLoader in one of our-- two of our agent tools. One tool is already done for you, and that's the tool to get relevant papers from arXiv, and you'll also use the same, um, document loader to-- for the summary tool as well.

[20:22](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1222s) So the simplest way to create tools in LangChain is using the tool decorator, which makes tools out of functions. So for this tool, we have used the load method of ArxivLoader to load data into document objects, and the query argument takes a topic or paper ID, and the load_max_docs indicates how many documents to download from arXiv.

[20:42](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1242s) And finally, we are only extracting the metadata because we want to only provide a list of papers, uh, and not the full paper content.

[20:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1251s) We'll also be using PyMongo, which is the Python driver for MongoDB. We'll use it to connect to MongoDB databases and collections, and also delete and insert documents, uh, from and to MongoDB to build the knowledge base for our agent.

[21:07](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1267s) Uh, we'll also be using a few LangChain integrations, which are essentially standalone packages for, uh, third-party providers such as MongoDB and, uh, LangChain to make things like versioning, dependency management, and testing kind of easier.

[21:20](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1280s) Uh, so we'll use the LangChain MongoDB integration to use MongoDB Atlas as a vector store, and also to store and retrieve chat history for the agent. We'll also use, uh, langchain-huggingface to access open source embedding models from Hugging Face.

[21:37](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1297s) And finally, we'll use langchain-fireworks to access, uh, chat completion models from Fireworks AI.

[21:45](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1305s) And you'll be using the LangChain Expression Language or LCEL to create RAG and agent workflows using LangChain, and it's essentially a declarative way to chain together prompts, data processing steps, LLMs, and tools in a LangChain fashion.

[22:02](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1322s) Uh, and each unit in the chain is called a runnable, and the way to chain them together is using the pipe operator that takes the output from the left of the pipe and passes it as input to the right of the pipe.

[22:14](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1334s) Uh, and here's a simple example of just passing a prompt to an LLM, generating an answer, and formatting its output. And finally, if you want to call the chain, then you use the inv-invoke method on it.

[22:25](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1345s) And you'll be using this to test out some of the things that you're building during the workshop.

[22:31](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1351s) And finally, you have this thing called RunnableLambda. Uh, and this is a runnable that converts any arbitrary Python function into a LangChain runnable, and it's as simple as defining the function and then wrapping the function into a RunnableLambda.

[22:47](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1367s) So yeah. Let's take another twenty minutes to now create the tools for your research agent. Uh, so yeah. Just work through the Create Agent Tools section of the lab that you were, uh, just working through.

[23:01](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1381s) Okay. Um, so hopeful-hopefully, we are kind of at least midway through creating tools for our agent. But in the next section, we are gonna be creating, uh, the agent itself and experiment with, um, the different reasoning design patterns that we were talking about, like chain-of-thought and ReAct.

[23:20](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1400s) So to create the agent, we are gonna start with the simplest way of creating a tool calling agent in LangChain, which is using the create_tool_calling_agent constructor.

[23:30](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1410s) Um, and you're gonna be us- starting with that abstraction, but let's try to understand what's happening behind the scenes of that abstraction, right? So it's essentially creating a runnable sequence consisting of a prompt template, which has a placeholder for the agent scratchpad, which is the agent's intermediate steps as it's taking different actions and making observations, uh, an

[23:51](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1431s) LLM with knowledge of the tools that we were just creating, and an output parser for formatting the agent's response.

[24:00](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1440s) And then we'll also be exploring a ReAct agent that uses ReAct prompting to guide the agent to take a series of reasoning and action-taking steps to arrive at the final answer.

[24:11](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1451s) And for this, we'll use the create_react_agent constructor, which follows a similar series of steps as the tool calling agent, except it uses a ReAct prompt template, and the LLM has a knowledge of when to stop the reason action-taking sequence using, uh, a stop sequence.

[24:29](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1469s) And then the output parser has logic to parse these ReAct style LLM calls, and you can see what those look like right there. So it has, um, a thought, an action, an action input, and an observation.

[24:41](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1481s) So just parsing that to make it more readable to the user in the end.

[24:47](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1487s) And finally, uh, you'll come across the AgentExecutor, which is the runtime for the agent. This is what actually calls the agent, executes the action that the agent is choosing, passes the action outputs back to the agent, and repeats any steps as the agent decides what to do next.

[25:03](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1503s) And that's the pseudocode what-- for what the AgentExecutor is essentially doing. So as long as the agent thinks that it hasn't finished its task, which is the while loop there, the agent determines and runs a series of actions until it finally finishes the task.

[25:21](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1521s) So yeah, let's take another, another twenty minutes to complete the create agent section and any other things that you were working on previously.

[25:31](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1531s) Yeah.

[25:32](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1532s) One more time.

[25:33](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1533s) All right. Uh, we have one last thing to do with our research agent, which is to give it memory or add short-term memory to it. Um, and in this case, we are going to do that by giving it access to its chat message history.

[25:50](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1550s) So in LangChain, the way to do this is by wrapping the agent runnable that you created using the create_tool_calling_agent or create_react_agent, wrapping that runnable inside another runnable called RunnableWithMessageHistory, which is specifically designed to manage the memory of other runnables.

[26:08](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1568s) So essentially, this runnable can take a function that persists the chat message history for your agent to a database. We'll use MongoDB in this case, and by default, it organizes the chat history using a session ID that you pass in, uh, along with your input query or prompt.

[26:26](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1586s) So yeah, let's play around with that for the remainder of the time, and if you have any more questions or stuck at something, we can talk through that too for the rest of the time.

[26:39](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1599s) One last thing I would request, uh, once you're done with all your stuff is, um, yeah, if you want to connect, that's not the, uh, mandatory thing. Uh, nothing is mandatory.

[26:50](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1610s) But yeah, I'd really appreciate if you could fill out a short survey that's at the QR code link that you, uh, scanned in the beginning. This is the first time I'm doing this workshop, so any feedback you have will only help me make this, uh, better in the future.

[27:05](https://www.youtube.com/watch?v=IRp7lvBlbHs&t=1625s) So yeah, that'd be much appreciated. Um, other than that, this is it from me for today, and thanks for being here. [upbeat music]
