# How to Build Planning Agents Without Losing Control - Yogendra Miraje, FactSet

Yogendra Miraje

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/sl3icG-IjHo-build-planning-agents-without-losing-control) · [Video](https://www.youtube.com/watch?v=sl3icG-IjHo)

## How to Build Planning Agents Without Losing Control

A natural-language blueprint can give an enterprise agent room to plan while keeping its tools, task dependencies, and evaluation criteria explicit.

Familiarity with LLM tool calling and basic service APIs will help; the article introduces the planning and workflow terminology.

### Capability exceeds control

[0:17](https://www.youtube.com/watch?v=sl3icG-IjHo&t=17s)

Building AI applications can feel like driving a monster truck through a crowded mall with tiny joysticks. The model has considerable power; the developer has surprisingly little control over how it behaves. Yogendra Miraje, working at FactSet, a financial data and software company, uses this mismatch to explain why he believes AI applications have yet to experience their own ChatGPT moment despite rapid improvements in model intelligence.

One source of that mismatch is **missing context**. In an enterprise, the missing information is often knowledge of the organization's workflows: what needs to happen, in what order, and with which capabilities. A capable model cannot reliably follow a business process it has never been given.

![Slide lists LLM errors, unclear instructions, tool problems, and poorly written input/output contracts, then highlights missing context and limited enterprise-specific workflow knowledge.](https://ai.engineer/api/talk-essay-media/c49421f3ab82e1f3c643a19d48f7796bf01ccd6bc1f5308dca4bc9556f100d51)

Why agents misbehave: missing context and enterprise workflow knowledge.

### Who controls the workflow?

[1:24](https://www.youtube.com/watch?v=sl3icG-IjHo&t=84s)

Tools extend what an LLM can do beyond its training-time knowledge. Combine an LLM with tools and memory, and Miraje calls the result an **augmented LLM**. Put it on a static, predefined path, and it becomes a workflow. Give it substantial autonomy and a feedback loop, and it becomes an agent. Workflows offer control and reliability; agents offer flexibility and autonomy. An agentic workflow aims to combine those properties by planning and executing a workflow according to the goal, context, and feedback.

The distinction rests on who decides the path:

| System | Who controls the path? | Workflow |
|---|---|---|
| Workflow agent | Predefined workflow | Static; run by an agent |
| Agentic workflow | Agent | Dynamic; planned and run by the agent |

These are Miraje's working definitions, useful because the terms are often used interchangeably. He also invokes Andrew Ng's view of agentic systems as a spectrum: an agentic workflow generally sits farther toward autonomy than a workflow agent.

### Reuse enterprise services, then plan around them

[3:25](https://www.youtube.com/watch?v=sl3icG-IjHo&t=205s)

For enterprises, the attraction goes beyond predictability. Agentic workflows offer a way to automate processes while building on existing microservices—capabilities that may represent years or decades of investment. The agent layer can compose those services instead of requiring the organization to replace them. Although the examples here concern enterprises, the architecture applies more broadly.

Miraje recommends moving from a predominantly [ReAct-based](https://arxiv.org/abs/2210.03629) approach toward proactive planning. ReAct already supports tracking and updating plans through interleaved reasoning and action; the architectural change here is to make task decomposition an explicit stage. Alongside tools, memory, and reflection, introduce **planning by sub-goal division**: break the goal into simpler steps that the system can execute.

Different planning architectures make different tradeoffs. Miraje recommends LangChain's explanations and accompanying code; its [Plan-and-Execute Agents](https://www.langchain.com/blog/planning-agents) article provides relevant implementation reading. At FactSet, his team adapts [LLMCompiler](https://arxiv.org/abs/2312.04511) to its own problems, adding a high-level planning stage before the task planner.

### From a user question to a replanning decision

[5:29](https://www.youtube.com/watch?v=sl3icG-IjHo&t=329s)

The architecture wraps enterprise microservices in tools, then routes a user question through four responsibilities:

1. **Blueprint generator:** produce a high-level plan for the requested workflow.
2. **Planner:** turn that blueprint into a low-level task plan.
3. **Executor:** execute the planned tasks.
4. **Joiner:** combine the outputs from those tasks.

The blueprint describes the intended work before the planner commits to the details of individual calls.

After the joiner, replanning logic determines whether the system should take another pass or terminate and return a response. A recursion limit can bound the loop so the agent does not continue indefinitely. FactSet implements the blueprint generator, planner, executor, and joiner as separate [LangGraph](https://docs.langchain.com/oss/python/langgraph/overview) nodes. The graph makes both the forward execution path and the return path for replanning explicit.

![Architecture diagram shows a blueprint generator, planner, executor, joiner, response formatter, and replanning loop alongside an MCP tool server and enterprise microservices.](https://ai.engineer/api/talk-essay-media/b48d748b9f0b9a671c6dc95239b122e7b419d42f8745e6a7e0b57d4c4b9261bb)

Agentic workflow architecture linking planning, execution, tools, and enterprise microservices.

### Design tools from the agent's point of view

[6:37](https://www.youtube.com/watch?v=sl3icG-IjHo&t=397s)

Miraje expects much of the implementation effort to go into building tools around enterprise microservices. Their relationship is **many-to-many**, not necessarily one tool per service. A tool interface should reflect the work an agent can understand and perform, rather than simply copying the organization's service boundaries. The design question is whether the agent has enough information to choose and use the capability correctly.

An [MCP tool server](https://modelcontextprotocol.io/specification/2025-03-26/server/tools) provides a standard interface for those capabilities. Within that interface, different pieces of documentation answer different questions:

| Tool information | Agent's question |
|---|---|
| Purpose | Which tool should I select? |
| Detailed description | When should I invoke it? |
| Input/output contracts | How do I use it? |

Here, input/output contracts are interface-design requirements. The historical March 2025 MCP specification defines `inputSchema` and returned result content, but not an `outputSchema` field. Validation checks complete the interface by acting as brakes on incorrect tool use.

### Give the planner a blueprint

[8:00](https://www.youtube.com/watch?v=sl3icG-IjHo&t=480s)

A **blueprint** is a sequence of natural-language workflow steps matched to available tool capabilities. It is fed to the planner rather than executed directly. FactSet introduced this stage after finding that the planner became overloaded when asked to handle too much at once. Decomposing the task in natural language separates deciding what work is needed from specifying the low-level calls.

The blueprint also provides finer control over task planning by narrowing the tools placed in the planner's context. Large collections of tool descriptions consume the context window and burden tool selection. The blueprint stage can select the relevant tools, leaving the planner with a smaller set of capabilities appropriate to the workflow.

That intermediate representation makes agent behavior easier to interpret. It also gives nontechnical collaborators something they can discuss directly: natural-language steps are less intimidating than a collection of function calls. The same artifact therefore helps constrain planning and expose the intended business process for review.

### Preparing for NVIDIA's earnings call

[9:34](https://www.youtube.com/watch?v=sl3icG-IjHo&t=574s)

Preparing for a company's earnings call is a common financial research workflow. Miraje demonstrates a deliberately simplified version for NVIDIA. The blueprint pairs a tool with a task; the executable plan pairs a tool with a function call. He describes two tools in the blueprint, supporting a sequence of four tasks.

The workflow first summarizes NVIDIA's previous earnings call, then retrieves financial data for the company. It uses that material to suggest questions for the upcoming call and finally generates a comprehensive report. Task context passes into the corresponding function calls, allowing later work to use earlier results.

![Two four-step lists pair retrieval, reasoning, and reporting tasks with corresponding function calls and task-context references.](https://ai.engineer/api/talk-essay-media/eed48a230023bba1fea91574184d2335b00d8359ac5cdad4f8ded5d0c93b4976)

An NVIDIA earnings-call blueprint paired with its executable plan.

A compact JSON representation makes the intended dependencies explicit. The identifiers below describe a proposed plan, with `context_from` naming earlier task outputs that a later step will need:

```json
{
  "company": "NVIDIA",
  "status": "planned",
  "steps": [
    {
      "id": "previous_call",
      "task": "Summarize the previous earnings call",
      "context_from": []
    },
    {
      "id": "financial_data",
      "task": "Retrieve company financial data",
      "context_from": []
    },
    {
      "id": "questions",
      "task": "Suggest questions for the upcoming earnings call",
      "context_from": ["previous_call", "financial_data"]
    },
    {
      "id": "report",
      "task": "Generate a comprehensive preparation report",
      "context_from": ["previous_call", "financial_data", "questions"]
    }
  ]
}
```

This representation illustrates the task-context relationship; the planner's next responsibility is to bind the work to tool calls and their input contracts.

In the displayed response comparison, Miraje describes the original answer as generic and the answer after introducing the agentic workflow as structured around the financial research process. The useful change is that the response reflects the requested workflow, rather than merely supplying a broadly relevant answer. This is a qualitative demonstration; no numerical improvement is reported.

### Evaluate the behavior you need to control

[10:49](https://www.youtube.com/watch?v=sl3icG-IjHo&t=649s)

The architecture needs an evaluation framework that is built and maintained alongside it. Miraje calls for both **component evaluations** and **end-to-end evaluations**, using metrics that matter to the application. Evaluating individual stages helps locate failures; evaluating the complete workflow checks whether those stages together produce the desired result. Code-based checks, LLM judges, and human review serve different purposes.

Choose the method according to the aspect being tested:

| Aspect | Suggested evaluation |
|---|---|
| Blueprint resembles a golden blueprint | LLM-as-a-judge |
| Correct tools selected | Code-based checks |
| Plan follows the blueprint | LLM-as-a-judge |
| Report formatting | Human review |

A single overall score can hide which part of the system failed. These separate checks distinguish an unsuitable workflow from incorrect tool selection, a plan that departs from the blueprint, or an unsatisfactory presentation of the results.

### When planning is the wrong choice

[11:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=715s)

The ability to plan dynamically is not useful in every application. Miraje identifies several cases where an agentic workflow is a poor fit:

- **Fixed, repeated tasks:** an ETL pipeline is probably the better choice.
- **Work that cannot be captured as a workflow:** the architecture has little useful process structure to operate on.
- **Deterministic outcomes:** strict compliance or safety-critical requirements may rule out agentic planning.
- **Tight latency or cost constraints:** the planning architecture may not fit the operating budget.

These boundaries belong in the architecture decision before investing in a planner and its supporting infrastructure.

### Start with simple blueprints

[12:32](https://www.youtube.com/watch?v=sl3icG-IjHo&t=752s)

Start with simple blueprints and work toward a more complex retrieval-augmented system for blueprints as the need develops. The immediate job of the blueprint remains concrete: supply the high-level plan and reduce the tools in the planner's context. Keep tool usage simple, and build safety guardrails, evaluations, and observability into the system.

The reliability at scale that Miraje seeks depends on this combination of task decomposition, plan-and-execute architecture, and tools that complement existing microservices. His own blueprint stage is an example of adapting a research architecture to the problem rather than treating it as fixed. Experiment with those architectural changes, but make evaluations first-class components so there is a basis for judging whether the changes help.

### Implementation references and orchestration choices

[14:10](https://www.youtube.com/watch?v=sl3icG-IjHo&t=850s)

Asked for a GitHub project or implementation reference, Miraje points back to LangChain's planning-agent material and its code for the research architectures. The reference slide groups Plan-and-Solve, ReWOO, and LLMCompiler as starting points for exploring plan-and-execute agents.

![Slide lists Plan-and-Solve, ReWOO, and LLM Compiler with brief descriptions, a Plan + Execute Agents image, and the link https://blog.langchain.dev/planning-agents/.](https://ai.engineer/api/talk-essay-media/5326c9cc019fcdca829de3db91179ff8d9207ef76abf8b16f65f8c39ca28c594)

Recommended planning architectures and a LangChain implementation reference.

The final question asks whether MCP, LangGraph, or another approach will become the primary orchestration method. Miraje's answer separates two complementary jobs. MCP provides an organizational standard for exposing functionality: wrap a shared capability once, then reuse it across different AI applications.

LangGraph handles orchestration—the arrangement of planning, execution, and feedback inside an application. Miraje regards it as a useful choice while expecting multiple frameworks to remain useful. Standardizing tool access does not require every application to use the same orchestration framework; the appropriate framework depends on the use case.

## Resources

- [Plan-and-Execute Agents](https://www.langchain.com/blog/planning-agents) — LangChain's guide to plan-and-execute, ReWOO and LLMCompiler architectures, with links to implementations.
- [An LLM Compiler for Parallel Function Calling](https://arxiv.org/abs/2312.04511) — The research architecture behind LLMCompiler's dependency-aware planning and parallel function execution.
- [LLMCompiler research code](https://github.com/SqueezeAILab/LLMCompiler) — Original implementation with setup instructions, benchmark runners and examples for configuring custom tools.
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) — The original ReAct paper explains how interleaved reasoning, actions and observations support interactive task solving.
- [MCP tool specification, March 2025](https://modelcontextprotocol.io/specification/2025-03-26/server/tools) — Historical protocol reference for exposing tools, defining inputs, returning results and enforcing validation.
- [LangGraph overview](https://docs.langchain.com/oss/python/langgraph/overview) — Current introduction to LangGraph's orchestration runtime, including a minimal graph example.
- [Using Large Language Models to Converse with Your Data](https://insight.factset.com/using-large-language-models-to-converse-with-your-data) — Miraje's earlier explanation of enterprise data retrieval, semantic metadata and domain knowledge for financial-data assistants.

## Slides

### Innovation Partner and Platinum Sponsors

[0:10](https://www.youtube.com/watch?v=sl3icG-IjHo&t=10s) [Sponsor slate featuring AWS and six platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/ee4ce3f056d2945ae91accd40b579fd46352d134e6ddc3a7cb63db78abb7b8aa.webp)

INNOVATION PARTNER
aws

PLATINUM SPONSORS
Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### Evolution of AI

A schematic curve shows intelligence rising from AlexNet in 2012 through transformers and Gen-AI to agents in 2025, steepening over time.

[0:25](https://www.youtube.com/watch?v=sl3icG-IjHo&t=25s) [A rising curve marks AlexNet in 2012, transformers in 2017, generative AI in 2023, and agents in 2025.](https://assets.ai.engineer/sites/aiecode2025/sha256/9bfa239227cf40c471bb5d79dc0f1f0074b6ea1f8cdc84017a80c8cc77041fe2.webp)

Evolution of AI

Intelligence

AlexNet
Transformers
Gen-AI
Agents

Timeline: 2012, 2017, 2023, 2025
Year

Other visible text: AIE; Microsoft; smol ai; World's Fair

### Why Don’t Agents Behave?

Agents can fail because of model errors, unclear instructions, tool problems, or poor input/output contracts. Miraje singles out missing context, especially knowledge of enterprise-specific workflows.

[0:52](https://www.youtube.com/watch?v=sl3icG-IjHo&t=52s) [Reasons include model errors, incomplete instructions, tool usage, input and output contracts, and missing enterprise workflow context.](https://assets.ai.engineer/sites/aiecode2025/sha256/51d66824ab0a5c1ac5bb76af0280bcc9f13df5020355601eefe60f5f628d1754.webp)

Why don’t Agents “behave” ?

▪ LLM messes up
▪ Unclear or Incomplete Instructions
▪ Lack of tools or Incorrect usage of tools
▪ Poorly written input/output contracts

Missing the right Context
▪ Limited knowledge of enterprise-specific workflows

AIE
AI Engineer World’s Fair
Microsoft
smol ai

[0:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=55s) [Slide lists model errors, unclear instructions, tool problems, poor input and output contracts, and missing enterprise workflow context.](https://assets.ai.engineer/sites/aiecode2025/sha256/81cb0738fc750b1b520cf64c4e71ba22154241e072ed0daa2eef2c4f5cc17b7c.webp)

Why don’t Agents “behave” ?

▪ LLM messes up
▪ Unclear or Incomplete Instructions
▪ Lack of tools or Incorrect usage of tools
▪ Poorly written input/output contracts

Missing the right Context
▪ Limited knowledge of enterprise-specific workflows

AIE
AI Engineer World’s Fair
aws

### Context

Miraje introduces shared context before defining terms used to discuss agents.

[1:11](https://www.youtube.com/watch?v=sl3icG-IjHo&t=71s) [Section slide titled Context.](https://assets.ai.engineer/sites/aiecode2025/sha256/c9e0d6daff4a9f0420f286956d6b42b92e7c8986e45e3705ff35339be4e00920.webp)

Context

Interface text:
AIE
Microsoft
smol ai
AI Engineer World's Fair

### Key Definitions

A tool extends an LLM’s reach; tools and memory make an augmented LLM. A predefined path makes it a workflow, while high autonomy and a feedback loop characterize an agent.

[1:20](https://www.youtube.com/watch?v=sl3icG-IjHo&t=80s) [Slide defines tool, augmented LLM, workflow, and agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/503a09ed87717988c74e197c00743b62b817b0bd4b653741aa617c34c7702735.webp)

Key Definitions

Tool
Any external function that enhances LLM’s real world reach

Augmented LLM
LLM augmented with Tools and memory

Workflow
Augmented LLMs with predefined /static path

Agent
Augmented LLMs with high autonomy and feedback loop

AIE
AI Engineer World’s Fair
aws

[1:41](https://www.youtube.com/watch?v=sl3icG-IjHo&t=101s) [Definitions of tool, augmented LLM, workflow, and agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/42fc1f09f54aa5e53e7f950f7bc25cb690d5d1cb7b7699d38495b7fe6a422953.webp)

Key Definitions

Tool
Any external function that enhances LLM’s real world reach

Augmented LLM
LLM augmented with Tools and memory

Workflow
Augmented LLMs with predefined /static path

Agent
Augmented LLMs with high autonomy and feedback loop

AIE
AI Engineer World’s Fair
aws

### Can We Have Best of Both Worlds?

The slide contrasts controllable, reliable workflows with flexible, autonomous agents. Its chart sketches reliability declining as autonomy increases.

[1:57](https://www.youtube.com/watch?v=sl3icG-IjHo&t=117s) [A comparison of workflows and agents alongside a chart relating reliability and autonomy.](https://assets.ai.engineer/sites/aiecode2025/sha256/74c84e8167024dcdda056570a31011325fe04467e65d281931ab670a0a7002f4.webp)

Can we have best of both worlds?

Workflows
Controllable
Reliable
Pre-defined paths
Limited Autonomy

Agents
Less Controllable
Less Reliable
Flexible
High Autonomy

Y-axis: Reliability
X-axis: Autonomy
Workflows
Agents

Credit - Langchain blog

### Agentic Workflows

An agent plans and executes a workflow, then decides whether to replan or return a response. The slide presents this loop as combining workflow reliability with agent flexibility.

[2:04](https://www.youtube.com/watch?v=sl3icG-IjHo&t=124s) [A query enters a plan and execute loop, with a replan decision before the response.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b98e2ee8fa3a7231bb88e022d38adc59cbd28e1e718b3919a82ca6da8f35917.webp)

Agentic Workflows

Agent plans and execute the workflow based on the goal, context and feedback

User Query → Plan → Execute → Replan
Replan: Yes → Plan; No → Response

Controllable and Reliable like workflows, Autonomous and Flexible like agents

### Workflow Agents vs Agentic Workflow

A workflow agent runs a predefined workflow, so the workflow is in control. In an agentic workflow, the agent plans and runs a dynamic workflow and is in control.

[2:16](https://www.youtube.com/watch?v=sl3icG-IjHo&t=136s) [A Venn diagram distinguishes a predefined workflow run by an agent from a dynamic workflow planned and run by an agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/38a994e4b3bd3fe61af35c7bc18442deb292ec670054a4a71ba37b8c8be5c2af.webp)

Workflow Agents vs Agentic Workflow

[ Workflow is in control ]
Pre-defined workflow is run by Agent

Workflow
Agent
Workflow Agent
Agentic Workflow

Agent plans and runs the dynamic workflow
[ Agent is in control ]

[2:54](https://www.youtube.com/watch?v=sl3icG-IjHo&t=174s) [A Venn diagram contrasts a predefined workflow run by an agent with a dynamic workflow planned and run by an agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/b953bf81cee0028c7fdd82176c838b6488fbbb8308ae7ba6af378274d5e40638.webp)

Workflow Agents vs Agentic Workflow

[ Workflow is in control ]
Pre-defined workflow is run by Agent

Workflow
Agent
Workflow Agent
Agentic Workflow

Agent plans and runs the dynamic workflow
[ Agent is in control ]

Interface text:
AIE
AI Engineer World's Fair
Microsoft
smol ai

### Agentic Spectrum

The spectrum places agentic workflows between workflow agents and autonomous agents. Miraje says agentic workflows generally have more agenticness than workflow agents.

[3:07](https://www.youtube.com/watch?v=sl3icG-IjHo&t=187s) [A spectrum moves from augmented-LLM workflows through workflow agents and agentic workflows to autonomous agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/0a1f8157c329b67348422d53e0308663f6dbcb6a5c13a13bde0fa4272e6b9a68.webp)

Agentic Spectrum

Augmented-LLM Workflows
Workflow Agents
Agentic Workflows
Autonomous Agents

Increasing Agentic-ness

“Rather than arguing over which work to include or exclude as being a true agent, we can acknowledge that there are different degrees to which systems can be agentic.”

- Andrew Ng

### Why Agentic Workflows Matter

A four-row illustration places agentic workflows after LLMs, augmented LLMs and workflows. Miraje says enterprises can use them to automate workflows at scale and build on existing microservices.

[3:22](https://www.youtube.com/watch?v=sl3icG-IjHo&t=202s) [The slide lists control, reliability, predictability, automation at scale, and use of existing enterprise microservices.](https://assets.ai.engineer/sites/aiecode2025/sha256/15a2182ac8671251875b2499d401083625bbff1dfba1183a3d6c96a409a151b2.webp)

Why agentic workflows matter?

• Control, Reliability and Predictability
• Automate workflows at scale
• Build on existing enterprise microservices

LLM
Augumented LLM
Workflows
Agentic Workflows

### Where to Begin?

Miraje calls for a shift from ReAct-based agents to proactive agents. The slide contrasts a tangled reactive system with an ordered proactive one.

[4:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=249s) [A slide calls for a shift from reactive to proactive systems, illustrated by tangled and ordered halves of a diagram.](https://assets.ai.engineer/sites/aiecode2025/sha256/75a3231bdacafb5a06e3b0a9d1f058caad71eb2f0e233e7f642d17edebac551f.webp)

Where to begin?

Shift from ReActive to Pro-Active

( Great life philosophy too, by the way !)

Reactive Systems
Proactive Systems

### Design Patterns for Agentic Workflows

The diagram connects an agent to tools, memory, action and planning. Miraje emphasizes subgoal decomposition: breaking a goal into simpler steps.

[4:29](https://www.youtube.com/watch?v=sl3icG-IjHo&t=269s) [Diagram connecting an agent to tools, memory, action and planning patterns.](https://assets.ai.engineer/sites/aiecode2025/sha256/9958187d246fc322c1b73039299c9f5521656cdbee6ad577e685fc8b61a94914.webp)

Design Patterns for Agentic Workflows

Short term
Long term
Memory

Calendar
Calculator
Web Search
Code Interpreter
more...
Tools

Agent
Action

Planning
Reflection
Self-critics
Chain of Thoughts
Subgoal Decomposition

Credit: Harrison Chase

### Recommended Agentic Architectures

Plan-and-Solve targets simple sequential tasks, ReWOO variable-dependent workflows, and LLM Compiler parallel work with multiple dependencies. FactSet is adapting LLM Compiler; LangChain’s guide offers code.

[4:57](https://www.youtube.com/watch?v=sl3icG-IjHo&t=297s) [Recommendations for Plan-and-Solve, ReWOO and LLM Compiler, with their suggested workflow types.](https://assets.ai.engineer/sites/aiecode2025/sha256/ba13da362d24de795abc70496ae6e6c6ac5db89cd51b52a7c68bf8ed19cb6e1e.webp)

Recommended Agentic Architectures

1 Plan-and-Solve
Simple and Sequential multi-step tasks

2 ReWOO
Context Aware, Sequential, Variable-dependent workflows

3 LLM Compiler
Fast, Parallel and Multiple Dependencies

Plan + Execute Agents

https://blog.langchain.dev/planning-agents/

### Agentic Workflow Architecture

FactSet adapts LLMCompiler: a blueprint guides the planner and executor, while a joiner combines task outputs. The workflow can replan or format a response, using tools connected to enterprise microservices.

[5:22](https://www.youtube.com/watch?v=sl3icG-IjHo&t=322s) [A workflow from blueprint generator through planner, executor, joiner and response formatter, connected to tools and microservices.](https://assets.ai.engineer/sites/aiecode2025/sha256/8d232d97a8709f1d4dda21227a10f0fb496d1b027de0c7841b14e316b5825ad3.webp)

Agentic Workflow Architecture

User
User Query
Response

Blueprint Generator
Blueprint
Planner
Plan
Executor
Tool Output
Joiner
Joined Output
Response Formatter
Replan
Yes
No

Tools
Retrieval
Reasoning
Web Search
Reporting
MCP Tool Server

m:n

Microservices
Service 1
Service 2
Service 3
Service n
Enterprise Infrastructure

[6:05](https://www.youtube.com/watch?v=sl3icG-IjHo&t=365s) [Workflow from user query through blueprint generation, planning, execution, joining, replanning and response formatting, with tools connected to enterprise microservices.](https://assets.ai.engineer/sites/aiecode2025/sha256/3fca242b5aceca65febfbd79563714a607287eacfd88ad9255ab076383baa63a.webp)

Agentic Workflow Architecture

User
User Query
Response

Blueprint Generator
Blueprint
Planner
Plan
Executor
Tool Output
Joiner
Joined Output
Response Formatter
Replan
Yes
No

Tools
Retrieval
Reasoning
Web Search
Reporting
MCP Tool Server

m:n

Microservices
Service 1
Service 2
Service 3
Service n
Enterprise Infrastructure

Interface text:
AIE
aws
World's Fair

### Tools

An MCP tool server connects tools to enterprise microservices in an m:n relationship. Purpose, descriptions and input/output contracts help the agent select and use tools; validation checks catch obvious errors.

[6:41](https://www.youtube.com/watch?v=sl3icG-IjHo&t=401s) [Guidance to build tools around enterprise microservices, use an MCP tool server, describe tool purpose and contracts, and add validation checks; a diagram shows tools connected to microservices.](https://assets.ai.engineer/sites/aiecode2025/sha256/111472b654006131a00456338b2525135005a703c9ac24a9da08624260d18661.webp)

Tools

• Build tools empowered by enterprise Microservices
• Follow a standard – Build MCP Tool Server
• Provide clear Tool details to agent
  ○ Tool Purpose – What tools to select?
  ○ Detailed Descriptions – When to invoke these tools?
  ○ Input / Output Contracts - How to use the tools?
• Add validation checks for tools to catch obvious errors

Tools
Retrieval
Reasoning
Web Search
Reporting
MCP Tool Server

m:n

Microservices
Service 1
Service 2
Service 3
Service n
Enterprise Infrastructure

[7:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=429s) [Guidance on tool purpose, descriptions, input and output contracts, and validation, with a diagram linking MCP tools to enterprise microservices.](https://assets.ai.engineer/sites/aiecode2025/sha256/96c7b39b30dc792151454d0eeabce772f9aa80ef9650dd7f1c05f18f7bef0bc8.webp)

Tools

• Build tools empowered by enterprise Microservices
• Follow a standard – Build MCP Tool Server
• Provide clear Tool details to agent
  ○ Tool Purpose – What tools to select?
  ○ Detailed Descriptions – When to invoke these tools?
  ○ Input / Output Contracts - How to use the tools?
• Add validation checks for tools to catch obvious errors

Tools
Retrieval
Reasoning
Web Search
Reporting
MCP Tool Server

m:n

Microservices
Service 1
Service 2
Service 3
Service n
Enterprise Infrastructure

Other visible text:
AIE
AI Engineer World's Fair
aws

### Blueprint

A blueprint gives the planner natural-language workflow steps and limits the tools it sees. Miraje says this reduces planner overload and makes agent behavior easier to interpret and discuss with non-technical collaborators.

[8:25](https://www.youtube.com/watch?v=sl3icG-IjHo&t=505s) [A blueprint gives the planner natural-language workflow steps, reducing cognitive load and helping control planning, interpret behavior, and collaborate.](https://assets.ai.engineer/sites/aiecode2025/sha256/3b60ceae14a7a602713e2f615002ebb349d891237bca6cbf103e982134140306.webp)

Blueprint

A series of steps for a workflow as per tool capabilities in natural language provided as input to planner

Ok...but Why ?

• Reduces cognitive load of the planner
• Achieve finer control over the task planning
• Limits the in-context tools for planner
• Makes interpreting agentic behavior easier
• Facilitates collaboration with non-technical collaborators

18

### Workflow: Prepare for NVIDIA’s earnings call

A four-step blueprint describes retrieval, reasoning, and reporting tasks. The plan turns them into function calls, passing earlier task outputs into question generation and the final report.

[9:39](https://www.youtube.com/watch?v=sl3icG-IjHo&t=579s) [A four-step earnings-call blueprint for retrieval, reasoning, and reporting, paired with corresponding function calls.](https://assets.ai.engineer/sites/aiecode2025/sha256/afd05262f04ae77f1575a9ff7f7611c0cc8f51d011777378e32abab9e8184a5d.webp)

Workflow - Prepare for NVIDIA's earning call

Blueprint
Tool: Task

1. Retrieval: Summarize NVIDIA's previous earnings call.
2. Retrieval: Gather NVIDIA's latest EPS and revenue.
3. Reasoning: Suggest questions for the next earnings call based on all the information fetched
4. Reporting: Generate a comprehensive report from fetched information

Plan
Tool.Function (params)

1. Retrieval.summarize_earning_call(company=“NVIDIA”, date=“Q2_2025”)
2. Retrieval.get_financial_metrics(query= “NVIDIA’s latest EPS and revenue”]
3. Reasoning.suggest_quesions(context=[task_1, task_2])
4. Report.generate_report(context= [task_1, task_2, task_3])

Simplified Example of Blueprint and Plan

### Response

The earnings-call workflow turns a general response into a structured report with financial metrics, targeted questions, and an executive summary.

[10:31](https://www.youtube.com/watch?v=sl3icG-IjHo&t=631s) [A basic NVIDIA earnings-call response compared with a structured report.](https://assets.ai.engineer/sites/aiecode2025/sha256/39f9938778c8528d6cebcf0056e8c0c38935c95b6e69eb28589cccf1e7fca5fc.webp)

Response

Before

In Q1 FY26, NVIDIA reported strong revenue growth, approximately $44 billion, driven largely by Data Center expansion, although exact margin details were affected by some export issues. They also returned a significant amount of cash to shareholders. The company expects around $45 billion revenue next quarter, despite facing ongoing challenges like export controls, particularly related to China.

To prepare for the earnings call, consider asking about recent revenue trends, especially in Data Center. It might also help to discuss how NVIDIA plans to handle geopolitical challenges. Additionally, focusing on their product strategies, like the Blackwell architecture, and asking broadly about their competitive landscape could provide useful insights.

After

1. Earnings Call Recap
• Q1 FY26 revenue: $44B (+69% YoY); Data Center: $39B (+73%).
• Non-GAAP gross margin: 71.3% (excl. $4.5B export-control charge).
• Returned $14.3B to shareholders. Q2 guidance: ~$45B (±2%), ~$8B China impact.

2. Latest Financial Metrics (TTM)
• EPS (Non-GAAP): $2.94
• Revenue (TTM): $130.5B

3. Key Questions for Next Call
1. China Impact: Strategy to offset H20 embargo revenue losses?
2. Blackwell Ramp: Capacity & margin targets as Blackwell hits ~70% of DC compute?
3. Competitive Edge: Response to ASIC and open-source competition?

4. Executive Summary
NVIDIA demonstrates robust growth, resilient margins, and strategic agility amidst China-related headwinds. Focus areas: roadmap clarity, operational flexibility, competitive positioning.

21

### Evals are Key!

Miraje argues that the workflow depends on proper evaluations and recommends maintaining an eval framework with component and end-to-end tests.

[10:47](https://www.youtube.com/watch?v=sl3icG-IjHo&t=647s) [Section slide stating that evaluations are key.](https://assets.ai.engineer/sites/aiecode2025/sha256/c2213e0e82cd8d4a03d260ac3b05d67e6077bc1cf1845f6e11a79ec8ddabdf1a.webp)

Evals are Key!

### Evaluation Framework

Build and maintain an eval framework with component and end-to-end evals. Use code-based checks, LLM-as-a-judge or human review, and measure the qualities that matter to you.

[10:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=655s) [Evaluation types, techniques and metrics arranged in three columns.](https://assets.ai.engineer/sites/aiecode2025/sha256/7395411de80b4de13c9345b183bf9d768aba15d1f91207acc10a52bd779dd17f.webp)

Invest in building and maintaining Eval Framework

Types
Component
End-to-End

Techniques
Code-based
LLM-as-a-judge
Human-in-the-loop

Metrics
Correctness
Relevancy
Completeness
Efficiency

### Wrapping up

Miraje recommends ETL for fixed, repeated tasks and cautions against agentic workflows when a use case cannot be captured as a workflow, strict determinism is essential, or latency and cost are constrained.

[11:53](https://www.youtube.com/watch?v=sl3icG-IjHo&t=713s) [Section slide reading Wrapping up.](https://assets.ai.engineer/sites/aiecode2025/sha256/306b1a354e9a0eb8e7693278e092b176f7121743c3e6cf22b7e0719639af2262.webp)

Wrapping up

### Key Takeaways

An agent plans and runs the workflow. Miraje recommends subgoal decomposition and plan-and-execute, tools that complement existing microservices, adapting the architecture, and treating evals as first-class citizens.

[12:31](https://www.youtube.com/watch?v=sl3icG-IjHo&t=751s) [Takeaways on agentic workflows, planning, tools, architecture and evaluations.](https://assets.ai.engineer/sites/aiecode2025/sha256/e41c9ea3eb237b930f0cd0d930f6cd342d426bcb2628d83ef7f81ad1e11ef206.webp)

Key Takeaways

• Agentic Workflow is planned and run by agent
• Agentic workflows bring reliability of workflows at scale
• Planning by Subgoal decomposition is key design pattern
• Plan-and-Execute is key agentic architecture
• Build your tools to complement existing microservices
• Modify the architecture to solve your problems !
• Treat your evals like first class citizen

[13:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=789s) [Seven takeaways on agentic workflows, planning, architecture, tools, and evaluations.](https://assets.ai.engineer/sites/aiecode2025/sha256/2d77cf5315c8f1b9841fc54cfd8e429c6aaf5b9cf225f1f9fc2d49607d5a307b.webp)

Key Takeaways

• Agentic Workflow is planned and run by agent
• Agentic workflows bring reliability of workflows at scale
• Planning by Subgoal decomposition is key design pattern
• Plan-and-Execute is key agentic architecture
• Build your tools to complement existing microservices
• Modify the architecture to solve your problems !
• Treat your evals like first class citizen

### Thank You

Miraje closes with QR codes for LinkedIn and the AI Blindspot Podcast, alongside his X handle.

[13:53](https://www.youtube.com/watch?v=sl3icG-IjHo&t=833s) [Closing slide with QR codes for a professional profile and podcast, and a social media contact.](https://assets.ai.engineer/sites/aiecode2025/sha256/50f2b423a4709426f1f9238424a6192f06b51d5613801fa357ae39aabee95eab.webp)

Thank You

Keep in touch!
X @yogi_miraje

LinkedIn
AI Blindspot Podcast

### Recommended Agentic Architectures

Plan-and-Solve fits simple sequential tasks; ReWOO handles context-aware, variable-dependent workflows; LLM Compiler targets fast, parallel work with multiple dependencies. Miraje points to LangChain as a starting place.

[14:41](https://www.youtube.com/watch?v=sl3icG-IjHo&t=881s) [Three recommended architectures: Plan-and-Solve, ReWOO, and LLM Compiler, with their workflow characteristics.](https://assets.ai.engineer/sites/aiecode2025/sha256/e1a1a3c20602f149cd47d773819490be1a4c44a7ede82c59c0e290de894a1063.webp)

Recommended Agentic Architectures

1 Plan-and-Solve
Simple and Sequential multi-step tasks

2 ReWOO
Context Aware, Sequential, Variable-dependent workflows

3 LLM Compiler
Fast, Parallel and Multiple Dependencies

Plan + Execute Agents
https://blog.langchain.dev/planning-agents/

AIE
Microsoft
smol ai
AI Engineer World's Fair

## Full transcript

[0:00](https://www.youtube.com/watch?v=sl3icG-IjHo&t=0s) [upbeat music] Hi everyone, I'm Yogi.

[0:17](https://www.youtube.com/watch?v=sl3icG-IjHo&t=17s) I work at FactSet, a financial data and software company. And today I'll be sharing some of my experience while building agent.

[0:27](https://www.youtube.com/watch?v=sl3icG-IjHo&t=27s) In last few years, we have seen tremendous growth in AI, and especially in last couple of years, we are on exponential curve of intelligence growth.

[0:38](https://www.youtube.com/watch?v=sl3icG-IjHo&t=38s) And yet, it feels like when we are develop AI applications, driving a monster truck through a crowded mall with the tiny joysticks. So AI applications have not seen its ChatGPT moment yet.

[0:53](https://www.youtube.com/watch?v=sl3icG-IjHo&t=53s) There are many reasons why agents don't behave, but probably one reason that strikes out is it misses the right context.

[1:04](https://www.youtube.com/watch?v=sl3icG-IjHo&t=64s) And in case of enterprises, often it means that it does not have knowledge of enterprise-specific workflows.

[1:13](https://www.youtube.com/watch?v=sl3icG-IjHo&t=73s) But before that, we will see some common context, and just like agents, human also need a common context. So let's start with some key definitions.

[1:24](https://www.youtube.com/watch?v=sl3icG-IjHo&t=84s) So as you know, LLMs are limited by the knowledge at the time of training, so we enhance their functionality by

[1:34](https://www.youtube.com/watch?v=sl3icG-IjHo&t=94s) increase it by tool. And when you combine this LLM with tool and memory, we call it augmented LLM. When you place this augmented LLM on a static and predefined path, we call it a workflow.

[1:48](https://www.youtube.com/watch?v=sl3icG-IjHo&t=108s) And if these augmented LLMs have high autonomy and feedback loop, we call it as an agent.

[1:56](https://www.youtube.com/watch?v=sl3icG-IjHo&t=116s) Now, workflows are controllable and reliable, while agents have flexibility and they are highly autonomous. So the question is: Can we get best of both worlds? So the answer is yes.

[2:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=129s) With agentic workflows, we can plan and execute the workflows based on the goal, context, and feedback.

[2:19](https://www.youtube.com/watch?v=sl3icG-IjHo&t=139s) I see these terms being used very loosely and at times interchangeably, so I would like to make a key distinction between workflow agent and agentic workflow.

[2:32](https://www.youtube.com/watch?v=sl3icG-IjHo&t=152s) Workflow agent is a predefined workflow run by agent,

[2:37](https://www.youtube.com/watch?v=sl3icG-IjHo&t=157s) while agentic workflow is a workflow planned and run by an agent.

[2:44](https://www.youtube.com/watch?v=sl3icG-IjHo&t=164s) I know these terms are, like, quite confusing and in AI we are very bad at naming things, so if you are confused, don't worry. In case of workflow agent, just remember that workflow is in control and workflow is static.

[2:58](https://www.youtube.com/watch?v=sl3icG-IjHo&t=178s) In case of agentic workflow, agent is always in control and the workflow is dynamic.

[3:07](https://www.youtube.com/watch?v=sl3icG-IjHo&t=187s) It is also important to view these systems, uh, as agentic system, as Andrew Ng pointed out correctly. On agentic spectrum, agentic workflows have more agenticness than workflow agents, generally speaking.

[3:25](https://www.youtube.com/watch?v=sl3icG-IjHo&t=205s) So why all of this matter? Apart from control, reliability, predictability for enterprises, agentic workflows provide a way to automate the workflows at scale. And perhaps most important thing is

[3:45](https://www.youtube.com/watch?v=sl3icG-IjHo&t=225s) enterprises can use their existing enterprises, uh, microservices to build on top of it. And in some cases,

[3:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=235s) these enterprises have invested years, not, uh, if not decades.

[4:02](https://www.youtube.com/watch?v=sl3icG-IjHo&t=242s) So before diving deep, I would like to say that even though I'm speaking in terms of enterprise context here, the concepts are generally applicable.

[4:11](https://www.youtube.com/watch?v=sl3icG-IjHo&t=251s) So where do we begin? In last few years, the focus really has been on the ReAct-based agent. And in building agentic workflow, we need to move on from ReAct-based agent to proactive agents.

[4:25](https://www.youtube.com/watch?v=sl3icG-IjHo&t=265s) By the way, great philosophy for life as well.

[4:30](https://www.youtube.com/watch?v=sl3icG-IjHo&t=270s) So for building agentic workflows, you need tools, memory, and reflection. But more importantly, you will need a design pattern called planning by sub-goal divi-- sub-goal division,

[4:46](https://www.youtube.com/watch?v=sl3icG-IjHo&t=286s) sometimes also referred as a task decomposition. And it is just a fancy way of saying that take your goal and break it down into simpler steps.

[4:57](https://www.youtube.com/watch?v=sl3icG-IjHo&t=297s) Here are some specific agentic architecture and research papers that you will find useful, and each of that has, like, its own pro and cons. And LangChain has done fantastic job of, uh, creating a blog from this and, uh, also gi-given the code.

[5:14](https://www.youtube.com/watch?v=sl3icG-IjHo&t=314s) So I highly recommend checking it out. So how does it look in practice? So what-- In, uh, in FactSet what we have done is we are taking this LLMCompiler architecture and trying to adapt for our problems.

[5:29](https://www.youtube.com/watch?v=sl3icG-IjHo&t=329s) And you can see some components here, uh, that you also find that in your organization, uh, microservices, and you build tools around those microservices. And when a user question ask, it goes to blueprint generator, and I will get to that in a bit.

[5:45](https://www.youtube.com/watch?v=sl3icG-IjHo&t=345s) But consider it as a, like a high level plan.

[5:49](https://www.youtube.com/watch?v=sl3icG-IjHo&t=349s) What we call it is a blueprint that gets fed to planner. Planner is your low-level task planner. It gives the plan to the executor, and executor is supposed to execute it.

[6:03](https://www.youtube.com/watch?v=sl3icG-IjHo&t=363s) And joiner combines the outputs from different tasks.

[6:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=369s) Based on your replanning logic, either you do replanning again or you just, like, terminate and give the response back to the user.

[6:18](https://www.youtube.com/watch?v=sl3icG-IjHo&t=378s) Sometimes you also set some recursion limits so that your agent just, like, doesn't go into loop.

[6:24](https://www.youtube.com/watch?v=sl3icG-IjHo&t=384s) On LangGraph, we are using each of these component as a nodes. So blueprint generator, planner, executor, and joiner are all nodes on the LangGraph.

[6:37](https://www.youtube.com/watch?v=sl3icG-IjHo&t=397s) When building these, uh, tools in, in your enterprises a-around microservices, probably this is where you will spend most of your time.

[6:47](https://www.youtube.com/watch?v=sl3icG-IjHo&t=407s) And it's important to consider how this relation between tools and microservices goes. And here, the relationship is definitely not one-to-one or n-to-n, it's m-to-n. It's up to you how you want to design your tools according to your microservices so that your agent knows how to use this tool.

[7:07](https://www.youtube.com/watch?v=sl3icG-IjHo&t=427s) Perhaps this is, like, the most key point here that you need to make, really put yourself into agent's shoes so that agent really understand what tool to use, and it has that knowledge of your microservices.

[7:22](https://www.youtube.com/watch?v=sl3icG-IjHo&t=442s) Always follow standard. I know MCP is everyone's favorite, so build a MCP tool server for your tools. And for providing the tool details, just think from agent's point of view that you need to provide a tool purpose, description, and input/output contracts.

[7:40](https://www.youtube.com/watch?v=sl3icG-IjHo&t=460s) So tool purpose will help you what tools to be selected, tool detail description will tell you when these tools need to be invoked, and input/output contracts will tell you how to use this tool.

[7:53](https://www.youtube.com/watch?v=sl3icG-IjHo&t=473s) And lastly, add some validation checks which acts as a brake for your agent.

[8:00](https://www.youtube.com/watch?v=sl3icG-IjHo&t=480s) Now, I would like to a little bit zoom in into this blueprint, uh, because this is, like, one of the key architecture change that we made. A blueprint is just a series of steps for workflow as per tool capabilities in natural language.

[8:15](https://www.youtube.com/watch?v=sl3icG-IjHo&t=495s) And it gets fed to planner, but why we are doing it? So what we realized was,

[8:23](https://www.youtube.com/watch?v=sl3icG-IjHo&t=503s) planner really gets cognitively, uh, loaded, uh, when you try to just put too much onto it. So introducing a blueprint, which is just a natural language of breaking down of a task, is very helpful.

[8:38](https://www.youtube.com/watch?v=sl3icG-IjHo&t=518s) But we also notice that it brings a lot of other benefits as well. For example, it achieves the finer control over task planning. It limits the in-context tool for the planner.

[8:50](https://www.youtube.com/watch?v=sl3icG-IjHo&t=530s) So when blueprint, you can select what tools to be... need to be given to the planner, and sometimes this, uh, planners has lot of tool description, and you run all sort of problems as context window limit and planner gain, uh, getting very much overloaded.

[9:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=549s) So using blueprint, you can limit what tools really goes to the, uh, planner, and thus, uh, it really helps in, in the planning.

[9:21](https://www.youtube.com/watch?v=sl3icG-IjHo&t=561s) It also helps interpreting the agentic behavior. And lastly, when you need to collaborate with non-technical, uh, people, it's, like, really helpful because natural language is less intimidating.

[9:34](https://www.youtube.com/watch?v=sl3icG-IjHo&t=574s) Let's see a concrete example. So in financial research, preparing for an, uh, company's earnings call is a common workflow. So this is a very, very simplified version of a workflow of preparing for a company's earnings call.

[9:50](https://www.youtube.com/watch?v=sl3icG-IjHo&t=590s) And for example, we are showing you preparing for NVIDIA's earnings call.

[9:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=595s) Now, you can see in the blueprint, there is a tool and there is task, and in the plan, there is tool and the function call.

[10:03](https://www.youtube.com/watch?v=sl3icG-IjHo&t=603s) So how does it look in, in the blueprint is you have two tools, and then your first step is summarizing the NVIDIA's previous earnings call. And the next step is retrieval, gathering some of the financial data from, uh, for NVIDIA.

[10:17](https://www.youtube.com/watch?v=sl3icG-IjHo&t=617s) And then your reasoning, suggesting some questions for the earnings call. And finally, reporting, uh, generate a comprehensive report from the all the information.

[10:26](https://www.youtube.com/watch?v=sl3icG-IjHo&t=626s) And there are corresponding function calls, and as you can see, context is being fed from a task.

[10:34](https://www.youtube.com/watch?v=sl3icG-IjHo&t=634s) A, a concrete, uh, example of the response is before you implement agentic workflow, the response is very much vanilla. But after this, it can easily capture your workflow and give a very structured response.

[10:49](https://www.youtube.com/watch?v=sl3icG-IjHo&t=649s) So whatever we talked about, none of this would really work without writing a proper evals. So always make sure to invest and build and maintain your eval framework. You should have at least component and end-to-end evals.

[11:04](https://www.youtube.com/watch?v=sl3icG-IjHo&t=664s) You should really use the correct techniques like code-based, LLM-as-a-judge, human-in-the-loop. And more importantly, write evals for metrics that you really care for.

[11:15](https://www.youtube.com/watch?v=sl3icG-IjHo&t=675s) Aspect-based eval is something like we should really, uh, think about. And for example, for blueprint, uh, you can check an aspect like how many, uh, blueprint, whether it resembles a golden blueprint or not, and you can use LLM-as-a-judge.

[11:31](https://www.youtube.com/watch?v=sl3icG-IjHo&t=691s) If you want to see whether tools are selected correct or not, you should leverage code-based evals.

[11:37](https://www.youtube.com/watch?v=sl3icG-IjHo&t=697s) If you want to check whether plan is in line with the blueprint or not, LLM-as-a-judge probably the right technique. And for some cases, leveraging human-in-the-loop is good because report formatting, uh, that's the best approach to deal with report formatting.

[11:55](https://www.youtube.com/watch?v=sl3icG-IjHo&t=715s) So when not to use agentic workflows. So in some cases, definitely agentic workflow doesn't make sense. In case of fixed and repeated task, just probably go for ETL pipelines.

[12:06](https://www.youtube.com/watch?v=sl3icG-IjHo&t=726s) If your workflow cannot be really captured, uh, you cannot really capture use case in workflows, agentic workflows are probably not worth. And if deterministic outcome is paramount in cases of strict compliance and a safety-critical context, uh, you should probably n- should not go with agentic workflow.

[12:24](https://www.youtube.com/watch?v=sl3icG-IjHo&t=744s) And in case of la- low latency and cost-constrained environment also, uh, you should probably try to avoid agentic workflow.

[12:32](https://www.youtube.com/watch?v=sl3icG-IjHo&t=752s) So wrapping up, some learnings. Um, start with simple blueprints. Work wir- work, uh, work your way up, uh, building a complex RAG system for the blueprints. Use blueprint to reduce the in-context, uh, tools and provide the high level plan to the planner.

[12:52](https://www.youtube.com/watch?v=sl3icG-IjHo&t=772s) Design tools from agent point of view. Um, always aim for the tool usage simplicity. Implement safety guardrails and evals observability and all the good software engineering, uh, that should, uh, help you a lot.

[13:11](https://www.youtube.com/watch?v=sl3icG-IjHo&t=791s) And from the whole, uh, presentation, the key takeaways are agentic workflow is planned and run by agent. Agentic workflows bring the reliability

[13:21](https://www.youtube.com/watch?v=sl3icG-IjHo&t=801s) at scale, and planning by sub-goal division is a key design pattern. Plan-and-execute is a key agentic architecture.

[13:30](https://www.youtube.com/watch?v=sl3icG-IjHo&t=810s) And build your tools to complement your microservices. Always try to leverage your microservices in the tools, and modify your architecture to solve the problems. Don't really shy away from changing, taking research paper and experimenting on it.

[13:49](https://www.youtube.com/watch?v=sl3icG-IjHo&t=829s) And finally, treat your evals like first-class citizen.

[13:54](https://www.youtube.com/watch?v=sl3icG-IjHo&t=834s) And with that, thank you very much for your time. [audience applauding]

[14:02](https://www.youtube.com/watch?v=sl3icG-IjHo&t=842s) All right. Uh, thank you. Any questions? We have a little bit of time to spare.

[14:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=849s) I have a question.

[14:09](https://www.youtube.com/watch?v=sl3icG-IjHo&t=849s) Sure.

[14:10](https://www.youtube.com/watch?v=sl3icG-IjHo&t=850s) Um, do you, um, have, uh, in top of your mind any like, uh, GitHub project or reference that we can follow?

[14:19](https://www.youtube.com/watch?v=sl3icG-IjHo&t=859s) Sure, sure. So if you just go back here, um, I kind of, uh, shared some of the

[14:28](https://www.youtube.com/watch?v=sl3icG-IjHo&t=868s) links, um, for the LangChain. It ha- it should have all the code for this research paper, and that's probably the most, you know, best place to start with this plan-and-execute kind of agents.

[14:42](https://www.youtube.com/watch?v=sl3icG-IjHo&t=882s) Thank you.

[14:43](https://www.youtube.com/watch?v=sl3icG-IjHo&t=883s) Yeah.

[14:46](https://www.youtube.com/watch?v=sl3icG-IjHo&t=886s) Any other questions? Uh, all right. Um-

[14:52](https://www.youtube.com/watch?v=sl3icG-IjHo&t=892s) Yeah.

[14:52](https://www.youtube.com/watch?v=sl3icG-IjHo&t=892s) I guess one question I would have for you-

[14:53](https://www.youtube.com/watch?v=sl3icG-IjHo&t=893s) Sure

[14:54](https://www.youtube.com/watch?v=sl3icG-IjHo&t=894s) ... is the... When you talk about MCP and other forms of orchestration, what do you foresee being the, the primary method of orchestration going forward? Is it gonna be LangGraph or some other...

[15:07](https://www.youtube.com/watch?v=sl3icG-IjHo&t=907s) Yeah, I think the answer is probably, like, everything. MCP, you use it so that you provide a standard across the org, and MCP will really help for organization to, you know, build once, use it everywhere.

[15:21](https://www.youtube.com/watch?v=sl3icG-IjHo&t=921s) Uh, you can have... Oftentimes in organizations we see that, uh, people just like trying to just use this functionality in different AI apps. But if you can build an MCP around it, you can keep using it.

[15:33](https://www.youtube.com/watch?v=sl3icG-IjHo&t=933s) And obviously for orchestration, LangGraph is great, and whatever the other tools that you find to solve your problem, that will be also. Um, so the answer is probably there will be like multiple things that is useful.

[15:45](https://www.youtube.com/watch?v=sl3icG-IjHo&t=945s) It depends on your use case, what is the, uh, most optimal framework that you want to use.

[15:50](https://www.youtube.com/watch?v=sl3icG-IjHo&t=950s) Amazing. Thank you so much, Yogi. [outro music]
