AI Engineer Summit 2025
Stop Guessing: Build Robust AI with Layered CoT
Read the talk
Layered CoT: Verify Each Step Before Building on It
Specialized agents can divide a complex task, but their intermediate outputs still need checking. Layered CoT adds verification before each reasoning step becomes the basis for the next.
From a talk by Manish Sanwal
Divide the task among specialized agents
Consider a self-driving car: one agent detects pedestrians, another reads traffic signals, and a third selects a route. Each handles a distinct part of a larger task that requires coordination. This is Manish Sanwal’s opening illustration of a multi-agent system: a collection of specialized agents working together rather than one monolithic system handling everything.
The accompanying diagram places agents alongside orchestration and human oversight, connected to prompt context, an LLM, and tools. Specialization gives each agent a narrower task to tune. Modularity also lets developers update one agent without replacing the whole system. Sanwal argues that this supports flexibility and scalability, and that other agents can sometimes compensate when one encounters a problem; compensation depends on how the system is coordinated.
Dividing the work addresses who performs each task. Adding Chain-of-Thought, or CoT, addresses how an agent explains its output: it produces intermediate steps rather than only a conclusion. Those steps create places to inspect the work—and, eventually, to verify it before another step depends on it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make intermediate steps available for inspection
A detailed prompt can supply extensive context while still requesting only a final answer. The resulting output does not necessarily explain how the model reached its conclusion. CoT prompting instead asks for intermediate reasoning steps. The slide makes the distinction concrete: one path connects Input directly to Output; the other passes through three Thought boxes.
Breaking a complex problem into manageable steps gives a reader more to inspect. If an intermediate output contains a mistake, a developer can adjust the prompt or the surrounding process. That is an opportunity for debugging, not yet an automatic correction mechanism: someone or something still has to detect the mistake and intervene.
An inspectable explanation is not a guaranteed account of the model’s internal computation. Sanwal describes CoT as making reasoning visible and verifiable; the useful engineering interpretation is that generated intermediate outputs can be checked. Research on unfaithful explanations in Chain-of-Thought prompting shows why that distinction matters: explanations can omit influences that affected the answer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Where an unchecked chain can fail
Making intermediate steps visible leaves several failure modes unresolved:
- Prompt sensitivity. Small changes in wording or context can produce substantially different reasoning sequences, complicating reproducibility.
- Missing feedback. Ordinary stepwise generation does not itself include a mechanism that checks and corrects each step as it appears.
- Error propagation. A flawed early inference can become an assumption for later steps. Without ongoing checks, correction waits until inference has finished.
- Missed dependencies. A problem with multiple interdependent factors can be oversimplified if the sequence omits variables or fails to connect them.
The common weakness is that producing a sequence does not establish that its intermediate claims are valid.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate, verify, then advance
Layered CoT inserts verification between generating a step and allowing subsequent reasoning to depend on it. The initial output is a hypothesis to check, rather than an established premise. Sanwal describes two recurring stages:
- Generate a candidate step. The agent produces an initial piece of reasoning from the input prompt.
- Verify the candidate. Before proceeding, cross-reference that output against a structured knowledge base or an external database.
The suggested verification methods include a fact-checking algorithm, a contextual consistency check, or an ensemble of models assessing accuracy. These are alternative ways to perform the checking stage, not a specified implementation that combines all three.
Once a step is verified, the system advances to the next step and repeats the process. The intended constraint is that only accepted information influences later reasoning. This changes the dependency between steps:
| Approach | What happens before the next step? |
|---|---|
| Ordinary CoT | The model generates the preceding step. |
| Layered CoT | The preceding step is generated and checked. |
The consequential change is the verification boundary, not merely the presence of a knowledge base. The talk leaves acceptance thresholds, correction procedures, retry limits, and termination behavior unspecified; those decisions determine what happens when a candidate fails verification.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What the verification boundary is meant to improve
Checking each step creates an earlier opportunity for self-correction: a detected mistake can be addressed before it propagates through the chain. Sanwal also proposes that independent checks reduce sensitivity to prompt wording. Different prompts may still generate different candidates, but verification is intended to constrain which candidates become premises for later reasoning.
Discrete verification points can also make a system easier to audit. Instead of inspecting only a final answer, a reviewer can examine the intermediate outputs on which it depends and the checks applied to them. The intended result is a conclusion built from validated information, with a more legible path through the task.
These are proposed benefits, not measured guarantees in the recording. The companion paper’s performance discussion does not provide enough detail about tasks, model configurations, metric definitions, or evaluation procedures to establish a general accuracy or reproducibility gain. The strength of the approach therefore depends on the quality and coverage of its verification, rather than on adding a checking stage alone.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build verification into the system
Layered CoT returns to the modular system introduced at the start: specialized agents contribute to a shared task, with verification added to the reasoning workflow. Sanwal presents existing LLM tools as sufficient to implement the approach. The companion paper qualifies that feasibility with additional model interactions, external-data dependencies, and coordination overhead between agents. Integration is a system-design task, not a demonstrated turnkey feature.
The closing emphasis is on validating each inference before proceeding and making systems structured, explainable, and reliable alongside improving model capability. The conclusion slide groups the intended outcomes as verification at each step, self-correction, reduced prompt sensitivity, and repeatability and reproducibility. Those outcomes follow the same proposed design choice: give intermediate outputs an explicit check before allowing the system to build on them.
Sanwal closes by pointing to his Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems arXiv preprint. Its research direction extends the talk’s central engineering concern: trustworthy AI requires collaboration and validation within the system, with opportunities to inspect and correct its work as that work develops.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Sanwal's framework for checking intermediate reasoning against external sources and user feedback, with examples across three domains.
Further reading
Foundational research on using intermediate-reasoning examples in prompts for arithmetic, commonsense and symbolic tasks.
Experiments showing that plausible reasoning explanations can conceal the influence of biases in model inputs.
Read the complete timestamped transcript
- 0:00
True AI isn't about one giant leap of faith. It's built incrementally with every step verified and refined through collaborative effort. Hi, I'm Manish Sanwal, Director of AI at News Corp.
- 0:13
My work focuses on AI reasoning, explainability, and automation. Today, I'm excited to show you how we can build AI that just isn't smarter, but also more structured and self-correcting using Layered Chain-of-Thought with multi, Multi-Agent Systems.
- 0:33
Let's start with the basics. What are Multi-Agent Systems? In simple terms, they are collection of specialized AI agents that work together to tackle a complex task. Each agent is designed to handle a specific part of the overall problem rather than relying on massive monolith systems.
- 0:53
Take self-driving cars, for example. Instead of depending on a massive system, picture it as a team of specialized agents. One detects pedestrian, other reads traffic signal, maybe a third one which checks for the best route.
- 1:09
With each agent doing its part in harmony, the entire system becomes much more robust and efficient. The modular approach offers several concrete advantages. Specialization. Uh, each agent can be finely tuned for a specific task, leading to a better accurate, accuracy and performance.
- 1:32
Since the system is distributed, individual agents can be updated or improved without overhauling the entire system, so the system becomes flexible and scalable.
- 1:44
If one agent encounters an issue, the other can often compensate, ensuring that overall system remains reliable and fault-tolerant. By integrating these well-coordinated agents, we create a system that is inherently more robust and effective.
- 2:02
And when we add Chain-of-Thought reasoning into the mix, each agent not only performs its task, but also explains its decision-making process step by step. This com-- this combination enhances both transparency and resiliency in our AI system.
- 2:22
So what is Chain-of-Thought? Chain-of-Thought is a method that guides AI to think through the problem step by step rather than simply guessing the answers. Traditionally, when we work with Large Language Models, we provide them with a detailed prompt and ask for a final answer.
- 2:41
Even if we supply extensive context, the model often jumps directly to a conclusion without revealing how it arrived there, almost as if it's just guessing.
- 2:54
Now, imagine if instead of demanding the answer outright, we ask the model to walk us through its reasoning process, outlining every step along the way. This is the essence of Chain-of-Thought prompting.
- 3:09
By breaking down a complex problem into a series of manageable step, the model not only demonstrate how it processes the information but also exposes the path it takes to reach the conclusion.
- 3:24
It-- This approach has two key benefits. Transparency for one. We get to see each stage of reasoning process, which helps us understand how the model is tracking the problem.
- 3:38
Second, opportunity for fine-tuning and debugging. If we spot a mistake in any of the intermediate step, we can adjust the prompt or the process, allowing us to correct errors before the final answ-answer is provided.
- 3:54
So in short, Chain-of-Thought transforms the AI's internal reasoning into viable and verifiable sequence, making the entire process more interpretable and robust. In summary, instead of simply guessing, the AI follows clear logical sequence of steps.
- 4:14
This approach, Chain-of-Thought, makes the AI reasoning process transparent, but it comes with several limitations.
- 4:24
The process is highly sensitive to how the prompts are phrased. Even a slight change in wording or context can lead to a very different output. This means that two almost identical prompts might yield vastly different Chain-of-Thought, complicating both reproducibility and reliability.
- 4:46
As the AI generates its reasoning step by step, there is no built-in mechanism to verify or correct mistakes during the process. This absence of real-time feedback means that there is no error correction opportunity.
- 5:02
Each step in the chain is produced without continuous validation. If an early inference was flawed, this can cause a cascade of errors that compromises the integrity of the entire process.
- 5:15
Without ongoing checks, the model is forced to rely on initial assumptions, and the only opportunity to correct is correct it is after the inference is complete.
- 5:29
When faced with problem that involves multiple interdependent factors, Chain-of-Thought can sometime miss critical connection. The model might not fully integrate all the possible variables into its reasoning, resulting in oversimplified or incomplete conclusion.
- 5:48
In a sense, while Chain-of-Thought provides a transparent step-by-step framework for AI reasoning, it's sensitive to prompt design, lack of real-time feedback loop, and unverified reasoning These are some of the challenges that we try to address.
- 6:06
It brings us to Layered Chain-of-Thought Prompting, what I like to call Layered CoT for short. This approach is designed to overcome the limitation of standard Chain-of-Thought methods by integrating a verification step at every stage of the reasoning process.
- 6:22
It works in two steps. Step one, generation of initial thought. The AI agent begins by producing an initial thought. This is the first piece of reasoning generated from the input prompts.
- 6:36
At this stage, the model formulates an early hypothesis of the problem, and it serves as the starting point for the further reasoning. Step two, verification against the knowledge base.
- 6:49
Before moving on, the generated thought is immediately verified. This involves cross-referencing the output against a structured knowledge base or an external database. In practice, this might include a fact-checking algorithm, a consistency check through contextual reasoning, or maybe a model-- ensemble model to check for the accuracy.
- 7:14
But this verification step is really crucial. It ensures that only accurate and reliable information is allowed to influence subsequent reasoning.
- 7:25
Once the thought is verified, the process continues to the next reasoning step. This iterative process repeats... repeatedly generates a new thought, verifies it, and then process it. The chain of reasoning is thus built up step by step, with each link in the chain confirmed before the next step.
- 7:48
The benefit of this additional verification step are significant. Self-correction for one. The verification at each step allows the system to catch and correct errors early, preventing mistakes from propagating through the entire reasoning chain.
- 8:06
Second, robustness against prompt variability. Because each step is independently verified, the overall process becomes less sensitive to small changes in the input, leading to high reproducibility.
- 8:21
Each verified step ensures that the final output is built on the foundation of accurate and validated information, resulting in more trustworthy conclusions. Breaking down the reasoning into discrete verifiable step makes the AI thought process much more transparent, allowing for easier auditing and interpretation.
- 8:46
In essence, Layered Chain-of-Thought transforms the AI reasoning into robust iterative frameworks where every step is checked for accuracy. This not only mitigates the inherent weaknesses of traditional Chain-of-Thought, but also leads to more reliable, reproducible, and interpretable AI models.
- 9:10
In summary, Layered Chain-of-Thought Prompting overcomes the limitation of lay-- traditional CoT by adding verification step after each thought it generates. This method can be seamlessly implemented using existing LLM tools and integrates perfectly within the Multi-Agent Systems where each specialized agent contribute to a robu-robust system.
- 9:35
Overall, Layered CoT enhances both accuracy and reproducibility by ensuring every inference is validated before proceeding. Remember, the future of AI isn't just about building bigger models, but it's about creating systems that are structured, explainable, and reliable.
- 9:55
By prioritizing transparency, self-correc-self-corrections, collaboration, and validation, we lay the foundation for the true-- truly trustworthy AI. We have a paper published on Layered Chain-of-Thought Prompting. The link, uh, to the arXiv is below.
- 10:11
Um, I'd love to hear your thoughts on it. Thank you for your time.