# Every step you take, every call you make: the reliable agent stack — Giselle van Dongen, Restate

Giselle van Dongen, Restate

[Talk page](https://ai.engineer/talks/cI7zfqusmFU-every-step-you-take-every-call-you) · [Video](https://www.youtube.com/watch?v=cI7zfqusmFU)

## Every step you take, every call you make: the reliable agent stack

Giselle van Dongen shows how Restate gives long-running agents journal-based recovery, suspended human approvals, isolated session state, midflight signals, cancellation, and shared infrastructure controls.

### Persistent agents change the infrastructure problem

[0:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=12s)

A question-answering model can finish in seconds. A tool-using application may stay active for a session. The next step is harder: persistent, asynchronous agents that remain in an organization’s infrastructure, use tools and context, and communicate with other agents. Once work becomes long-running, stateful, and distributed, the surrounding infrastructure must survive failures and connect components that no single agent SDK owns.

![Recording frame at 123 seconds](https://ai.engineer/api/talk-essay-media/38ddc1a2daa3f3c0e70e0f80cae247daacd49ab695402af2eeb32cde88297ae7)

Recording frame at 123 seconds

Agent SDKs and memory can accelerate a proof of concept, but production systems still need retries, recovery, state consistency, communication, and operational control. **Restate** is presented as an open-source foundation for those backend concerns rather than an agent framework. Its ideas draw on Apache Flink and work by architects associated with Meta’s event infrastructure.

### Four jobs underneath the agent loop

[2:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=162s)

The infrastructure layer has four distinct jobs. **Durable execution** preserves a run through failure instead of restarting a week-long task. **Session consistency** lets thousands of conversations proceed without corrupting one another’s state. **Distributed communication** connects agents, MCP servers, and tools. **Execution control** stops work that is stuck or no longer wanted. Recovery preserves desired work; cancellation deliberately ends it.

![Recording frame at 218 seconds](https://ai.engineer/api/talk-essay-media/89a65d0cc6a0b7d85f232952fe99dde1ef250de713341234e0f0ec5e95c86958)

Recording frame at 218 seconds

Restate runs as a separate server in front of the agent service, acting somewhat like a proxy or message broker. It pushes an invocation to the service and keeps an open connection that van Dongen calls a “lifeline.” As the service works, operations emit events to Restate. Those events form a journal from which the execution can be reconstructed after a failure.

What relationship does that journal create between an ordinary request and a recoverable agent run? The flow below makes the persistence boundary visible: application work produces journal entries before later recovery depends on them.

The result is intended to feel like an ordinary application function while gaining long-running execution and state. The demonstration applies that model to a Slack research agent available to employees inside a company.

The journal is the recovery path
Restate pushes work to the agent service, records events emitted through the open connection, and uses those persisted events to resume after failure.
- Incoming request — A request targets the agent service through Restate.
- Restate server — Proxies and pushes the invocation while maintaining the connection.
- Agent service — Runs application logic and emits events as durable operations occur.
- Execution journal — Persists the event history used for recovery.
- Recovered execution — Continues from recorded progress rather than restarting the whole run.
- Incoming request → Restate server (enters)
- Restate server → Agent service (pushes invocation)
- Agent service → Execution journal (emits events)
- Execution journal → Recovered execution (replays recorded progress)

### A research run survives a failed search

[5:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=312s)

The concrete example begins when a user asks Slack, “What is new in AI?” Restate’s UI shows the corresponding execution. A planner first calls an LLM and returns proposed research subtopics. Clicking Approve unblocks the workflow, launches parallel research agents, and eventually hands their output to a writer agent. Planning, authorization, parallel research, and synthesis are separate stages of one durable run.

![Recording frame at 379 seconds](https://ai.engineer/api/talk-essay-media/92a075876394ebfdbca64e8df1a6386c702b26a1ae2397582ca4c2549372977a)

Recording frame at 379 seconds

An injected failure makes the recovery mechanism observable. One subagent completes an LLM call, begins web searches, and encounters a search API outage. The failed search retries and later completes successfully. The surrounding research does not restart because the journal already records the run’s progress; recovery returns to the failed operation rather than discarding earlier work.

In code, the application unit is an HTTP handler using the Restate SDK. Its Restate object context represents the connection to the server, and operations performed through that context create journal events. The planner’s Python LLM function becomes a durable step when wrapped in `restate.run`. If the process fails much later, the journal supplies the recorded execution point from which it can continue.

### A month-long wait without a month-long function

[8:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=492s)

Durability also handles intentional inactivity. A human approval may take weeks or a month, spanning service restarts and redeployments. Keeping a conventional function active for that entire period would tie the workflow’s lifetime to one process invocation.

Restate instead creates a durable promise in the journal. The promise marks a suspension point; Slack presents the approval button, and the function suspends while it waits. On serverless infrastructure, van Dongen says the suspended process consumes no function execution time. When approval arrives, the promise resolves and execution resumes where it stopped. The claim concerns serverless execution time during the wait, not the absence of storage or platform operating costs.

This works well for a sequence of durable steps, but a persistent agent needs more than a workflow-shaped history. It also needs an identity, memory, and a way to accept new input while work is already underway.

### Virtual objects turn sessions into stateful actors

[9:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=552s)

Restate models that persistent entity as a **virtual object**, similar to a stateful actor. Each object has a unique key such as a session ID, isolated key-value state such as message history, and handlers that can run durable functions for the session. A user can therefore add context while research is underway instead of waiting ten minutes for the first run to finish.

![Recording frame at 680 seconds](https://ai.engineer/api/talk-essay-media/8276e0c064529027abb59f499cabaea1ba9d4265de3aa6e395c45de5d35c3b87)

Recording frame at 680 seconds

Isolation alone does not prevent two messages in the same session from racing. If both handlers updated the same conversation state concurrently, one could overwrite the other. The session controller prevents this by allowing one execution at a time for a given object key; another execution waits in a queue behind it. Separate sessions can still run concurrently, but serialization within one session trades some per-session latency for consistent updates.

### Signal relevant context; cancel a new direction

[11:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=687s)

A session ID addresses persistent conversation state. A separate execution ID addresses one active run. Other processes can use that execution ID to retrieve output, send a signal containing new state, or cancel the run. Together, the two identities let a controller remember the conversation while directly contacting the particular research execution already in flight.

![Recording frame at 813 seconds](https://ai.engineer/api/talk-essay-media/2790d61e03df8760ce69fb32f835943fc7305a6f22a85c8ce9b95f405a3400e2)

Recording frame at 813 seconds

The application adds an LLM classifier on top of those control primitives. When a new Slack message arrives, the classifier asks whether it is relevant to the current research. Relevant context is sent as a signal; unrelated context causes cancellation followed by a new run. Restate supplies addressability and control, while the classifier makes the application-level routing decision.

The first follow-up—“focus on frontier models”—stays within the original “What is new in AI?” assignment. The controller classifies it as relevant and injects it into the active loop. The UI then shows the message entering the research execution, which takes the added focus into account and starts its work again. The talk does not evaluate classifier accuracy or establish precisely how much prior research is reused after this signal.

The second follow-up changes the assignment: “Forget about that. Research AI policy.” The coordinator cancels the existing run and starts a new one. Cancellation travels through the call chain, stopping spawned subagents before the controller unwinds. This can give application code an opportunity to roll back, but stopping execution does not by itself reverse external effects that have already completed.

What decides whether a live run is amended or replaced? The comparison below separates the classifier’s semantic choice from Restate’s execution mechanisms.

Routing a follow-up into an active session
The controller classifies new input, then either signals the addressed execution or cancels its call chain and starts a replacement.
- New Slack message — Additional context arrives while research is active.
- Relevance classifier — An LLM decides whether the message belongs to the current task.
- Signal active execution — Injects relevant state through the execution ID.
- Cancel call chain — Stops subagents before unwinding the controller.
- Start replacement run — Begins research for the newly requested topic.
- New Slack message → Relevance classifier (classify)
- Relevance classifier → Signal active execution (still relevant)
- Relevance classifier → Cancel call chain (new direction)
- Cancel call chain → Start replacement run (after unwinding)

### Move shared model controls into a gateway

[14:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=897s)

Production requirements rarely stop changing after the first deployment. If a newly adopted model is good but expensive, an inline LLM call offers no convenient place for organization-wide controls. Restate’s programming model allows that call to move from a local Python step into a separate handler without changing the broader durable-execution model.

![Recording frame at 994 seconds](https://ai.engineer/api/talk-essay-media/9f7b139e1e4fb6b6eb836583dd4621659e9071b2dd06d7fba4c709ca6f5fa4f2)

Recording frame at 994 seconds

That handler becomes an LLM gateway. It can run a policy check before calling the provider, while agents invoke it through Restate’s distributed communication primitives. Flow control can then limit one department to 300 simultaneous calls. This bounds concurrency rather than total spending, but it creates one enforceable location for policy and capacity controls instead of duplicating them across every agent.

Van Dongen also presents the foundation as able to recover from infrastructure failures such as network partitions and zombie failures. Those capabilities are stated here rather than demonstrated; the demo’s directly observable recovery case is the failed web-search retry.

### The distributed log underneath the programming model

[16:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1017s)

Under the SDK, Restate uses an event-driven distributed log between clients and services. The log persists journal events, while an event loop interprets them. Depending on the event, it may update an embedded state store, set a timer, or send a request to another agent. The same substrate therefore backs recorded function steps, session state, suspension points, and distributed calls.

![Recording frame at 1060 seconds](https://ai.engineer/api/talk-essay-media/0c980f1c6fd524d3e7adea14d5efcf88f9312eb857fdc2e9d8548a3e12459dd2)

Recording frame at 1060 seconds

What turns the journal into actual application behavior? The diagram shows the event loop dispatching persisted events into three different effects rather than treating the log as passive history.

A notable architectural choice is push-based invocation. Van Dongen contrasts it with workflow workers polling a server for new tasks: Restate sends the request directly to the target service, which also fits serverless functions that wake on incoming requests. She reports 45 milliseconds at p99 for a 10-step workflow, but supplies no benchmark setup, workload, or deployment conditions, so the figure should be read as a reported example rather than a general latency guarantee.

The server packages the state store and UI in one binary. A highly available deployment is described as multiple instances snapshotting to object storage, although the talk does not detail replication, coordination, or restoration. Adoption options include six SDKs, integrations with popular agent frameworks, direct use with arbitrary LLM SDKs, open-source self-hosting, deployment into a customer’s cloud account, and a managed cloud. Keeping Restate in the customer account can keep this layer’s data there; external tools and model providers remain separate data paths.

The closing lesson is narrower than “use a workflow engine for every agent.” Persistent agents need recoverable steps, addressable sessions, communication with live work, and a reliable way to stop it. Putting those mechanics beneath the agent loop lets the application evolve—from an inline LLM call to a governed gateway—without rebuilding retries, state management, and control each time.

From journal event to durable effect
The distributed log persists events; the event loop interprets them and applies the corresponding state, timing, or communication operation.
- Clients — Submit requests through the Restate layer.
- Distributed log — Persists the journal events produced by executions.
- Event loop — Reads each event and selects the corresponding effect.
- Embedded state store — Receives persistent state updates.
- Durable timer — Represents work that must wait until a later time or event.
- Agent or service request — Dispatches work to another application component.
- Clients → Distributed log (submit work and events)
- Distributed log → Event loop (feeds persisted events)
- Event loop → Embedded state store (persist state)
- Event loop → Durable timer (set timer)
- Event loop → Agent or service request (send request)

## Key takeaways

- Reliable agent infrastructure separates four concerns: recoverable execution, consistent concurrent sessions, distributed communication, and deliberate execution control.
- Journaled durable steps preserve completed progress through failures, and durable promises use the same mechanism to suspend human-approval waits across restarts and redeployments.
- Virtual objects isolate state by session key and serialize updates within a session; execution IDs separately make active runs retrievable, signalable, and cancellable.
- Signaling can amend work already in progress, while cancellation unwinds active subagents and controllers. Completed external side effects still require application-specific compensation.
- Moving inline model calls into a shared gateway creates one place for policy checks and concurrency limits, such as the example limit of 300 simultaneous calls per department.
- The distributed log and event loop connect journal events to state updates, timers, and service requests; push-based dispatch is intended to reduce latency and wake serverless functions directly.

## Related talks

- [Building Durable, Production-Ready Agents with OpenAI SDK and Temporal](https://ai.engineer/talks/k8cnVCMYmNc-building-durable-production-ready-agents-openai) — A complementary implementation-focused workshop on persisted LLM calls, event-sourced recovery, state management, and exposing durable workflow activities as agent tools.
- [Breaking the Chain: Agent Continuations for Resumable AI Workflows](https://ai.engineer/talks/ZB7l4uxW3Yo-breaking-chain-agent-continuations-resumable-ai) — Explores another mechanism for capturing nested agent and tool state so executions can suspend for approval, recover after failure, and resume without keeping loops active.
- [Two Roads to Durable Agents: Replay vs. Snapshot — Eric Allam, Co-founder, Trigger.dev](https://ai.engineer/talks/svCnShDvgQg-two-roads-durable-agents-replay-vs-snapshot) — Compares replay-based durability like the journal model discussed here with snapshot-based preservation of complete stateful execution environments.

## Slides

### The reliable agent stack

Giselle van Dongen introduces the infrastructure needed to run agents reliably in production.

[0:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=15s) [Title slide for Giselle van Dongen’s talk on the reliable agent stack.](https://assets.ai.engineer/sites/aiecode2025/sha256/b47b0af3cebf3cb851c89be9782821aff208484b8962982cf9326ca4c331728d.webp)

♫ Every step you take, every call you make
- the reliable agent stack

Giselle van Dongen

restate

### Persistent asynchronous agents

Giselle van Dongen frames agents as a third wave of LLM interaction: long-running processes with organization-wide tools and context, following websites and downloadable apps.

[0:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=27s) [A post by Andrej Karpathy highlights persistent, asynchronous agents with organization-wide tools and context.](https://assets.ai.engineer/sites/aiecode2025/sha256/7242357b3b7519fc50dbd652c4c2b7b4cb40132235c8e53d4db4f844f8e7e022.webp)

Andrej Karpathy
@karpathy

Imo this is the 3rd major redesign of LLM UIUX. The first paradigm was that the LLM is a website you go to, the second was that it is an app you download to your computer. This third one is that it is a self-contained, persistent, asynchronous entity with org-wide tools and context working alongside teams of humans. It really takes a while to wrap your head around it, but it works and it is awesome.

workloads.

Interface text:
12:26 AM · Jun 24, 2026 · 2M Views
587 · 1.2K · 12K · 6.4K

### The agent infrastructure stack

Agent SDKs and memory sit above a tangled infrastructure layer of queues, retries, checkpoints and locks. Van Dongen says these distributed pieces are necessary for long-running, stateful production agents.

[1:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=75s) [Agent tools and deployment platforms surround a complex infrastructure layer labeled with queues, retry logic, checkpoints, cron jobs, and session locks.](https://assets.ai.engineer/sites/aiecode2025/sha256/0854028bd39771355e1f7b44c47592c10715d3e6a6f6a9f2f708d485e2b112df.webp)

Agentic workflows
Harnesses
Background agents
Agent platforms

A2A, MCP,...

Agent SDKs
LLM SDKs
Memory
mem0
Letta
Observability & Evals

queue
KV store
retry logic
checkpoints
cron jobs
session lock

Infrastructure diagram: [illegible]

Serverless functions
Containers
Sandboxes
E2B

### Restate as a durable foundation

Restate is an open-source foundation for backends, including agents. Its ideas draw on Apache Flink and Meta’s core event infrastructure.

[2:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=136s) [Restate is presented as a flexible, durable foundation for building backends.](https://assets.ai.engineer/sites/aiecode2025/sha256/2cded082df9374beaaca906a9c6b86a1ea91acbc2dc678a99c52f4670f4528a1.webp)

Restate: a flexible, durable foundation to build any backend

By Apache Flink creators and lead ex-architects of Meta’s core event infra

### Restate’s durable runtime for agents

Restate sits between agent tooling and compute, coordinating execution, sessions, communication, and control. A failed agent run can resume where it stopped, while concurrent sessions keep their state consistent.

[2:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=167s) [The agent stack includes a Restate runtime layer for durable execution, state and session coordination, communication and flow control, and observability and control.](https://assets.ai.engineer/sites/aiecode2025/sha256/80aa50101e493aad0a39fb44210498a50c77d1dab4b78e792f011a8f24e8429a.webp)

Agentic workflows
Harnesses
Background agents
Agent platforms

A2A, MCP,...

Agent SDKs
LLM SDKs
Memory
mem0
Letta
Observability & Evals

Restate: durable runtime for agents
Durable Execution
State & session coordination
Communication & flow control
Observability & Control

Serverless functions
Containers
Sandboxes
E2B

### A durable, stateful function

Restate sits between clients and agent code, pushing invocations to the agent and receiving streamed events. It uses the event journal to recover the process after a failure.

[3:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=229s) [Clients invoke a Restate server, which pushes invocations to agent code and receives streamed events.](https://assets.ai.engineer/sites/aiecode2025/sha256/efc8d8175c94862dc855a7eb3e5f6f77e813a68b0771a16a0a21326e8a934534.webp)

Turn a normal function
into a long-running, durable, stateful entity

pushes invocations
streams events

HTTPS
f( )

Clients
Restate Server
Agent Code

### Asking the research agent

Giselle asks the Slack-connected research agent “What is new in AI?” beneath an earlier response about AI governance.

[5:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=311s) [A submitted question about new developments in AI appears beneath earlier research results.](https://assets.ai.engineer/sites/aiecode2025/sha256/d234ca256916cf5f26dc295ff8c6b7e68351c29176c8c777eb94ed44651555d0.webp)

The differences in cultural perspectives on AI ethics lead to fragmentation in global governance [illegible]. Disparities in technological resources hinder unified policy implementation, particularly between developed and developing nations, complicating international cooperation on AI reg[illegible].

Sources:
- https://darkreading.com/data-privacy/nist-updates-privacy-framework-ai-governance
- https://www.legal500.com/developments/thought-leadership/india-technology-law-legal-and-regulatory-developments-in-2023-authors-rahul-goel-and-anu-monga
- https://indiaai.gov.in/article/top-ai-developments-by-the-government-of-india-in-2023
- https://regulations.ai/regulations/pakistan-2023-5-draft-national-ai-policy
- https://hiddenlayer.com/insight/the-eu-ai-act-a-groundbreaking-framework-for-ai-regulation

IndiaAI
Top AI developments by the Government of India in 2023
2023 witnessed numerous developments from the Government of India. (1 MB)

hiddenlayer.com
The EU AI Act: A Groundbreaking Framework for AI Regulation
The EU AI Act seeks to regulate the development, commercialization, and use of AI technologies across the EU. (93 kB)

Giselle van Dongen  2:00 PM
What is new in AI?

Interface text:
Restate
ResearchAgent
Messages  About
Today
Home
DMs
Activity  12
Files
Later  21
More
Threads
Huddles
Drafts & sent  25
Directories
Starred
Drag and drop important stuff here
External connections
Channels
Direct messages
Apps
Slackbot
ResearchAgent
Slack
Message ResearchAgent
Google Chrome

### Restate services overview

After submitting a question to the Slack agent, Giselle opens Restate’s overview to inspect registered services and activity. It shows 2 running invocations and 18 successes out of 19 completed in the last hour.

[5:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=315s) [The Restate dashboard shows invocation status and registered agent services.](https://assets.ai.engineer/sites/aiecode2025/sha256/24a20b38daf312215b698cc53097f2c509395304f91b8eff095c76df6cdfc7de.webp)

Ready 0
Scheduled 0
Pending 0
Running 2
Backing off 0
Suspended 0
Paused 0

2
In-flight / scheduled

95%
Succeeded
of 19 completed

Succeeded 18
Failed, Cancelled or Killed 1

21 invocations
in last 1h

Services 5
Deployments 1
Handlers 6
+ Deployment

Sort by: Last deployed
Filter services, handlers, or deployments...

LLMGateway
1 handler
https://giselle--deep-research-agent-restate-services.modal.run/
REV. 4
7

ResearchAgent
1 handler
https://giselle--deep-research-agent-restate-services.modal.run/
REV. 4
3

DeepResearchAgent
1 handler

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Restate: the durable runtime
Overview - agent-demo - Re[illegible]
Restate - Invocation - inv_17[illegible]
Restate - Invocation - inv_17[illegible]
cloud.restate.dev/accounts/[illegible]/environments/[illegible]/overview
Work
Finish update
Thu Jul 2 14:00

### Approve the research plan

The planner posts three AI research subtopics to Slack for approval. Giselle explains that approving the plan unblocks the workflow and starts parallel research agents.

[5:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=343s) [Slack shows a research plan with three subtopics and an Approve button.](https://assets.ai.engineer/sites/aiecode2025/sha256/9de7198bfd298193927e250b736d822382cd734ec6c0aafa96a1601d2cb11a40.webp)

hiddenlayer.com
The EU AI Act: A Groundbreaking Framework for AI Regulation
The EU AI Act seeks to regulate the development, commercialization, and use of AI technologies across the EU. (93 kB)

Giselle van Dongen  2:00 PM
What is new in AI?

ResearchAgent  APP  2:00 PM
Research plan — needs approval
To provide an updated perspective on new advancements in AI, it's important to delve into breakthroughs, emerging applications, and the latest regulatory frameworks.

Subtopics:
• What are the latest breakthroughs in AI technologies and algorithms this year?
• What new applications and use cases of AI have emerged across different industries in 2023?
• What are the recent regulatory changes influencing the development and deployment of AI technologies?

Approve
Or chat about this:

Interface text:
Restate
ResearchAgent
Messages  About
Today
Home
Threads
Huddles
Drafts & sent  25
Directories
Starred
Drag and drop important stuff here
External connections
Channels
Direct messages
Apps
Slackbot
ResearchAgent
Slack
DMs
Activity  12
Files
Later  21
More
Message ResearchAgent
Search: in:@ResearchAgent conference

### Research completes and writing begins

Three overlapping research calls finish before another LLM call begins. The speaker says Restate uses the journaled events to recover execution after a crash.

[6:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=381s) [The journal shows completed research calls and a subsequent LLM call in progress.](https://assets.ai.engineer/sites/aiecode2025/sha256/c840760e5b09aad18f78d678871aff02187521aaeeeb30d1552eefdc8553e34b.webp)

DeepResearchAgent [illegible] · research
Running... for 1m 8s
Cancel...
SDK/python 1.0.1
State: No state
In-flight while no pending invocation(s) in queue.

Journal
Jul 2, 2026
2:00:16 PM PDT
Compact  Detailed  LIVE

research( Input )
Invoked by inv_1d76Til... · 1m 8s ago
Pending for 0.003s
Running since 1m 8s ago

1 call( LLMGateway ).call_llm( Input ) → Result
2 send( Controller [illegible] ).update_slack( Input ) →
  awakeable() → Result
3 call( ResearchAg... ).investiga...( Inpu... ) → Resul... · 25.352s
4 call( ResearchAg... ).investiga...( Inpu... ) → Resul... · 20.491s
5 call( ResearchAg... ).investiga...( Inpu... ) → Resul... · 17.55s
6 call( LLMGateway ).call_llm( Input ) → · 2.274s

Timeline: 36s, +40s, +44s, +48s, +52s, +56s, +1m, +1m 4s, Now
retained for 1d after completion

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Restate: the durable runtime
agent-demo - Restate Cloud
Restate - Invocation - [illegible]
cloud.restate.dev/[illegible]
Work  Finish update
Thu Jul 2 14:01

### Completed research journal

The completed research journal pairs web searches and LLM calls with their timing. The speaker explains that Restate uses these recorded events to recover execution if the agent crashes.

[6:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=386s) [A succeeded execution displays web search and LLM steps on a timeline.](https://assets.ai.engineer/sites/aiecode2025/sha256/d9836c1996ebcc91b41249565859f01a4d3bcda14a41d5e2e0769369a4fa5b10.webp)

DeepResearchAgent [illegible] ƒ research
Succeeded · 0.225s ago
Restart as new...

Journal
Jul 2, 2026
2:00:16 PM PDT
Compact · Detailed

ƒ research( Input )
Timeline: +8s, +16s, +24s, +32s, +40s, +48s, +56s, +1m 4s, +1m 12s
2 run("web_search:latest breakthroughs in AI 2023") → Result · 2.791s
3 run("web_search:AI technologies advancements 20...") → Resul... · 2.434s
4 run("web_search:AI algorithms news 2023") → Result · 0.554s
5 run("LLM call") → Result · 1.158s
6 run("web_search:AI breakthroughs 2023") → Result · 3.895s
7 run("web_search:latest AI technologies 2023") → Result · 0.302s
8 run("web_search:advancements in AI algorithms 2...") → Resul... · 2.839s
9 run("web_search:AI breakthroughs 2023") → Result · 0.363s
10 run("web_search:latest AI technologies 2023") → Result · 0.261s
11 run("web_search:advancements in AI algorithms 2...") → Resul... · 0.5s
12 run("LLM call") → Result · 4.061s
13 return Result

Interface text:
Chrome · File · Edit · View · History · Bookmarks · Profiles · Tab · Window · Help
Restate: the durable runtime · agent-demo - Restate Cloud · Restate - Invocation - [illegible]
cloud.restate.dev/accounts/[illegible]/environments/[illegible]/invocations/[illegible]
Work · Finish update
Thu Jul 2 14:01

### A search error in the journal

The journal shows an injected Websearch API down error during a subagent’s searches. The timeline traces its retry and successful completion.

[6:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=394s) [The detailed journal highlights a web search error within a succeeded execution.](https://assets.ai.engineer/sites/aiecode2025/sha256/e45f85fa45a190913780514bf6e057833e904718d3540ff404da2c1d95986dfb.webp)

DeepResearchAgent [illegible] research
Succeeded  0.225s ago
Restart as new...

Journal
Jul 2, 2026
2:00:16 PM PDT
Compact  Detailed 1

research( Input )
Timeline: +8s  +16s  +24s  +32s  +40s  +48s  +56s  +1m 4s  +1m 12s

3 call( ResearchAg... ).investiga... ( Inpu... ) → Resul...  25.352s
Invoked by inv_1d76Til...  31.281s ago
Pending for 0.001s
Running for 25.344s

1 run("LLM call") → Result  1.958s
2 run("web_search:latest breakthroughs in AI 2023") → Result  2.791s
3 run("web_search:AI technologies advancements 20...") → Resul...  2.434s
4 run("web_search:AI algorithms news 2023") → Result  0.554s
#2 Exception('Websearch API down')
5 run("LLM call") → Result  1.158s
6 run("web_search:AI breakthroughs 2023") → Result  3.895s
7 run("web_search:latest AI technologies 2023") → Result  0.302s

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Restate: the durable runtime
agent-demo - Restate Cloud
Restate - Invocation - inv_17...
cloud.restate.dev/[illegible]
Work  Finish update
Thu Jul 2 14:01

### Transient failure details

A web search fails because its API is down. The journal shows the transient error; the speaker explains that the search was retried and completed, recovering progress from the journal.

[6:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=403s) [A transient failure panel reports a retryable web search API error and shows a traceback.](https://assets.ai.engineer/sites/aiecode2025/sha256/880f2699ea092dc5f232d8fef7b30a626f263a78363f09e9e1b190655e1ca152.webp)

DeepResearchAgent  [illegible]  research
Succeeded  0.225s ago
Restart as new...

Journal
Jul 2, 2026
2:00:16 PM PDT
Compact  Detailed 1

research( Input )
Timeline: +8s  +16s  +24s  +32s  +40s  +48s  +56s  +1m 4s  +1m 12s

3  call( ResearchAg... ).investiga... ( Inpu... ) → Resul...  25.352s
Invoked by  inv_1d76Til...  31.281s ago

Transient failure
A retry-able error was received from the service while processing the invocation.

Exception('Websearch API down')
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/site-packages/restate/server_context.py", line 810, in create_run_coroutine
    action_result = typing.cast(T, await action_result_future)
Exception: Websearch API down

#2  Exception('Websearch API down')
5  run("LLM call") → Result  1.158s
6  run("web_search:AI breakthroughs 2023") → Result  3.895s
7  run("web_search:latest AI technologies 2023") → Result  0.302s

Interface text:
Chrome  File  Edit  View  History  Bookmarks  Profiles  Tab  Window  Help
Restate: the durable runtime
agent-demo - Restate Cloud
Restate - Invocation - inv_17...
cloud.restate.dev/accounts/[illegible]
Work  Finish update
Thu Jul 2 14:01

### Durable research workflow

The research handler wraps LLM calls in durable steps, waits for human approval, then runs subagent research in parallel and sends a report to Slack. The approval wait can suspend and resume when a response arrives.

[7:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=423s) [Python code for planning, human approval, parallel research, writing, and sending a report.](https://assets.ai.engineer/sites/aiecode2025/sha256/7c5fbd6c6d9c87e16fe6c90a5db6fff9f885b35d8f1358dae5dbd4a6493a9d22.webp)

# DURABLE EXECUTION FOR LONG-RUNNING AGENTS

@deep_research_agent.handler()
async def deep_research(restate: ObjectContext, history: ChatHistory):
    session = restate.key()

    # 1. plan
    plan = await restate.run_typed("plan", llm_call, req=plan_request(history))

    # 2. human approval
    awk_id, decision = restate.awakeable(type_hint=Decision)
    restate.object_send(update_slack, key=session, arg=format_plan(plan, awk_id))
    if not (await decision).approved:
        return

    # 3. parallel subagent research
    handles = [restate.service_call(investigate, arg=topic) for topic in plan["subtopics"]]
    await rst.gather(*handles)
    sub_reports = [await h for h in handles]

    # 4. write
    draft = await restate.run_typed("write", llm_call, req=write_request(history, plan, sub_reports))

    # 5. Slack message
    restate.object_send(update_slack, key=session, arg=format_report(draft, restate.request().id))

Interface text:
PyCharm; File; Edit; View; Navigate; Code; Refactor; Run; Tools; Git; Window; Help
resilient-research-agent; main; Current File; Unlock Pro
deep_research.py; session_coordinator.py; llm_gateway.py
Giselle van Dongen
Parameter hints: name: "plan"; name: "write"
resilient-research-agent > app > deep_research.py
82:15; LF; UTF-8; 4 spaces; uv (resilient-research-agent) [Python 3.14.2]
Thu Jul 2 14:04

### Session coordination and control

The session controller saves each message and checks for running research. An LLM classifies the request so the controller can cancel the current invocation or send it a steering signal; it can then start new research.

[9:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=577s) [Python code stores message history and cancels or steers ongoing research; the lower portion continues off screen.](https://assets.ai.engineer/sites/aiecode2025/sha256/1d9bc29a548e8c2612116d5f70a4a2a0df60f9a105eba66374dfd7ee9725cfcf.webp)

# BEYOND WORKFLOWS: STATEFUL ACTORS & SESSION COORDINATION & CONTROL

@controller.handler()
async def message(restate: ObjectContext, text: str) -> None:
    # 1. Add to context
    history = await restate.get("messages", type_hint=ChatHistory) or ChatHistory()
    history.messages.append(user(text))
    restate.set("messages", history)

    # 2. Check for ongoing research
    current = await restate.get("current", type_hint=str)

    # 3a. Cancel or steer ongoing research
    if current is not None:
        decision = await call_llm_gateway(restate, classify_request(history, text))
        match decision["strategy"]:
            case "cancel":
                restate.cancel_invocation(current)
            case _:
                restate.resolve_signal(current, "steer", text)
                return

    # 3b. Start new research
    from deep_research import research
    handle = restate.object_send(research, key=restate.key(), arg=history)

Interface text:
PyCharm; resilient-research-agent; main
Tabs: deep_research.py, session_coordinator.py, llm_gateway.py
Giselle van Dongen
Parameter hints: name: for "messages", "current", and "steer"
21:26 (7 chars); LF; UTF-8; 4 spaces; uv (resilient-research-agent) [Python 3.14.2]

## Full transcript

[0:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=12s) Hi everyone. This talk will be about how

[0:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=14s) to run agents reliably in production. It

[0:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=17s) will not be about the eile part, but it

[0:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=19s) will be about all the other things you

[0:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=22s) need to get going in order to run agents

[0:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=24s) resiliently. So the infrastructure layer

[0:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=27s) basically. I want to set the scene with

[0:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=30s) this uh quote of Andre Apathy of last

[0:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=32s) week. It describes that the way we

[0:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=34s) interact with agents and LLMs has been

[0:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=37s) evolving in three waves. The first wave

[0:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=40s) was an LLM being something like a

[0:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=42s) website where we go to we ask it a

[0:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=44s) question, it thinks for a few seconds

[0:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=47s) and then gives us a response. The second

[0:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=49s) wave was going towards agents. It was an

[0:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=52s) app that we download to our computer. It

[0:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=54s) has some tools at its disposal and it

[0:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=57s) can do some work with our interaction.

[1:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=60s) Now the third wave will be going more

[1:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=62s) and more towards persistent and

[1:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=64s) asynchronous entities. So agents being

[1:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=67s) longunning processes in our

[1:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=69s) infrastructure with access to tools and

[1:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=72s) other agents around the organization and

[1:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=74s) context.

[1:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=77s) And so as our use cases are evolving

[1:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=79s) more and more from single agents to

[1:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=81s) agentic platforms that connect parts

[1:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=84s) around uh the organization our

[1:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=86s) infrastructure layer should also evolve

[1:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=88s) with that. So when we look at the types

[1:31](https://www.youtube.com/watch?v=cI7zfqusmFU&t=91s) of tools that are currently out there to

[1:33](https://www.youtube.com/watch?v=cI7zfqusmFU&t=93s) implement agents, a lot of innovation

[1:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=95s) has been done on sites such as agent

[1:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=97s) SDKs and memory. And agent SDKs are

[1:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=101s) really cool to implement PC's and get

[1:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=104s) started quickly, but they don't

[1:46](https://www.youtube.com/watch?v=cI7zfqusmFU&t=106s) necessarily help with like connecting

[1:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=107s) the distributed bits around an

[1:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=109s) organization.

[1:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=111s) And if you want to implement more

[1:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=113s) complex agentic systems, you actually

[1:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=116s) need all of those things. So that is the

[1:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=118s) layer that you see below here where um

[2:01](https://www.youtube.com/watch?v=cI7zfqusmFU&t=121s) you have to deploy extra infrastructure.

[2:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=123s) Uh you need to write things like retry

[2:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=125s) logic, recovery logic and all of that is

[2:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=128s) actually pretty complex to get right but

[2:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=131s) completely necessary to run longunning

[2:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=133s) stateful and distributed processes in

[2:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=135s) production.

[2:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=137s) So today I want to talk about an

[2:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=139s) open-source framework called restate.

[2:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=141s) And you can see it a bit as a flexible

[2:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=143s) durable foundation that lets you build

[2:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=146s) any backend. So it's not specific for

[2:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=148s) agents but a as agents are also just a

[2:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=152s) type of a backend uh it also works well

[2:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=154s) for them. The ideas behind restate come

[2:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=157s) from Apache Flink which is a popular

[2:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=160s) distributed stream processing engine and

[2:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=162s) also from some of the exarchitects

[2:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=165s) behind Meta Score event infra.

[2:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=169s) So what are the ingredients in restate?

[2:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=171s) Basically four parts. First of all, it

[2:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=174s) makes sure that a single run of an agent

[2:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=176s) is resilient. This is called durable

[2:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=179s) execution in the industry. Think about

[3:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=182s) things like when an agent runs for a

[3:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=184s) week and then crashes. We want to be

[3:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=186s) able to bring it back and let it

[3:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=188s) continue exactly at the point where it

[3:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=190s) failed. We don't want it to start over

[3:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=192s) from the beginning.

[3:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=194s) Another um area here is running many

[3:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=198s) concurrent sessions in parallel. Imagine

[3:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=200s) running thousands of concurrent agent

[3:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=202s) sessions at the same time and needing

[3:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=204s) needing to make sure that state is

[3:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=206s) always consistent and that different

[3:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=208s) agents don't interfere with each other.

[3:31](https://www.youtube.com/watch?v=cI7zfqusmFU&t=211s) And then going more towards things like

[3:33](https://www.youtube.com/watch?v=cI7zfqusmFU&t=213s) communication between agents, between

[3:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=215s) agents and MCP servers and other tools.

[3:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=218s) And finally also control, making sure

[3:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=220s) that when an agent for example uh is

[3:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=223s) doing um something you don't want it to

[3:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=225s) continue or when it's stuck being able

[3:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=227s) to actually cancel or kill the

[3:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=228s) execution.

[3:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=231s) So the way that you can think of it is

[3:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=233s) as follows. Restate is basically a

[3:55](https://www.youtube.com/watch?v=cI7zfqusmFU&t=235s) server which runs in front of your agent

[3:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=238s) service. So as a separate component it

[4:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=240s) sits there a bit like a like a message

[4:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=242s) broker or a proxy and when there's a

[4:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=245s) request for your agent restate proxies

[4:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=247s) the request to the service and pushes it

[4:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=250s) to the service basically and from that

[4:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=252s) moment there's a connection open

[4:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=255s) connection between restate and the agent

[4:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=257s) and that connection will basically be a

[4:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=259s) bit like a lifeline for the agent. So as

[4:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=261s) the agent is doing stuff, it sends

[4:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=264s) events over to restate and restate will

[4:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=267s) use that journal of events to recover

[4:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=269s) the process after a failure.

[4:33](https://www.youtube.com/watch?v=cI7zfqusmFU&t=273s) So from a slightly higher level um

[4:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=275s) explanation, you could say that it's

[4:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=277s) turning a normal function in your

[4:39](https://www.youtube.com/watch?v=cI7zfqusmFU&t=279s) application into something that is long

[4:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=281s) running, durable, and stateful without

[4:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=284s) having to do um a lot of the complex

[4:46](https://www.youtube.com/watch?v=cI7zfqusmFU&t=286s) things you otherwise need to do for

[4:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=288s) this. So my talk today will be mainly a

[4:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=291s) demo. So I'll be showing you um a

[4:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=294s) research agent that is connected to

[4:55](https://www.youtube.com/watch?v=cI7zfqusmFU&t=295s) Slack. Imagine we are like working at

[4:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=298s) some company and we want to make an

[5:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=300s) Slack agent available to all of our

[5:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=302s) employees.

[5:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=303s) So if I go here into Slack then can I

[5:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=306s) can here in this channel for example ask

[5:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=309s) what is new in AI.

[5:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=312s) Now let's have a look at what it's doing

[5:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=314s) under the hood. So if I go back here, I

[5:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=317s) have here the restate uh UI. This is a

[5:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=320s) bit like a cockpit for your agents. So

[5:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=322s) you can see a registry of all the agents

[5:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=325s) that are currently registered and you

[5:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=327s) can also see for example which execution

[5:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=329s) is currently happening. So here is the

[5:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=332s) deep research agent that I spinned up a

[5:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=334s) few seconds ago. We can see what it's

[5:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=337s) currently doing. Now it called first an

[5:39](https://www.youtube.com/watch?v=cI7zfqusmFU&t=339s) LLM and then it sent me an answer via

[5:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=342s) Slack. This first LLM call was a planner

[5:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=345s) agent. So what it did is it planned the

[5:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=348s) research and sent me um a list of

[5:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=350s) subtopics that it wants to research.

[5:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=353s) Now if I press here approve then this

[5:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=356s) will unblock the workflow and will spin

[5:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=358s) up a set of parallel research agents. So

[6:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=362s) this is basically like the classical

[6:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=363s) deep research workflow, right? You have

[6:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=365s) a planner then a set of subress research

[6:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=368s) agents and then finally someone uh who

[6:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=371s) writes a report on this like a writer

[6:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=373s) agent

[6:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=375s) and so this journal you see here on the

[6:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=377s) left that is basically the events that

[6:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=379s) get sent from the agent to the restate

[6:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=381s) server and if this now crashes at some

[6:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=384s) point this journal is what will be used

[6:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=387s) to uh recover the execution to the point

[6:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=389s) where it failed. I don't know if uh

[6:31](https://www.youtube.com/watch?v=cI7zfqusmFU&t=391s) there were some errors. I injected a bit

[6:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=394s) of like tool errors in here. Yeah, here

[6:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=396s) you can for example see that um the sub

[6:39](https://www.youtube.com/watch?v=cI7zfqusmFU&t=399s) agent first did an LLM call then started

[6:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=402s) doing some web searches and eventually

[6:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=404s) uh one of the web searches didn't go

[6:46](https://www.youtube.com/watch?v=cI7zfqusmFU&t=406s) through because the API was down and

[6:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=408s) then you see here on the right how it

[6:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=410s) got retrieded and eventually completed

[6:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=412s) successfully. So instead of starting

[6:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=414s) over, it uses the journal to recover the

[6:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=417s) progress.

[6:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=418s) Let's now have a look at what this looks

[7:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=420s) like in code.

[7:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=422s) So the basic unit of how you implement

[7:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=426s) applications in restate is by writing

[7:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=427s) HTTP handlers and those handlers become

[7:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=430s) durable by using the restate SDK. So

[7:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=433s) here in this case we have here our deep

[7:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=436s) research handler and here as a first

[7:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=438s) argument we have a restate object

[7:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=440s) context and the way you can imagine that

[7:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=442s) is basically as that uh connection to

[7:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=445s) that restate server. whenever I do an

[7:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=448s) action on this uh restate object, it

[7:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=450s) will lead to an event being sent to

[7:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=452s) restate. So for example, when I did that

[7:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=456s) planner LLM call, what actually happened

[7:39](https://www.youtube.com/watch?v=cI7zfqusmFU&t=459s) under the hood was it executed here this

[7:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=461s) Python function. This is just a simple

[7:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=464s) light um light lm like

[7:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=468s) LLM call and the way I made it durable

[7:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=471s) is by wrapping it in restate.run.

[7:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=474s) So what happens is by doing these

[7:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=476s) durable steps if this fails somewhere

[7:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=479s) here two hours or two months later it

[8:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=482s) will recover to exactly that point.

[8:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=485s) So that's the idea of durable execution.

[8:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=487s) You're always able to recover a process

[8:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=489s) to where it was. You can also use that

[8:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=491s) for other things not necessarily for

[8:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=493s) failure recovery. For example, imagine

[8:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=496s) we want to ask a human to approve

[8:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=497s) something and this approval might take

[8:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=500s) weeks or a month. this process needs to

[8:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=503s) be able to um to survive restarts and

[8:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=507s) redeploys uh over those kind of long

[8:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=509s) periods of time and so with durable

[8:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=512s) execution you can actually also uh

[8:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=514s) suspend a function and let bring it back

[8:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=518s) when it's able to make progress. So in

[8:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=520s) the case of a human approval what we do

[8:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=522s) here is basically we we create a durable

[8:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=524s) promise which lives in that journal a

[8:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=527s) bit like a suspension point. Then we ask

[8:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=531s) uh a human to click that button in

[8:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=533s) select as I showed in the beginning and

[8:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=536s) while we are waiting this process

[8:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=537s) actually suspends. So if it's running on

[8:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=539s) serverless this is not using uh

[9:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=542s) execution uh time on our functions.

[9:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=546s) Once the response comes in this then

[9:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=548s) gets unblocked and can continue where it

[9:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=550s) left off. So what we see here is a bit

[9:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=553s) like a workflow. It's a set of steps

[9:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=555s) that get executed durably. But when we

[9:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=558s) think about agents and also the way that

[9:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=560s) Karpathy described it in the tweet, it's

[9:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=562s) more like a persistent stateful entity

[9:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=564s) that lives for a longer period of time

[9:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=567s) that has some memory. Um, so a workflow

[9:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=570s) is not the nicest way to model this kind

[9:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=572s) of thing. So the way that we can model

[9:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=576s) this in restate is by using something

[9:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=578s) called a virtual object. So imagine in

[9:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=581s) the use case that I'm showing this slack

[9:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=583s) research agent. Imagine that I don't

[9:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=585s) want to wait for 10 minutes to give it

[9:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=588s) some follow-up context or maybe I think

[9:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=591s) about something else that I should have

[9:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=592s) told it. Um I want to actually be able

[9:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=594s) to interact with it, not wait till that

[9:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=596s) research is finished before I can send a

[9:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=599s) follow-up.

[10:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=600s) And so this is basically what a virtual

[10:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=603s) object in restate is. It's a bit like a

[10:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=605s) stateful actor. It has a unique ID, for

[10:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=607s) example, a session ID. It has uh some

[10:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=610s) key value states that is isolated for

[10:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=614s) that specific session that you can write

[10:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=616s) to. Uh imagine for example your history

[10:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=618s) of messages and it also has like a set

[10:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=621s) of handlers that can execute durable

[10:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=625s) functions uh for this session. So here

[10:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=628s) the way I implemented this use case that

[10:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=630s) I mentioned of interacting with a

[10:33](https://www.youtube.com/watch?v=cI7zfqusmFU&t=633s) running process is as follows. This is a

[10:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=637s) um a bit a session controller. Again, it

[10:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=640s) has like this restate object context at

[10:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=642s) its disposal to do things in a

[10:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=645s) recoverable way. Uh it can write to this

[10:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=648s) session store. Here it I'm retrieving

[10:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=650s) the chat history.

[10:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=652s) And one thing that's interesting there

[10:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=654s) is that in order to run these kind of

[10:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=656s) sessions in very high uh paralyzed ways,

[10:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=659s) so thousands of sessions at the same

[11:01](https://www.youtube.com/watch?v=cI7zfqusmFU&t=661s) time, we need to make sure that agents

[11:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=664s) do not interfere with each other.

[11:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=666s) Imagine I'm sending two messages on

[11:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=668s) Slack and now two agents are actually

[11:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=670s) overwriting each other each other's

[11:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=672s) session state. To prevent that, this

[11:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=675s) will guarantee that only one execution

[11:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=678s) is running at a time. So a second

[11:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=679s) execution will be cued behind the

[11:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=682s) current one.

[11:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=686s) Then let's have a look at how we

[11:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=688s) implement this like interacting with

[11:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=690s) another execution. So an execution in

[11:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=692s) reset has a unique identifier and you

[11:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=695s) can use that identifier to connect to it

[11:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=698s) from other processes. for example, to

[11:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=700s) retrieve uh the output, but also to

[11:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=703s) cancel it or maybe to signal it being

[11:46](https://www.youtube.com/watch?v=cI7zfqusmFU&t=706s) injecting a bit of state into an already

[11:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=709s) running agent loop. And so this is like

[11:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=712s) a very flexible type of um uh

[11:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=716s) capabilities that you can do to

[11:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=717s) implement things like for example

[12:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=720s) signaling an already ongoing agent loop.

[12:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=722s) So what we do here is if there is a

[12:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=724s) current execution ongoing then we will

[12:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=727s) ask an LLM is this like something that

[12:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=730s) is relevant for the current agent loop.

[12:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=733s) If that is the case inject this via a

[12:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=736s) signal if it's not really relevant for

[12:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=739s) what we're currently doing then cancel

[12:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=741s) what you're currently doing and start

[12:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=742s) over again with this new information.

[12:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=746s) And so this goes a little bit further

[12:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=748s) than workflows. it goes a bit more

[12:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=749s) towards like writing persistent stateful

[12:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=752s) entities that can interact with each

[12:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=754s) other and have memory at uh their

[12:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=756s) disposal. So let me show you uh how this

[12:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=760s) works. So here if I now ask again what

[12:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=762s) is new in AI and I wait a few seconds

[12:46](https://www.youtube.com/watch?v=cI7zfqusmFU&t=766s) then it should respond again with a

[12:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=768s) plan. Um and then I can say for example

[12:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=771s) some extra info focus on frontier models

[12:55](https://www.youtube.com/watch?v=cI7zfqusmFU&t=775s) let's say.

[12:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=779s) So once I have the plan I will inject

[13:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=782s) that bit of extra state.

[13:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=786s) Now let's look at the UI of what this is

[13:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=789s) now doing. So here I have that

[13:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=790s) controller which I just showed. It

[13:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=793s) started calling an LLM to classify uh

[13:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=796s) this new input.

[13:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=799s) Once this comes back, it will probably

[13:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=801s) decide that it should signal it because

[13:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=803s) it's it's still relevant to the research

[13:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=805s) it's currently doing. So this inject

[13:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=808s) that new message into the ongoing agent

[13:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=810s) loop. So let me show you in the deep

[13:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=812s) research agent again. Um so first it

[13:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=815s) called an LLM then asked us then we

[13:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=818s) injected this uh new message of focus on

[13:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=821s) frontier models and then it uh took that

[13:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=824s) into account and started over again.

[13:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=827s) Here

[13:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=829s) I can now for example also say something

[13:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=831s) like uh forget about that

[13:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=837s) research AI policy.

[14:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=840s) And if I send this then the coord

[14:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=842s) coordinator will um decide to cancel the

[14:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=845s) ongoing run and start a new one that

[14:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=848s) will

[14:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=850s) research this new topic. And so this

[14:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=852s) cancellation is basically like a signal

[14:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=854s) that gets um sent down the stack of or

[14:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=859s) the call chain. So if my agent was

[14:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=861s) already spinning up sub agents first

[14:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=863s) those sub aents would be cancelled then

[14:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=865s) uh the controller itself and like that

[14:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=868s) it would basically rewind the stack and

[14:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=870s) give agents also the ability to roll

[14:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=872s) back.

[14:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=874s) Okay. Okay, so this went a bit more into

[14:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=876s) the direction of like stateful

[14:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=877s) persistent entities that we can interact

[14:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=880s) with over longer periods of time. Now

[14:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=882s) the last part of the demo that I want to

[14:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=884s) show is um going more towards like being

[14:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=887s) able to write highly customized

[14:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=889s) applications. Imagine that we deploy

[14:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=891s) this in production but then a few months

[14:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=894s) later a new model provider brings out a

[14:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=896s) new model for example fabulous and even

[14:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=899s) though the model is very good it's also

[15:01](https://www.youtube.com/watch?v=cI7zfqusmFU&t=901s) very expensive and we notice that this

[15:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=903s) research agent is actually starting to

[15:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=905s) cost a lot. These kind of uh things that

[15:08](https://www.youtube.com/watch?v=cI7zfqusmFU&t=908s) pop up halfway through a project require

[15:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=912s) you to then deploy a a lot of new extra

[15:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=915s) infra or like find a good way to solve

[15:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=917s) this. This is the kind of things that

[15:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=918s) Restate really excels at. It doesn't

[15:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=921s) really peg you into a specific way of

[15:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=923s) how you should write your application.

[15:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=925s) It basically gives you like a durable

[15:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=927s) programming model that lets you

[15:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=929s) implement an application in the way that

[15:31](https://www.youtube.com/watch?v=cI7zfqusmFU&t=931s) fits for you and also extend it if

[15:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=934s) necessary. So first I showed this um LLM

[15:39](https://www.youtube.com/watch?v=cI7zfqusmFU&t=939s) call in the first example as an inline

[15:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=941s) step. It was just a Python function that

[15:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=944s) got persisted. But imagine this use case

[15:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=947s) that we want to actually have a bit more

[15:49](https://www.youtube.com/watch?v=cI7zfqusmFU&t=949s) control over those LLM calls. For

[15:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=951s) example, what you can do is then pull

[15:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=953s) this out into its own handler.

[15:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=956s) And this handler can now do things like

[15:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=958s) for example a policy check and then uh

[16:01](https://www.youtube.com/watch?v=cI7zfqusmFU&t=961s) do the LLM call. And the other agents

[16:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=964s) instead of doing this LLM call inline

[16:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=966s) can now use restates like distributed

[16:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=969s) communication primitives to actually

[16:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=971s) just call this LLM gateway instead of

[16:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=974s) doing it as an inline step. And this

[16:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=978s) service fabric that lets you communicate

[16:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=980s) between agents also gives you some um

[16:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=983s) things like flow control. So we can for

[16:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=985s) example say one department is only

[16:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=988s) allowed to run 300 calls to this LLM

[16:31](https://www.youtube.com/watch?v=cI7zfqusmFU&t=991s) gateway at the same time. So the reason

[16:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=994s) why I showed this was just to show you a

[16:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=996s) bit like that. Uh it's basically just a

[16:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=998s) a resilient foundation. It makes sure

[16:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1000s) that your process can uh recover from

[16:43](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1003s) even a more advanced types of

[16:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1005s) infrastructure failures, things like

[16:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1007s) network partitions and zombie failures.

[16:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1010s) And um it gives you like tooling to

[16:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1013s) extend and customize as your use case

[16:55](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1015s) grows.

[16:57](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1017s) Let's go back to the slides to have a

[17:00](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1020s) little more of an idea of how this thing

[17:03](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1023s) is actually implemented on the inside

[17:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1025s) because it's actually a pretty

[17:06](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1026s) interesting um design or architecture.

[17:11](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1031s) So the way it's implemented is basically

[17:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1033s) by having a a event-driven distributed

[17:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1036s) log implementation.

[17:19](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1039s) So inside the box you basically on one

[17:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1040s) side have the clients on the other side

[17:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1042s) the services and inside the box is a log

[17:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1046s) which persists all those journal events

[17:28](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1048s) and an event loop and that event loop

[17:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1050s) basically gets the events from the

[17:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1052s) service based on what the event is. It

[17:35](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1055s) either persists some state in the

[17:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1057s) embedded state store or it sets a timer

[17:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1060s) or it sends a request to another agent.

[17:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1062s) And by doing that you basically have a

[17:45](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1065s) durable um foundation for whatever an

[17:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1068s) application is doing.

[17:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1070s) The design of this distributed log is

[17:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1073s) heavily inspired by the way that the

[17:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1076s) core event infra layer at meta works. Uh

[17:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1079s) it's basically like an iteration on top

[18:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1082s) of that. Um and some of those architects

[18:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1084s) are now have designed that for restate

[18:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1087s) as an more generic solution that is

[18:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1090s) available in open source. There are two

[18:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1092s) important things related to this

[18:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1094s) architecture that make it interesting.

[18:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1096s) The first one is that it works as a push

[18:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1098s) model. So whereas most workflow

[18:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1101s) orchestrators actually pull for new

[18:23](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1103s) tasks um for pull from the workflow

[18:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1106s) server, restate actually pushes the

[18:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1109s) invocations and the benefit you get from

[18:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1112s) that is that it has a much lower

[18:33](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1113s) latency. So you can use these kind of

[18:36](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1116s) workflow guarantees in functions around

[18:38](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1118s) your application and uh have like a

[18:41](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1121s) latencies of for example 45 milliseconds

[18:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1124s) p99 for like a 10-step workflow.

[18:48](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1128s) Pushing invocations also works very well

[18:50](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1130s) for serverless because they require you

[18:53](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1133s) to basically uh send the request and

[18:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1136s) wake up the function. So this design

[18:59](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1139s) that I show here includes everything you

[19:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1142s) need. It includes uh as well that state

[19:05](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1145s) store where we were embedding the state

[19:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1147s) as the UI. It's a single binary so it's

[19:10](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1150s) pretty easy to operate as well to run it

[19:13](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1153s) in like a highly available way. You just

[19:15](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1155s) spin it up multiple times and let it

[19:17](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1157s) snapshot to object storage.

[19:21](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1161s) So restate has six different SDKs. We

[19:24](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1164s) also have integrations for most of the

[19:26](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1166s) popular agent frameworks out there. And

[19:30](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1170s) of course, because it's just like a

[19:32](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1172s) flexible layer, you can also just use

[19:34](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1174s) any LLM SDK and implement custom agents

[19:37](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1177s) by just wrapping some steps into uh

[19:40](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1180s) these SDK constructs. So, it's open

[19:42](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1182s) source. You can self-host it. We also

[19:44](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1184s) have a BYOC offering where we deploy

[19:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1187s) restate in your cloud account and uh

[19:51](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1191s) that gives you the benefit that data

[19:52](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1192s) doesn't leave your cloud account.

[19:54](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1194s) Otherwise, there's also a managed cloud

[19:56](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1196s) offering.

[19:58](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1198s) This was mainly what I wanted to show.

[20:01](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1201s) If you want to explore the code a bit

[20:02](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1202s) further, there is here this the GitHub

[20:04](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1204s) repo. It's publicly available. If you

[20:07](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1207s) like the project, then have a look at

[20:09](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1209s) the restate repo itself. We are hiring

[20:12](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1212s) across the board for all sorts of roles

[20:14](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1214s) going from engineering to marketing,

[20:16](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1216s) especially also here in the Bay Area.

[20:18](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1218s) So, if you're interested in that, uh,

[20:20](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1220s) then definitely check out our careers

[20:22](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1222s) page. and I will be outside in front of

[20:25](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1225s) the conference hall here if you want to

[20:27](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1227s) ask any questions or learn more about

[20:29](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1229s) restate. Thank you very much.

[20:47](https://www.youtube.com/watch?v=cI7zfqusmFU&t=1247s) >> [music]
