[
  {
    "startMs": 12480,
    "text": "Hi everyone. This talk will be about how"
  },
  {
    "startMs": 14880,
    "text": "to run agents reliably in production. It"
  },
  {
    "startMs": 17840,
    "text": "will not be about the eile part, but it"
  },
  {
    "startMs": 19920,
    "text": "will be about all the other things you"
  },
  {
    "startMs": 22080,
    "text": "need to get going in order to run agents"
  },
  {
    "startMs": 24480,
    "text": "resiliently. So the infrastructure layer"
  },
  {
    "startMs": 27279,
    "text": "basically. I want to set the scene with"
  },
  {
    "startMs": 30000,
    "text": "this uh quote of Andre Apathy of last"
  },
  {
    "startMs": 32480,
    "text": "week. It describes that the way we"
  },
  {
    "startMs": 34719,
    "text": "interact with agents and LLMs has been"
  },
  {
    "startMs": 37120,
    "text": "evolving in three waves. The first wave"
  },
  {
    "startMs": 40160,
    "text": "was an LLM being something like a"
  },
  {
    "startMs": 42800,
    "text": "website where we go to we ask it a"
  },
  {
    "startMs": 44800,
    "text": "question, it thinks for a few seconds"
  },
  {
    "startMs": 47120,
    "text": "and then gives us a response. The second"
  },
  {
    "startMs": 49680,
    "text": "wave was going towards agents. It was an"
  },
  {
    "startMs": 52800,
    "text": "app that we download to our computer. It"
  },
  {
    "startMs": 54960,
    "text": "has some tools at its disposal and it"
  },
  {
    "startMs": 57360,
    "text": "can do some work with our interaction."
  },
  {
    "startMs": 60559,
    "text": "Now the third wave will be going more"
  },
  {
    "startMs": 62559,
    "text": "and more towards persistent and"
  },
  {
    "startMs": 64479,
    "text": "asynchronous entities. So agents being"
  },
  {
    "startMs": 67439,
    "text": "longunning processes in our"
  },
  {
    "startMs": 69119,
    "text": "infrastructure with access to tools and"
  },
  {
    "startMs": 72080,
    "text": "other agents around the organization and"
  },
  {
    "startMs": 74720,
    "text": "context."
  },
  {
    "startMs": 77040,
    "text": "And so as our use cases are evolving"
  },
  {
    "startMs": 79439,
    "text": "more and more from single agents to"
  },
  {
    "startMs": 81360,
    "text": "agentic platforms that connect parts"
  },
  {
    "startMs": 84479,
    "text": "around uh the organization our"
  },
  {
    "startMs": 86720,
    "text": "infrastructure layer should also evolve"
  },
  {
    "startMs": 88720,
    "text": "with that. So when we look at the types"
  },
  {
    "startMs": 91040,
    "text": "of tools that are currently out there to"
  },
  {
    "startMs": 93600,
    "text": "implement agents, a lot of innovation"
  },
  {
    "startMs": 95680,
    "text": "has been done on sites such as agent"
  },
  {
    "startMs": 97920,
    "text": "SDKs and memory. And agent SDKs are"
  },
  {
    "startMs": 101040,
    "text": "really cool to implement PC's and get"
  },
  {
    "startMs": 104240,
    "text": "started quickly, but they don't"
  },
  {
    "startMs": 106079,
    "text": "necessarily help with like connecting"
  },
  {
    "startMs": 107920,
    "text": "the distributed bits around an"
  },
  {
    "startMs": 109680,
    "text": "organization."
  },
  {
    "startMs": 111439,
    "text": "And if you want to implement more"
  },
  {
    "startMs": 113360,
    "text": "complex agentic systems, you actually"
  },
  {
    "startMs": 116000,
    "text": "need all of those things. So that is the"
  },
  {
    "startMs": 118880,
    "text": "layer that you see below here where um"
  },
  {
    "startMs": 121360,
    "text": "you have to deploy extra infrastructure."
  },
  {
    "startMs": 123600,
    "text": "Uh you need to write things like retry"
  },
  {
    "startMs": 125439,
    "text": "logic, recovery logic and all of that is"
  },
  {
    "startMs": 128080,
    "text": "actually pretty complex to get right but"
  },
  {
    "startMs": 131039,
    "text": "completely necessary to run longunning"
  },
  {
    "startMs": 133599,
    "text": "stateful and distributed processes in"
  },
  {
    "startMs": 135680,
    "text": "production."
  },
  {
    "startMs": 137520,
    "text": "So today I want to talk about an"
  },
  {
    "startMs": 139360,
    "text": "open-source framework called restate."
  },
  {
    "startMs": 141599,
    "text": "And you can see it a bit as a flexible"
  },
  {
    "startMs": 143840,
    "text": "durable foundation that lets you build"
  },
  {
    "startMs": 146160,
    "text": "any backend. So it's not specific for"
  },
  {
    "startMs": 148560,
    "text": "agents but a as agents are also just a"
  },
  {
    "startMs": 152239,
    "text": "type of a backend uh it also works well"
  },
  {
    "startMs": 154800,
    "text": "for them. The ideas behind restate come"
  },
  {
    "startMs": 157840,
    "text": "from Apache Flink which is a popular"
  },
  {
    "startMs": 160239,
    "text": "distributed stream processing engine and"
  },
  {
    "startMs": 162879,
    "text": "also from some of the exarchitects"
  },
  {
    "startMs": 165440,
    "text": "behind Meta Score event infra."
  },
  {
    "startMs": 169120,
    "text": "So what are the ingredients in restate?"
  },
  {
    "startMs": 171519,
    "text": "Basically four parts. First of all, it"
  },
  {
    "startMs": 174879,
    "text": "makes sure that a single run of an agent"
  },
  {
    "startMs": 176959,
    "text": "is resilient. This is called durable"
  },
  {
    "startMs": 179680,
    "text": "execution in the industry. Think about"
  },
  {
    "startMs": 182239,
    "text": "things like when an agent runs for a"
  },
  {
    "startMs": 184239,
    "text": "week and then crashes. We want to be"
  },
  {
    "startMs": 186560,
    "text": "able to bring it back and let it"
  },
  {
    "startMs": 188879,
    "text": "continue exactly at the point where it"
  },
  {
    "startMs": 190800,
    "text": "failed. We don't want it to start over"
  },
  {
    "startMs": 192640,
    "text": "from the beginning."
  },
  {
    "startMs": 194720,
    "text": "Another um area here is running many"
  },
  {
    "startMs": 198080,
    "text": "concurrent sessions in parallel. Imagine"
  },
  {
    "startMs": 200720,
    "text": "running thousands of concurrent agent"
  },
  {
    "startMs": 202480,
    "text": "sessions at the same time and needing"
  },
  {
    "startMs": 204480,
    "text": "needing to make sure that state is"
  },
  {
    "startMs": 206480,
    "text": "always consistent and that different"
  },
  {
    "startMs": 208159,
    "text": "agents don't interfere with each other."
  },
  {
    "startMs": 211440,
    "text": "And then going more towards things like"
  },
  {
    "startMs": 213360,
    "text": "communication between agents, between"
  },
  {
    "startMs": 215200,
    "text": "agents and MCP servers and other tools."
  },
  {
    "startMs": 218000,
    "text": "And finally also control, making sure"
  },
  {
    "startMs": 220480,
    "text": "that when an agent for example uh is"
  },
  {
    "startMs": 223040,
    "text": "doing um something you don't want it to"
  },
  {
    "startMs": 225440,
    "text": "continue or when it's stuck being able"
  },
  {
    "startMs": 227120,
    "text": "to actually cancel or kill the"
  },
  {
    "startMs": 228799,
    "text": "execution."
  },
  {
    "startMs": 231280,
    "text": "So the way that you can think of it is"
  },
  {
    "startMs": 233599,
    "text": "as follows. Restate is basically a"
  },
  {
    "startMs": 235760,
    "text": "server which runs in front of your agent"
  },
  {
    "startMs": 238080,
    "text": "service. So as a separate component it"
  },
  {
    "startMs": 240720,
    "text": "sits there a bit like a like a message"
  },
  {
    "startMs": 242720,
    "text": "broker or a proxy and when there's a"
  },
  {
    "startMs": 245280,
    "text": "request for your agent restate proxies"
  },
  {
    "startMs": 247840,
    "text": "the request to the service and pushes it"
  },
  {
    "startMs": 250640,
    "text": "to the service basically and from that"
  },
  {
    "startMs": 252959,
    "text": "moment there's a connection open"
  },
  {
    "startMs": 255280,
    "text": "connection between restate and the agent"
  },
  {
    "startMs": 257440,
    "text": "and that connection will basically be a"
  },
  {
    "startMs": 259519,
    "text": "bit like a lifeline for the agent. So as"
  },
  {
    "startMs": 261600,
    "text": "the agent is doing stuff, it sends"
  },
  {
    "startMs": 264240,
    "text": "events over to restate and restate will"
  },
  {
    "startMs": 267280,
    "text": "use that journal of events to recover"
  },
  {
    "startMs": 269840,
    "text": "the process after a failure."
  },
  {
    "startMs": 273040,
    "text": "So from a slightly higher level um"
  },
  {
    "startMs": 275680,
    "text": "explanation, you could say that it's"
  },
  {
    "startMs": 277520,
    "text": "turning a normal function in your"
  },
  {
    "startMs": 279280,
    "text": "application into something that is long"
  },
  {
    "startMs": 281600,
    "text": "running, durable, and stateful without"
  },
  {
    "startMs": 284240,
    "text": "having to do um a lot of the complex"
  },
  {
    "startMs": 286639,
    "text": "things you otherwise need to do for"
  },
  {
    "startMs": 288160,
    "text": "this. So my talk today will be mainly a"
  },
  {
    "startMs": 291600,
    "text": "demo. So I'll be showing you um a"
  },
  {
    "startMs": 294240,
    "text": "research agent that is connected to"
  },
  {
    "startMs": 295919,
    "text": "Slack. Imagine we are like working at"
  },
  {
    "startMs": 298160,
    "text": "some company and we want to make an"
  },
  {
    "startMs": 300000,
    "text": "Slack agent available to all of our"
  },
  {
    "startMs": 302240,
    "text": "employees."
  },
  {
    "startMs": 303840,
    "text": "So if I go here into Slack then can I"
  },
  {
    "startMs": 306800,
    "text": "can here in this channel for example ask"
  },
  {
    "startMs": 309520,
    "text": "what is new in AI."
  },
  {
    "startMs": 312800,
    "text": "Now let's have a look at what it's doing"
  },
  {
    "startMs": 314560,
    "text": "under the hood. So if I go back here, I"
  },
  {
    "startMs": 317199,
    "text": "have here the restate uh UI. This is a"
  },
  {
    "startMs": 320080,
    "text": "bit like a cockpit for your agents. So"
  },
  {
    "startMs": 322960,
    "text": "you can see a registry of all the agents"
  },
  {
    "startMs": 325520,
    "text": "that are currently registered and you"
  },
  {
    "startMs": 327600,
    "text": "can also see for example which execution"
  },
  {
    "startMs": 329759,
    "text": "is currently happening. So here is the"
  },
  {
    "startMs": 332000,
    "text": "deep research agent that I spinned up a"
  },
  {
    "startMs": 334639,
    "text": "few seconds ago. We can see what it's"
  },
  {
    "startMs": 337360,
    "text": "currently doing. Now it called first an"
  },
  {
    "startMs": 339440,
    "text": "LLM and then it sent me an answer via"
  },
  {
    "startMs": 342639,
    "text": "Slack. This first LLM call was a planner"
  },
  {
    "startMs": 345759,
    "text": "agent. So what it did is it planned the"
  },
  {
    "startMs": 348560,
    "text": "research and sent me um a list of"
  },
  {
    "startMs": 350720,
    "text": "subtopics that it wants to research."
  },
  {
    "startMs": 353840,
    "text": "Now if I press here approve then this"
  },
  {
    "startMs": 356000,
    "text": "will unblock the workflow and will spin"
  },
  {
    "startMs": 358639,
    "text": "up a set of parallel research agents. So"
  },
  {
    "startMs": 362160,
    "text": "this is basically like the classical"
  },
  {
    "startMs": 363919,
    "text": "deep research workflow, right? You have"
  },
  {
    "startMs": 365840,
    "text": "a planner then a set of subress research"
  },
  {
    "startMs": 368639,
    "text": "agents and then finally someone uh who"
  },
  {
    "startMs": 371360,
    "text": "writes a report on this like a writer"
  },
  {
    "startMs": 373759,
    "text": "agent"
  },
  {
    "startMs": 375680,
    "text": "and so this journal you see here on the"
  },
  {
    "startMs": 377440,
    "text": "left that is basically the events that"
  },
  {
    "startMs": 379360,
    "text": "get sent from the agent to the restate"
  },
  {
    "startMs": 381680,
    "text": "server and if this now crashes at some"
  },
  {
    "startMs": 384479,
    "text": "point this journal is what will be used"
  },
  {
    "startMs": 387280,
    "text": "to uh recover the execution to the point"
  },
  {
    "startMs": 389680,
    "text": "where it failed. I don't know if uh"
  },
  {
    "startMs": 391919,
    "text": "there were some errors. I injected a bit"
  },
  {
    "startMs": 394240,
    "text": "of like tool errors in here. Yeah, here"
  },
  {
    "startMs": 396800,
    "text": "you can for example see that um the sub"
  },
  {
    "startMs": 399919,
    "text": "agent first did an LLM call then started"
  },
  {
    "startMs": 402240,
    "text": "doing some web searches and eventually"
  },
  {
    "startMs": 404800,
    "text": "uh one of the web searches didn't go"
  },
  {
    "startMs": 406639,
    "text": "through because the API was down and"
  },
  {
    "startMs": 408720,
    "text": "then you see here on the right how it"
  },
  {
    "startMs": 410240,
    "text": "got retrieded and eventually completed"
  },
  {
    "startMs": 412240,
    "text": "successfully. So instead of starting"
  },
  {
    "startMs": 414319,
    "text": "over, it uses the journal to recover the"
  },
  {
    "startMs": 417280,
    "text": "progress."
  },
  {
    "startMs": 418720,
    "text": "Let's now have a look at what this looks"
  },
  {
    "startMs": 420240,
    "text": "like in code."
  },
  {
    "startMs": 422800,
    "text": "So the basic unit of how you implement"
  },
  {
    "startMs": 426000,
    "text": "applications in restate is by writing"
  },
  {
    "startMs": 427919,
    "text": "HTTP handlers and those handlers become"
  },
  {
    "startMs": 430800,
    "text": "durable by using the restate SDK. So"
  },
  {
    "startMs": 433840,
    "text": "here in this case we have here our deep"
  },
  {
    "startMs": 436000,
    "text": "research handler and here as a first"
  },
  {
    "startMs": 438160,
    "text": "argument we have a restate object"
  },
  {
    "startMs": 440160,
    "text": "context and the way you can imagine that"
  },
  {
    "startMs": 442479,
    "text": "is basically as that uh connection to"
  },
  {
    "startMs": 445360,
    "text": "that restate server. whenever I do an"
  },
  {
    "startMs": 448160,
    "text": "action on this uh restate object, it"
  },
  {
    "startMs": 450800,
    "text": "will lead to an event being sent to"
  },
  {
    "startMs": 452960,
    "text": "restate. So for example, when I did that"
  },
  {
    "startMs": 456720,
    "text": "planner LLM call, what actually happened"
  },
  {
    "startMs": 459360,
    "text": "under the hood was it executed here this"
  },
  {
    "startMs": 461520,
    "text": "Python function. This is just a simple"
  },
  {
    "startMs": 464000,
    "text": "light um light lm like"
  },
  {
    "startMs": 468479,
    "text": "LLM call and the way I made it durable"
  },
  {
    "startMs": 471039,
    "text": "is by wrapping it in restate.run."
  },
  {
    "startMs": 474879,
    "text": "So what happens is by doing these"
  },
  {
    "startMs": 476960,
    "text": "durable steps if this fails somewhere"
  },
  {
    "startMs": 479039,
    "text": "here two hours or two months later it"
  },
  {
    "startMs": 482000,
    "text": "will recover to exactly that point."
  },
  {
    "startMs": 485199,
    "text": "So that's the idea of durable execution."
  },
  {
    "startMs": 487280,
    "text": "You're always able to recover a process"
  },
  {
    "startMs": 489039,
    "text": "to where it was. You can also use that"
  },
  {
    "startMs": 491440,
    "text": "for other things not necessarily for"
  },
  {
    "startMs": 493759,
    "text": "failure recovery. For example, imagine"
  },
  {
    "startMs": 496160,
    "text": "we want to ask a human to approve"
  },
  {
    "startMs": 497919,
    "text": "something and this approval might take"
  },
  {
    "startMs": 500639,
    "text": "weeks or a month. this process needs to"
  },
  {
    "startMs": 503280,
    "text": "be able to um to survive restarts and"
  },
  {
    "startMs": 507199,
    "text": "redeploys uh over those kind of long"
  },
  {
    "startMs": 509599,
    "text": "periods of time and so with durable"
  },
  {
    "startMs": 512240,
    "text": "execution you can actually also uh"
  },
  {
    "startMs": 514959,
    "text": "suspend a function and let bring it back"
  },
  {
    "startMs": 518399,
    "text": "when it's able to make progress. So in"
  },
  {
    "startMs": 520719,
    "text": "the case of a human approval what we do"
  },
  {
    "startMs": 522479,
    "text": "here is basically we we create a durable"
  },
  {
    "startMs": 524800,
    "text": "promise which lives in that journal a"
  },
  {
    "startMs": 527600,
    "text": "bit like a suspension point. Then we ask"
  },
  {
    "startMs": 531360,
    "text": "uh a human to click that button in"
  },
  {
    "startMs": 533440,
    "text": "select as I showed in the beginning and"
  },
  {
    "startMs": 536000,
    "text": "while we are waiting this process"
  },
  {
    "startMs": 537839,
    "text": "actually suspends. So if it's running on"
  },
  {
    "startMs": 539760,
    "text": "serverless this is not using uh"
  },
  {
    "startMs": 542399,
    "text": "execution uh time on our functions."
  },
  {
    "startMs": 546720,
    "text": "Once the response comes in this then"
  },
  {
    "startMs": 548399,
    "text": "gets unblocked and can continue where it"
  },
  {
    "startMs": 550480,
    "text": "left off. So what we see here is a bit"
  },
  {
    "startMs": 553360,
    "text": "like a workflow. It's a set of steps"
  },
  {
    "startMs": 555680,
    "text": "that get executed durably. But when we"
  },
  {
    "startMs": 558720,
    "text": "think about agents and also the way that"
  },
  {
    "startMs": 560560,
    "text": "Karpathy described it in the tweet, it's"
  },
  {
    "startMs": 562480,
    "text": "more like a persistent stateful entity"
  },
  {
    "startMs": 564800,
    "text": "that lives for a longer period of time"
  },
  {
    "startMs": 567360,
    "text": "that has some memory. Um, so a workflow"
  },
  {
    "startMs": 570640,
    "text": "is not the nicest way to model this kind"
  },
  {
    "startMs": 572720,
    "text": "of thing. So the way that we can model"
  },
  {
    "startMs": 576880,
    "text": "this in restate is by using something"
  },
  {
    "startMs": 578560,
    "text": "called a virtual object. So imagine in"
  },
  {
    "startMs": 581120,
    "text": "the use case that I'm showing this slack"
  },
  {
    "startMs": 583040,
    "text": "research agent. Imagine that I don't"
  },
  {
    "startMs": 585440,
    "text": "want to wait for 10 minutes to give it"
  },
  {
    "startMs": 588480,
    "text": "some follow-up context or maybe I think"
  },
  {
    "startMs": 591120,
    "text": "about something else that I should have"
  },
  {
    "startMs": 592560,
    "text": "told it. Um I want to actually be able"
  },
  {
    "startMs": 594800,
    "text": "to interact with it, not wait till that"
  },
  {
    "startMs": 596959,
    "text": "research is finished before I can send a"
  },
  {
    "startMs": 599360,
    "text": "follow-up."
  },
  {
    "startMs": 600959,
    "text": "And so this is basically what a virtual"
  },
  {
    "startMs": 603120,
    "text": "object in restate is. It's a bit like a"
  },
  {
    "startMs": 605120,
    "text": "stateful actor. It has a unique ID, for"
  },
  {
    "startMs": 607760,
    "text": "example, a session ID. It has uh some"
  },
  {
    "startMs": 610640,
    "text": "key value states that is isolated for"
  },
  {
    "startMs": 614079,
    "text": "that specific session that you can write"
  },
  {
    "startMs": 616000,
    "text": "to. Uh imagine for example your history"
  },
  {
    "startMs": 618720,
    "text": "of messages and it also has like a set"
  },
  {
    "startMs": 621600,
    "text": "of handlers that can execute durable"
  },
  {
    "startMs": 625040,
    "text": "functions uh for this session. So here"
  },
  {
    "startMs": 628800,
    "text": "the way I implemented this use case that"
  },
  {
    "startMs": 630959,
    "text": "I mentioned of interacting with a"
  },
  {
    "startMs": 633839,
    "text": "running process is as follows. This is a"
  },
  {
    "startMs": 637360,
    "text": "um a bit a session controller. Again, it"
  },
  {
    "startMs": 640399,
    "text": "has like this restate object context at"
  },
  {
    "startMs": 642480,
    "text": "its disposal to do things in a"
  },
  {
    "startMs": 645440,
    "text": "recoverable way. Uh it can write to this"
  },
  {
    "startMs": 648480,
    "text": "session store. Here it I'm retrieving"
  },
  {
    "startMs": 650640,
    "text": "the chat history."
  },
  {
    "startMs": 652720,
    "text": "And one thing that's interesting there"
  },
  {
    "startMs": 654240,
    "text": "is that in order to run these kind of"
  },
  {
    "startMs": 656480,
    "text": "sessions in very high uh paralyzed ways,"
  },
  {
    "startMs": 659920,
    "text": "so thousands of sessions at the same"
  },
  {
    "startMs": 661600,
    "text": "time, we need to make sure that agents"
  },
  {
    "startMs": 664320,
    "text": "do not interfere with each other."
  },
  {
    "startMs": 666640,
    "text": "Imagine I'm sending two messages on"
  },
  {
    "startMs": 668240,
    "text": "Slack and now two agents are actually"
  },
  {
    "startMs": 670160,
    "text": "overwriting each other each other's"
  },
  {
    "startMs": 672240,
    "text": "session state. To prevent that, this"
  },
  {
    "startMs": 675839,
    "text": "will guarantee that only one execution"
  },
  {
    "startMs": 678000,
    "text": "is running at a time. So a second"
  },
  {
    "startMs": 679839,
    "text": "execution will be cued behind the"
  },
  {
    "startMs": 682640,
    "text": "current one."
  },
  {
    "startMs": 686959,
    "text": "Then let's have a look at how we"
  },
  {
    "startMs": 688640,
    "text": "implement this like interacting with"
  },
  {
    "startMs": 690320,
    "text": "another execution. So an execution in"
  },
  {
    "startMs": 692959,
    "text": "reset has a unique identifier and you"
  },
  {
    "startMs": 695440,
    "text": "can use that identifier to connect to it"
  },
  {
    "startMs": 698480,
    "text": "from other processes. for example, to"
  },
  {
    "startMs": 700880,
    "text": "retrieve uh the output, but also to"
  },
  {
    "startMs": 703440,
    "text": "cancel it or maybe to signal it being"
  },
  {
    "startMs": 706800,
    "text": "injecting a bit of state into an already"
  },
  {
    "startMs": 709440,
    "text": "running agent loop. And so this is like"
  },
  {
    "startMs": 712959,
    "text": "a very flexible type of um uh"
  },
  {
    "startMs": 716320,
    "text": "capabilities that you can do to"
  },
  {
    "startMs": 717760,
    "text": "implement things like for example"
  },
  {
    "startMs": 720160,
    "text": "signaling an already ongoing agent loop."
  },
  {
    "startMs": 722880,
    "text": "So what we do here is if there is a"
  },
  {
    "startMs": 724800,
    "text": "current execution ongoing then we will"
  },
  {
    "startMs": 727920,
    "text": "ask an LLM is this like something that"
  },
  {
    "startMs": 730720,
    "text": "is relevant for the current agent loop."
  },
  {
    "startMs": 733200,
    "text": "If that is the case inject this via a"
  },
  {
    "startMs": 736399,
    "text": "signal if it's not really relevant for"
  },
  {
    "startMs": 739120,
    "text": "what we're currently doing then cancel"
  },
  {
    "startMs": 741120,
    "text": "what you're currently doing and start"
  },
  {
    "startMs": 742800,
    "text": "over again with this new information."
  },
  {
    "startMs": 746160,
    "text": "And so this goes a little bit further"
  },
  {
    "startMs": 748000,
    "text": "than workflows. it goes a bit more"
  },
  {
    "startMs": 749760,
    "text": "towards like writing persistent stateful"
  },
  {
    "startMs": 752399,
    "text": "entities that can interact with each"
  },
  {
    "startMs": 754160,
    "text": "other and have memory at uh their"
  },
  {
    "startMs": 756720,
    "text": "disposal. So let me show you uh how this"
  },
  {
    "startMs": 760320,
    "text": "works. So here if I now ask again what"
  },
  {
    "startMs": 762959,
    "text": "is new in AI and I wait a few seconds"
  },
  {
    "startMs": 766720,
    "text": "then it should respond again with a"
  },
  {
    "startMs": 768800,
    "text": "plan. Um and then I can say for example"
  },
  {
    "startMs": 771839,
    "text": "some extra info focus on frontier models"
  },
  {
    "startMs": 775760,
    "text": "let's say."
  },
  {
    "startMs": 779120,
    "text": "So once I have the plan I will inject"
  },
  {
    "startMs": 782160,
    "text": "that bit of extra state."
  },
  {
    "startMs": 786720,
    "text": "Now let's look at the UI of what this is"
  },
  {
    "startMs": 789200,
    "text": "now doing. So here I have that"
  },
  {
    "startMs": 790880,
    "text": "controller which I just showed. It"
  },
  {
    "startMs": 793440,
    "text": "started calling an LLM to classify uh"
  },
  {
    "startMs": 796560,
    "text": "this new input."
  },
  {
    "startMs": 799680,
    "text": "Once this comes back, it will probably"
  },
  {
    "startMs": 801600,
    "text": "decide that it should signal it because"
  },
  {
    "startMs": 803440,
    "text": "it's it's still relevant to the research"
  },
  {
    "startMs": 805279,
    "text": "it's currently doing. So this inject"
  },
  {
    "startMs": 808320,
    "text": "that new message into the ongoing agent"
  },
  {
    "startMs": 810880,
    "text": "loop. So let me show you in the deep"
  },
  {
    "startMs": 812880,
    "text": "research agent again. Um so first it"
  },
  {
    "startMs": 815920,
    "text": "called an LLM then asked us then we"
  },
  {
    "startMs": 818399,
    "text": "injected this uh new message of focus on"
  },
  {
    "startMs": 821200,
    "text": "frontier models and then it uh took that"
  },
  {
    "startMs": 824160,
    "text": "into account and started over again."
  },
  {
    "startMs": 827440,
    "text": "Here"
  },
  {
    "startMs": 829279,
    "text": "I can now for example also say something"
  },
  {
    "startMs": 831920,
    "text": "like uh forget about that"
  },
  {
    "startMs": 837279,
    "text": "research AI policy."
  },
  {
    "startMs": 840160,
    "text": "And if I send this then the coord"
  },
  {
    "startMs": 842480,
    "text": "coordinator will um decide to cancel the"
  },
  {
    "startMs": 845360,
    "text": "ongoing run and start a new one that"
  },
  {
    "startMs": 848079,
    "text": "will"
  },
  {
    "startMs": 850000,
    "text": "research this new topic. And so this"
  },
  {
    "startMs": 852160,
    "text": "cancellation is basically like a signal"
  },
  {
    "startMs": 854240,
    "text": "that gets um sent down the stack of or"
  },
  {
    "startMs": 859199,
    "text": "the call chain. So if my agent was"
  },
  {
    "startMs": 861360,
    "text": "already spinning up sub agents first"
  },
  {
    "startMs": 863199,
    "text": "those sub aents would be cancelled then"
  },
  {
    "startMs": 865839,
    "text": "uh the controller itself and like that"
  },
  {
    "startMs": 868000,
    "text": "it would basically rewind the stack and"
  },
  {
    "startMs": 870560,
    "text": "give agents also the ability to roll"
  },
  {
    "startMs": 872480,
    "text": "back."
  },
  {
    "startMs": 874480,
    "text": "Okay. Okay, so this went a bit more into"
  },
  {
    "startMs": 876240,
    "text": "the direction of like stateful"
  },
  {
    "startMs": 877680,
    "text": "persistent entities that we can interact"
  },
  {
    "startMs": 880320,
    "text": "with over longer periods of time. Now"
  },
  {
    "startMs": 882399,
    "text": "the last part of the demo that I want to"
  },
  {
    "startMs": 884240,
    "text": "show is um going more towards like being"
  },
  {
    "startMs": 887199,
    "text": "able to write highly customized"
  },
  {
    "startMs": 889360,
    "text": "applications. Imagine that we deploy"
  },
  {
    "startMs": 891839,
    "text": "this in production but then a few months"
  },
  {
    "startMs": 894160,
    "text": "later a new model provider brings out a"
  },
  {
    "startMs": 896560,
    "text": "new model for example fabulous and even"
  },
  {
    "startMs": 899519,
    "text": "though the model is very good it's also"
  },
  {
    "startMs": 901680,
    "text": "very expensive and we notice that this"
  },
  {
    "startMs": 903839,
    "text": "research agent is actually starting to"
  },
  {
    "startMs": 905600,
    "text": "cost a lot. These kind of uh things that"
  },
  {
    "startMs": 908959,
    "text": "pop up halfway through a project require"
  },
  {
    "startMs": 912399,
    "text": "you to then deploy a a lot of new extra"
  },
  {
    "startMs": 915120,
    "text": "infra or like find a good way to solve"
  },
  {
    "startMs": 917120,
    "text": "this. This is the kind of things that"
  },
  {
    "startMs": 918720,
    "text": "Restate really excels at. It doesn't"
  },
  {
    "startMs": 921760,
    "text": "really peg you into a specific way of"
  },
  {
    "startMs": 923839,
    "text": "how you should write your application."
  },
  {
    "startMs": 925680,
    "text": "It basically gives you like a durable"
  },
  {
    "startMs": 927519,
    "text": "programming model that lets you"
  },
  {
    "startMs": 929519,
    "text": "implement an application in the way that"
  },
  {
    "startMs": 931600,
    "text": "fits for you and also extend it if"
  },
  {
    "startMs": 934240,
    "text": "necessary. So first I showed this um LLM"
  },
  {
    "startMs": 939360,
    "text": "call in the first example as an inline"
  },
  {
    "startMs": 941519,
    "text": "step. It was just a Python function that"
  },
  {
    "startMs": 944240,
    "text": "got persisted. But imagine this use case"
  },
  {
    "startMs": 947519,
    "text": "that we want to actually have a bit more"
  },
  {
    "startMs": 949120,
    "text": "control over those LLM calls. For"
  },
  {
    "startMs": 951360,
    "text": "example, what you can do is then pull"
  },
  {
    "startMs": 953120,
    "text": "this out into its own handler."
  },
  {
    "startMs": 956160,
    "text": "And this handler can now do things like"
  },
  {
    "startMs": 958560,
    "text": "for example a policy check and then uh"
  },
  {
    "startMs": 961120,
    "text": "do the LLM call. And the other agents"
  },
  {
    "startMs": 964639,
    "text": "instead of doing this LLM call inline"
  },
  {
    "startMs": 966880,
    "text": "can now use restates like distributed"
  },
  {
    "startMs": 969920,
    "text": "communication primitives to actually"
  },
  {
    "startMs": 971759,
    "text": "just call this LLM gateway instead of"
  },
  {
    "startMs": 974480,
    "text": "doing it as an inline step. And this"
  },
  {
    "startMs": 978000,
    "text": "service fabric that lets you communicate"
  },
  {
    "startMs": 980320,
    "text": "between agents also gives you some um"
  },
  {
    "startMs": 983199,
    "text": "things like flow control. So we can for"
  },
  {
    "startMs": 985680,
    "text": "example say one department is only"
  },
  {
    "startMs": 988240,
    "text": "allowed to run 300 calls to this LLM"
  },
  {
    "startMs": 991680,
    "text": "gateway at the same time. So the reason"
  },
  {
    "startMs": 994160,
    "text": "why I showed this was just to show you a"
  },
  {
    "startMs": 996160,
    "text": "bit like that. Uh it's basically just a"
  },
  {
    "startMs": 998399,
    "text": "a resilient foundation. It makes sure"
  },
  {
    "startMs": 1000399,
    "text": "that your process can uh recover from"
  },
  {
    "startMs": 1003920,
    "text": "even a more advanced types of"
  },
  {
    "startMs": 1005519,
    "text": "infrastructure failures, things like"
  },
  {
    "startMs": 1007279,
    "text": "network partitions and zombie failures."
  },
  {
    "startMs": 1010000,
    "text": "And um it gives you like tooling to"
  },
  {
    "startMs": 1013199,
    "text": "extend and customize as your use case"
  },
  {
    "startMs": 1015680,
    "text": "grows."
  },
  {
    "startMs": 1017839,
    "text": "Let's go back to the slides to have a"
  },
  {
    "startMs": 1020240,
    "text": "little more of an idea of how this thing"
  },
  {
    "startMs": 1023360,
    "text": "is actually implemented on the inside"
  },
  {
    "startMs": 1025039,
    "text": "because it's actually a pretty"
  },
  {
    "startMs": 1026799,
    "text": "interesting um design or architecture."
  },
  {
    "startMs": 1031439,
    "text": "So the way it's implemented is basically"
  },
  {
    "startMs": 1033520,
    "text": "by having a a event-driven distributed"
  },
  {
    "startMs": 1036559,
    "text": "log implementation."
  },
  {
    "startMs": 1039039,
    "text": "So inside the box you basically on one"
  },
  {
    "startMs": 1040959,
    "text": "side have the clients on the other side"
  },
  {
    "startMs": 1042720,
    "text": "the services and inside the box is a log"
  },
  {
    "startMs": 1046079,
    "text": "which persists all those journal events"
  },
  {
    "startMs": 1048400,
    "text": "and an event loop and that event loop"
  },
  {
    "startMs": 1050880,
    "text": "basically gets the events from the"
  },
  {
    "startMs": 1052960,
    "text": "service based on what the event is. It"
  },
  {
    "startMs": 1055679,
    "text": "either persists some state in the"
  },
  {
    "startMs": 1057840,
    "text": "embedded state store or it sets a timer"
  },
  {
    "startMs": 1060320,
    "text": "or it sends a request to another agent."
  },
  {
    "startMs": 1062799,
    "text": "And by doing that you basically have a"
  },
  {
    "startMs": 1065039,
    "text": "durable um foundation for whatever an"
  },
  {
    "startMs": 1068160,
    "text": "application is doing."
  },
  {
    "startMs": 1070799,
    "text": "The design of this distributed log is"
  },
  {
    "startMs": 1073520,
    "text": "heavily inspired by the way that the"
  },
  {
    "startMs": 1076640,
    "text": "core event infra layer at meta works. Uh"
  },
  {
    "startMs": 1079679,
    "text": "it's basically like an iteration on top"
  },
  {
    "startMs": 1082000,
    "text": "of that. Um and some of those architects"
  },
  {
    "startMs": 1084880,
    "text": "are now have designed that for restate"
  },
  {
    "startMs": 1087440,
    "text": "as an more generic solution that is"
  },
  {
    "startMs": 1090080,
    "text": "available in open source. There are two"
  },
  {
    "startMs": 1092559,
    "text": "important things related to this"
  },
  {
    "startMs": 1094559,
    "text": "architecture that make it interesting."
  },
  {
    "startMs": 1096799,
    "text": "The first one is that it works as a push"
  },
  {
    "startMs": 1098720,
    "text": "model. So whereas most workflow"
  },
  {
    "startMs": 1101039,
    "text": "orchestrators actually pull for new"
  },
  {
    "startMs": 1103600,
    "text": "tasks um for pull from the workflow"
  },
  {
    "startMs": 1106799,
    "text": "server, restate actually pushes the"
  },
  {
    "startMs": 1109679,
    "text": "invocations and the benefit you get from"
  },
  {
    "startMs": 1112000,
    "text": "that is that it has a much lower"
  },
  {
    "startMs": 1113760,
    "text": "latency. So you can use these kind of"
  },
  {
    "startMs": 1116400,
    "text": "workflow guarantees in functions around"
  },
  {
    "startMs": 1118880,
    "text": "your application and uh have like a"
  },
  {
    "startMs": 1121520,
    "text": "latencies of for example 45 milliseconds"
  },
  {
    "startMs": 1124720,
    "text": "p99 for like a 10-step workflow."
  },
  {
    "startMs": 1128400,
    "text": "Pushing invocations also works very well"
  },
  {
    "startMs": 1130640,
    "text": "for serverless because they require you"
  },
  {
    "startMs": 1133919,
    "text": "to basically uh send the request and"
  },
  {
    "startMs": 1136720,
    "text": "wake up the function. So this design"
  },
  {
    "startMs": 1139840,
    "text": "that I show here includes everything you"
  },
  {
    "startMs": 1142799,
    "text": "need. It includes uh as well that state"
  },
  {
    "startMs": 1145520,
    "text": "store where we were embedding the state"
  },
  {
    "startMs": 1147520,
    "text": "as the UI. It's a single binary so it's"
  },
  {
    "startMs": 1150320,
    "text": "pretty easy to operate as well to run it"
  },
  {
    "startMs": 1153039,
    "text": "in like a highly available way. You just"
  },
  {
    "startMs": 1155280,
    "text": "spin it up multiple times and let it"
  },
  {
    "startMs": 1157600,
    "text": "snapshot to object storage."
  },
  {
    "startMs": 1161440,
    "text": "So restate has six different SDKs. We"
  },
  {
    "startMs": 1164559,
    "text": "also have integrations for most of the"
  },
  {
    "startMs": 1166880,
    "text": "popular agent frameworks out there. And"
  },
  {
    "startMs": 1170160,
    "text": "of course, because it's just like a"
  },
  {
    "startMs": 1172080,
    "text": "flexible layer, you can also just use"
  },
  {
    "startMs": 1174080,
    "text": "any LLM SDK and implement custom agents"
  },
  {
    "startMs": 1177360,
    "text": "by just wrapping some steps into uh"
  },
  {
    "startMs": 1180000,
    "text": "these SDK constructs. So, it's open"
  },
  {
    "startMs": 1182880,
    "text": "source. You can self-host it. We also"
  },
  {
    "startMs": 1184720,
    "text": "have a BYOC offering where we deploy"
  },
  {
    "startMs": 1187520,
    "text": "restate in your cloud account and uh"
  },
  {
    "startMs": 1191039,
    "text": "that gives you the benefit that data"
  },
  {
    "startMs": 1192880,
    "text": "doesn't leave your cloud account."
  },
  {
    "startMs": 1194559,
    "text": "Otherwise, there's also a managed cloud"
  },
  {
    "startMs": 1196720,
    "text": "offering."
  },
  {
    "startMs": 1198799,
    "text": "This was mainly what I wanted to show."
  },
  {
    "startMs": 1201039,
    "text": "If you want to explore the code a bit"
  },
  {
    "startMs": 1202720,
    "text": "further, there is here this the GitHub"
  },
  {
    "startMs": 1204960,
    "text": "repo. It's publicly available. If you"
  },
  {
    "startMs": 1207360,
    "text": "like the project, then have a look at"
  },
  {
    "startMs": 1209440,
    "text": "the restate repo itself. We are hiring"
  },
  {
    "startMs": 1212640,
    "text": "across the board for all sorts of roles"
  },
  {
    "startMs": 1214720,
    "text": "going from engineering to marketing,"
  },
  {
    "startMs": 1216720,
    "text": "especially also here in the Bay Area."
  },
  {
    "startMs": 1218799,
    "text": "So, if you're interested in that, uh,"
  },
  {
    "startMs": 1220640,
    "text": "then definitely check out our careers"
  },
  {
    "startMs": 1222559,
    "text": "page. and I will be outside in front of"
  },
  {
    "startMs": 1225280,
    "text": "the conference hall here if you want to"
  },
  {
    "startMs": 1227679,
    "text": "ask any questions or learn more about"
  },
  {
    "startMs": 1229840,
    "text": "restate. Thank you very much."
  },
  {
    "startMs": 1247789,
    "text": ">> [music]"
  }
]