AI Agents Are Just Distributed Systems Now — Salman Munaf, TikTok

A wrong model answer becomes a different engineering problem when the system can delete a database or make a refund decision. Salman Munaf, listed for this session as a Lead Site Reliability Engineer at TikTok, opens with two incident examples: a Replit agent deleting a production database and an Air Canada chatbot making what he describes as an incorrect refund. These are motivations for his proposed safeguards, rather than detailed incident reconstructions.

The examples expose two separate responsibilities:

• Limit destructive actions: Backups provide a recovery path, while scoped permissions can prevent an agent from deleting a production database in the first place.
• Retrieve authoritative policy: A chatbot making policy-dependent decisions needs a source of truth so that stale or incorrect information does not drive its answer.

The transition from chatbot to agent expands the architecture. A text-only interaction ends with an output; an agent loop can call tools, reach external services and change their state. The design inventory therefore has to include the systems the agent talks to, the state it reads or changes, the credentials it holds and the actions those credentials permit. Model quality covers only part of that system.

At 3:32, the useful architectural description is a probabilistic coordinator. Distributed services already coordinated multistep workflows, but the traditional comparison here is a mapped decision tree. An agent can vary both the kind and the number of actions it chooses. That flexibility requires deterministic controls around the actions whose consequences would be unacceptable.

The loop plans, acts, observes, may persist information and then decides what to do next. Planning can retrieve external data. Acting can call an API or write to a database. Observation may return only partial results, which then become inputs to the next decision. Persisting an incorrect observation can carry the mistake forward, and a decision to retry can multiply the work sent downstream.

Recording each step’s actions and retrieved context gives recovery a place to start: the workflow can identify where it failed and which earlier operations need attention. But recording an operation does not define its repair. Each step also needs an explicit failure response or compensating action. An incorrect email to a customer makes the distinction clear: knowing that the message was sent is necessary, yet the system still needs a decision about how to correct its consequences.

A tool call wraps an external API, database or queue. The wrapper inherits network delays, timeouts and duplicate requests. Crucially, the service may complete an operation even though the caller receives an error. A database write can exist in the authoritative store while the client’s observation suggests something went wrong.

Follow the refund example through that discrepancy. The agent invokes a customer-refund tool, then the request times out. The visible change is in the agent’s knowledge: it no longer has a confirmed outcome. The customer’s money may already have been refunded. Treating the timeout as proof of failure and issuing a second refund can therefore turn one intended operation into two financial effects.

Three tool capabilities address different parts of this problem:

• Request identifiers: Identify the earlier operation so that the workflow can refer to it again.
• Idempotency keys: Let the receiving tool recognize repeated requests and prevent repeated side effects.
• Status lookup: Check what happened to the earlier request against the system that owns its outcome.

How can a completed refund and an uncertain caller coexist? The diagram separates the operation’s effect from the response the agent receives. It also shows why recovery needs both an outcome lookup and duplicate protection: inspecting the earlier request resolves uncertainty, while recognizing a repeated operation prevents a retry from refunding the customer again.

Retrying is the failure response Munaf expects builders to plan around. Idempotency belongs in the receiving API or tool because the agent’s choice to retry does not establish that another side effect is safe. The tool must recognize the repeated request and avoid doing the operation again.

Even retries that produce no additional side effects still send traffic downstream. A large fan-out or aggressive retry loop can burden an external API and cause cascading failures. Maximum turns limit how long the loop continues; maximum parallel calls limit simultaneous work; spending ceilings limit the resource commitment. Exponential backoff increases the spacing between attempts, reducing pressure on dependencies while the workflow tries to recover.

These controls solve different problems. Idempotency prevents repeated effects, backoff and execution limits restrain repeated work, and compensation addresses the consequences of work that has already happened. None substitutes for the others.

At 10:25, the same systems thinking reaches memory. Context becomes state when it can influence an action. It can go stale, conflict with authoritative data and corrupt future decisions. A remembered policy is operationally consequential if it determines what the agent does next.

The memory distinction here concerns where information persists:

• Short-term memory: The chat thread belongs to a single execution thread.
• Long-term memory: Project files, system prompts, databases and cache layers can supply information beyond that thread.

Neither location settles which information should win when sources disagree.

Treating memory as a cache makes that decision explicit. Attach provenance so that remembered information has a known origin, choose the authoritative source when values conflict, and invalidate the corresponding memory when that source changes. The causal sequence matters: updating a database does not by itself remove the old value from the agent’s context. Invalidation prevents that retained value from continuing to drive actions.

A multistep workflow can update an internal ticket, send an email to a customer and then fail to update the CRM. The failure leaves the systems at different stages of completion. Recovery must consider the earlier ticket change and email alongside the failed CRM operation; simply treating the whole workflow as unsuccessful ignores effects that already exist.

What remains after the last step fails? The diagram preserves the successful earlier steps instead of collapsing the workflow into a single failure marker. That partial outcome is the input to compensation: the system needs a defined response for the consequences of each completed operation.

The email example also sets a limit on what recovery means. An incorrect message can be followed by an apology or a corrective email, but the original communication has already occurred. Compensation can repair its consequences through another action without restoring the exact prior state. Its meaning depends on the operation, so the agent needs that response defined rather than being left to improvise after failure.

Circuit breakers put a stopping condition around calls to unhealthy or saturated dependencies. An agent may keep trying to solve a problem, but the surrounding system should prevent those attempts from burdening a dependency that cannot support them. This helps contain cascading failures. Rate limits and maximum turns, parallelism and spending separately bound how much effort the agent can commit.

Those ceilings trade open-ended persistence for a predictable resource commitment. The agent can continue problem-solving only within the limits the system sets. A desire to complete the task cannot be allowed to expand traffic or cost indefinitely.

Permissions need the same specificity. Broad database read-and-write access may make it easy to equip an agent for a task, but it also makes unsafe decisions executable. Separate read and write permissions, scope credentials and allowlist the tools the agent can call. The permitted action set determines how much damage an incorrect decision can cause.

Human approval is another scoped permission. It should bind to the action and its parameters, the actor, timestamp and expiration. Approval for a $30 refund cannot become authorization for a subsequent $300 refund. Changing the amount changes the operation that was approved; a blanket approval loses precisely the information needed to enforce that distinction.

Observability has to reconstruct a workflow, including what information the agent reacted to. An isolated error log cannot explain whether a bad decision came from stale context, a tool response or an earlier write. Traces need to connect the model invoked and its prompt with retrieved context, tool requests, responses, errors, writes and approvals.

For the refund example, that connection distinguishes the request the agent attempted from the outcome it observed and the state the external system actually changed. For a policy-dependent action, it preserves the information used to decide. The purpose is to understand what happened and locate a recovery path across the model and the services around it.

The closing discussion at 17:31 retains a role for better models: greater capability improves the likelihood of correct operations. It cannot eliminate network failures, stale data or adversarial input. The architecture still needs to bound actions, observe their effects and recover when something goes wrong.

Tool contracts make those expectations concrete. Define the operations a tool permits, its request and response types and its schemas. Build idempotency into tools that receive repeated requests. Decide which source wins when memory conflicts, constrain retries with policies and rate limits, and provide permissions, traces and recovery paths. These are parts of the executable system around the coordinator.

The final design question is what the system lets the agent do when it is wrong. That question makes mistakes an expected operating condition. It asks whether an incorrect decision can become an unsafe operation, whether its effects can be understood and whether the workflow has a way back.