AI Engineer NYC 2026 Online Talks
Your Agent Is a Lethal Trifecta (And What to Do About It) — Kim Maida, Keycard
Read the talk
Your Agent Is a Lethal Trifecta (And What to Do About It)
Kim Maida follows an overprivileged on-call agent from leaked customer data to disruptive infrastructure changes, then repeats the work with delegated identity, task-scoped token exchange and policy-backed approval.
From a talk by Kim Maida
At a glance
Ideas worth remembering
Private data, untrusted content and external tools make agents useful while creating opportunities for harmful data movement and actions. Broad standing credentials turn those opportunities into executable authority.
Delegated token exchange lets a security token service evaluate the user, agent, requested resource and action before issuing short-lived access, with another exchange when an MCP tool needs downstream credentials.
Keycard’s proposed intent-based authorization adds a mission with a goal, allowed actions and limits. The second demo instead uses task-based policy governance.
Human approval remains subject to authorization: the restart stays blocked because the user lacks the role, while approved scaling proceeds because the user has authority.
An on-call agent with enough access to cause an incident
A billing anomaly arrives overnight: a batch of customers has been overcharged. In Kim Maida’s staged on-call triage demo, the agent reads the incident through an API key, queries a customer warehouse using a Snowflake key, and escalates the problem to the morning team. Its report includes rows pulled directly from the warehouse, carrying personally identifiable information into the incident workflow. The agent has completed a plausible triage task while moving private data somewhere it need not go.
The same queue includes work that fits the agent well. A failed backup power supply is a physical problem, so the agent escalates it without calling another service. An expiring website TLS certificate is actionable through the cloud hosting API: the agent calls the renewal endpoint with its API key, renews the certificate, and posts a report. Both cases show why an organization might want this system running overnight.
Then payments fail because the billing database is broken. The documented recovery procedure says to drop and restore the database. A connection string containing an administrator password gives the agent the power to take the destructive step. It then asks the morning team to check that snapshot recovery worked. Successful restoration is not established here: the agent cannot verify that part itself, yet it has already exercised the privilege that puts the data at risk.
The remaining tickets turn infrastructure permissions into operational consequences. A frozen main server prompts an API call to restart it, bringing production offline. A site failing for one in three visitors prompts a scale-up from four pods to 40, incurring $420 per hour. The agent “kindly leaves a note” for the morning team after committing the expense. What does that jump in capacity and cost look like in the demo’s record? The useful comparison is between the requested recovery and the authority already available to execute it: no new credential decision intervenes before the change.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three useful capabilities create the exposure
The demo brings together the capabilities that make agents useful. They can read information a user owns, research material outside that private context, and act through tools. Maida uses Simon Willison’s term “lethal trifecta” for the dangerous combination:
- Private data access. The triage agent can query the customer warehouse and interact with the billing database.
- Untrusted content. Tickets, reports from other agents and externally documented procedures supply material the agent must interpret.
- External communication. Third-party APIs and SaaS services let the agent transmit information and make changes outside its own process.
An agent trying to finish a task can use whatever powers its credentials provide. That makes overprivilege consequential even with supervision. Requiring approval for everything transfers decisions to a potentially consent-fatigued user; it also fails to address agents that run autonomously. A human approval button needs an access-control system behind it.
Removing private data, outside information or tools would reduce the agent’s usefulness. Maida’s proposed direction keeps those capabilities and narrows the access available when the agent acts. The opening demo establishes the consequences of broad credentials; it does not demonstrate a particular attacker’s prompt-injection payload. The problem addressed next is the authority available to turn a decision into an action.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A grant, a route and a process leave different gaps
OAuth grants, MCP gateways and sandboxes each control something useful. The question is whether their controls produce a runtime credential limited to the current task. Maida distinguishes their usual points of enforcement:
- OAuth binds access to a grant. A user consents to permissions before a particular task exists. A broad grant leaves standing read and write access available even when the current request only calls for reading. A narrow grant can leave the agent unable to perform other legitimate work.
- An MCP gateway binds enforcement to a route. It can provide observability, rate limits and per-request policy decisions. In the setup described, however, the gateway still holds credentials whose authority outlives the task.
- A sandbox binds execution to a process. It limits what happens inside that process, but useful work often reaches external services. Process containment does not determine which external operations those services should authorize.
Combining those controls does not, by itself, issue a credential down-scoped to one task. The first demo also lacks identity attribution: an API key authorizes a call without revealing which user delegated it or which agent used it. The desired change therefore has two parts—carry identity through the execution path, and make a fresh authorization decision before granting the access needed for an operation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Carry the user’s identity into each tool request
Token exchange supplies a foundation for that change. Maida introduces it as an OAuth 2 extension standardized in 2020. First, an authorization server verifies the user’s identity. The user consents to delegating a subset of their permissions, called scopes, and the server issues tokens identifying that user and their access. This is the first narrowing: the agent receives delegated access rather than the user’s entire set of permissions.
Next, the triage application gains an MCP server that defines production infrastructure tools and brokers access to downstream resources. The runtime runs the agent loop: it sends the user’s prompt to the model, receives proposed tool calls, and dispatches them through an MCP client. The MCP server executes the tools and calls resource APIs. Responses travel back to the runtime, which calls the model again until it can answer the user. The model proposes operations; the surrounding software executes them.
The authenticated user’s token becomes the subject token, identifying whom the agent acts for. An OAuth client capable of running code performs the exchange; that could be the custom application, a gateway between the MCP client and server, or a custom MCP server obtaining access to a downstream resource. This places credential handling in executable infrastructure around the agent loop.
For a proposed tool call, the runtime authenticates to a security token service, or STS, using its application credentials or workload identity. It also supplies the user’s subject token and requests access to the MCP server for that call. The STS can now evaluate the requested action and resource alongside the agent’s identity, the user’s identity and the access the user delegated. A broad credential no longer has to serve as the entire authorization decision.
There is still a distinction between an action that policy permits and an action that the current request calls for. As Maida puts it, “policies are ceilings.” An agent may stay within its overall permissions while doing something the user never intended. That motivates a further restriction around the task itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Constrain the mission, then exchange credentials at each hop
Keycard’s approach to intent-based authorization, or IBA, gives the current request an explicit mission: a goal, allowed actions and limits. Maida identifies this as a proposal rather than a finalized standard. For “triage incident 2294,” the mission might permit reading that ticket, summarizing it or escalating it, while excluding other tickets and services. An administrator’s policy could automatically approve that low-risk assessment.
Remediation changes the mission. After reading a usage-spike ticket, the agent might request permission to scale the service once, without gaining access to other tools or APIs. That mission goes to the user for review. Once approved, the STS assigns a mission ID. Tool calls carry the ID through a gateway, which asks the STS whether each call fits the mission. Only a fitting call proceeds to a token-exchange request for MCP access.
The STS then evaluates policy, the delegation chain and the requested access. If it approves, it issues an MCP access token that preserves the user as subject and records the application performing the exchange. The token should expire within minutes and should not be stored for later reuse. The gateway uses it to call the MCP server, which validates it before authorizing the tool.
An MCP tool may need another service to finish its work. In that case, the MCP server requests a second exchange, using its received token to ask for a downstream resource token. The STS evaluates policy again and, if permitted, issues a token scoped to that tool call while preserving the user subject through the delegation chain. The resource validates the token; the MCP server discards it immediately after the call. The response returns through the agent loop.
Where do the two credential decisions occur? The diagram separates the mission check from the exchanges for MCP and resource access. The second exchange matters because permission to invoke a tool does not eliminate the downstream service’s own authorization step. Both tokens retain the user’s identity, making the chain attributable across services.
Two exchanges preserve identity across two access decisions
Read the diagram as text
- Agent through gateway. requester — Carries the approved mission ID with the proposed tool call.
- STS: mission check. authorization — Checks whether the call fits the mission before token exchange.
- STS: MCP token decision. authorization — Evaluates policy, delegation and access; preserves user subject and records the app.
- MCP server. execution — Validates the MCP token and requests downstream access when the tool needs it.
- STS: resource token decision. authorization — Rechecks policy and issues a tool-scoped resource token with the user subject.
- Downstream resource. execution — Validates its token and authorizes access; the MCP server discards the token after use.
- Agent through gateway → STS: mission check: Ask whether this call fits the mission.
- STS: mission check → STS: MCP token decision: If it fits, gateway requests exchange.
- STS: MCP token decision → MCP server: If approved, gateway calls MCP with issued token.
- MCP server → STS: resource token decision: If needed, exchange received token for resource access.
- STS: resource token decision → Downstream resource: If approved, issued token accompanies proxied resource call.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The same tickets, with credentials issued only when policy permits
The second demo implements delegated authorization with task-based policy governance; it does not demonstrate the proposed IBA mission system. The triage app now uses authentication and an MCP server to broker resource access. Maida signs into the app with Google, then signs into the incident platform and consents to queue reading, escalation, notes and summaries. A separate cloud-hosting login and consent step delegates some hosting capabilities.
The queue remains the same, and the agent can propose the same actions. What changes is the path to execution: requests pass through STS policy evaluation before credentials are granted. On the billing anomaly, the agent again tries to query the customer warehouse. This policy forbids issuing a credential because customer PII should not enter the agent’s context or an incident ticket. The agent continues without the data and escalates the incident to the morning team.
Certificate renewal remains useful and permitted. The STS grants the requested access, allowing the agent to call the cloud hosting API and resolve the ticket. Restricting credentials has preserved an appropriate automated repair while blocking the earlier data movement.
How does the audit log distinguish tool access from resource access? The renewal example follows both requests: the on-call agent first obtains permission to invoke the appropriate production infrastructure MCP tool; the MCP server then requests access to the hosting service’s renewal API. Those separate entries make the delegation path visible instead of collapsing the operation into use of a shared API key.
ACTION ACTOR ENTITY TIME
STS credentials:issue ALLOW On-Call Triage Agent Prod Infra MCP 22:03:45
{ grant: token-exchange; scope: certificates:renew; }
STS credentials:issue ALLOW Prod Infra MCP Cloud Hosting 22:03:47
{ grant: token-exchange; scope: certs:renew; resource: https://compuhost.cloud/api/v2; }
The billing-database ticket reaches the same proposed drop-and-restore procedure as before. This time, policy forbids an agent from dropping a production database and reserves that operation for a human to perform manually. The STS refuses the token exchange for database deletion. The agent’s conclusion has not changed; its ability to execute that conclusion has.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the transcript and explore slides
Loading transcript and slides…