How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
Read the talk
How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
A private model covers only part of an agent’s data path. Joshua Mo traces how logs, credentials and tool calls create exposure—and how separating reasoning from execution changes what the model can leak.
From a talk by Joshua Mo
At a glance
Ideas worth remembering
Reducing retained sensitive data reduces what a breach can expose; application logs can recreate exposure even when the model provider retains no conversations.
Connect the credential vault to execution so the model can request an action without receiving the secret needed to perform it.
Narrow tool scope and prompt-injection screening address different risks. Internet-enabled sandboxes still require permission monitoring.
Attested confidential inference protects the reasoning environment; ergonomic, fully encrypted tool calls remain an unresolved part of the design presented here.
Privacy starts with what a breach can expose
A single breach can cost a product its users’ trust. For an enterprise vendor, it might mean losing the customer that supplies most of its revenue. Joshua Mo, Venice AI’s lead developer relations engineer and previously lead maintainer of the Rust AI framework Rig, opens with this practical reason to care about privacy: sensitive data increases the damage an incident can do. Storing less of it reduces the blast radius.
Venice supplies the opening example. Mo describes an OpenAI-compatible platform hosting open-source models, a chat application, and a studio for video, audio, image generation and transcription. Its stated policy is to retain no prompts or conversations. That makes retention a product architecture decision from the start: what the service keeps determines what a later compromise could reveal.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Different data paths need different protections
Venice’s privacy measures address several different places where data could remain or become visible. Mo describes these as distinct protections rather than a single property of the model:
- Conversation retention. Prompts and conversations are not saved on Venice’s side; conversation storage stays local.
- Requests to closed providers. Venice anonymizes the user footprint when forwarding requests to providers such as OpenAI and Claude. The talk does not explain the transformation, so this claim should not be read as a guarantee that identifying information inside a prompt disappears.
- Inference execution. Some models run in trusted execution environments, with evidence intended to let a user verify the protected inference process.
- Stored generation results. Video-generation endpoints require polling for a result. Venice stores that result as an encrypted payload using split-key custody, intended to prevent access even if an attacker compromises a team account.
The polling example exposes a useful distinction. Avoiding stored conversations does not eliminate every reason a service might need storage: an asynchronous job still needs to make its result available later. Encryption and key custody address that separate requirement. The model, the request path and the stored output each create a different privacy question.
An agent adds still more machinery. A model is one component inside a process that retrieves information, invokes tools and returns results. Each surrounding component can introduce another route for data leakage. Choosing a private inference service therefore leaves an application builder with substantial work elsewhere in the system.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Logs and vault integrations can copy secrets into new places
Observability is an early trap because its purpose is to preserve information about what happened. Mo separates metrics—numbers such as counters and gauges—from traces, which describe spans of work over time, and logs, which contain messages. Those messages are valuable when debugging production failures. They can also quietly retain prompts, personal information and credentials. A model provider’s retention policy does not prevent application logging from creating its own sensitive records.
The next example follows a credential through an agent. A user request reaches the runtime. To complete the task, the runtime retrieves a secret from a vault such as 1Password or HashiCorp Vault, then sends that secret to the LLM reasoning layer. The reasoning layer makes an API request through a tool call, and an answer returns to the user. The critical transfer happens before the API call: the credential has become model input.
Mo warns that this could expose the credential to training-data use. That possibility depends on the receiving provider’s policies; sending a secret to a model does not by itself establish that it will be used for training. The architectural exposure is clearer: the model receives a credential even though the eventual API request is the operation that needs it. Later in the talk, that same placement becomes the starting point for tool-based exfiltration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep identifiers small, split custody, and make access revocable
Rate limiting illustrates how an application can retain less data without abandoning a function it needs. The service must recognize requests associated with a user so it can apply a limit. Mo proposes storing a hash representing the needed identifier instead of directly storing the user’s personal data. The rate limiter still has an identifier to work with, while its record contains less direct personal information. This is a data-minimization pattern; the talk does not establish that every hashed identifier is anonymous.
Split-key custody addresses a different failure: someone obtains access to encrypted storage or part of its key material. Mo describes two key halves, called a read half and a write half, with both required for decryption. The intended protection comes from making one compromised part insufficient to recover the data. This is a high-level description of Venice’s custody approach, without enough cryptographic detail to reproduce the scheme.
Revocable handles move the discussion from identifying data to granting access. A handle is an object through which a caller can reach a resource. In the database-pool example, possession of the handle permits access; without it, access is unavailable. Making that access revocable gives the application a way to withdraw permission. Mo connects the pattern to authorization systems using JSON web tokens or session keys, although the recording does not specify a revocation implementation.
Privacy-conscious prompts also belong in the development workflow, including coding harnesses such as Cursor and Claude Code. But the next example moves beyond prompt wording to the scope of the agent itself: how many tools it can use, and what sensitive material accompanies those tools.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One needed tool should not bring twenty tools’ worth of exposure
Suppose an MCP server offers fifteen or twenty tools, but the task requires only one. That tool needs an API key. If the agent retrieves the key from a password vault and includes it in the model’s working material, a subsequent tool call can send it to a bad actor’s web server. The failure combines two kinds of access: the agent can see the secret, and it can call a destination that should never receive it. Narrowing tool scope reduces the opportunities available to the agent.
A classification layer in front of the agent adds a separate check for prompt injection. Mo gives several implementation options:
- LLM classification. Use a model-based layer to examine incoming material for injection attempts.
- Semantic search. Use vector search through systems such as Qdrant or LanceDB as part of the screening layer.
- An in-memory process. Keep the screening process in memory rather than requiring the named search systems.
These are proposed implementation choices. The recording does not specify classification thresholds, detection accuracy or how the application handles a flagged request.
Narrow scope also has a context benefit. Fewer tools mean less tool-related material competing for the model’s context window. Mo links this to avoiding performance degradation as the window fills. His advice to keep context nearly empty is best understood here as a push to remove unnecessary material; the talk provides no benchmark establishing an optimal context size. The useful connection is that limiting capabilities can reduce both exposure and irrelevant context.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let the model request an action; let execution supply the credential
The central architectural correction changes where the vault connects. A user request first reaches the reasoning layer, which formulates inputs for the agent runtime. The password vault connects to that runtime rather than supplying credentials to the model. The runtime becomes an execution layer: model-generated requests enter it, and the credential is supplied where the action runs.
A browser login makes the difference concrete. The task requires a website login, so something must use the user’s credentials. In this design, execution handles those credentials while carrying out the login; the reasoning layer does not need the password in its context. External API calls follow the same pattern. Mo calls the execution layer deterministic, contrasting its programmed behavior with the model’s nondeterministic outputs. The separation makes credential handling more controllable, while permission checks still matter.
Where does the secret enter the revised system? The diagram separates the model’s action inputs from the vault’s credential path. They meet at execution. That topology is the consequential change: the model can help choose an action without receiving the secret used to carry it out.
Action inputs and credentials meet at execution
Read the diagram as text
- User request. Ask the agent to perform a task.
- Reasoning layer. Formulate nondeterministic inputs for the runtime.
- Password vault. Connect credential access to the runtime.
- Execution layer. Use programmed execution to handle the requested action and its credentials.
- Website login or API call. Use the credential where the external operation runs.
- User request → Reasoning layer: Task request.
- Reasoning layer → Execution layer: Action inputs.
- Password vault → Execution layer: Credential access.
- Execution layer → Website login or API call: Perform authenticated operation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A sandbox can still send data out
Sandboxes and virtual machines strengthen isolation. Mo names Firecracker and EC2 deployments inside a VPC as examples. But isolation does not settle the question of outbound access: an agent with internet access can still leak data through an external API call. Monitoring permissions remains essential because an allowed network operation can carry sensitive content out of an otherwise isolated environment.
This motivates Mo’s recommendation to build and own the agent runtime or coding harness. Controlling the runtime lets a team add its own execution steps, observability and tools, and decide how reasoning connects to execution. The recommendation is about having a place to enforce those decisions: a sandbox alone cannot choose appropriate permissions or prevent sensitive values from entering logs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Attest inference, then confront the tool-call problem
Confidential compute moves privacy protection down to the hardware level. A trusted execution environment protects the inference process. In Venice’s described flow, a request produces a response and an attestation, which the user can check with an attestation verifier. The attestation supplies something inspectable about the protected reasoning environment, rather than requiring the user to rely only on a service’s privacy claim.
That protection has a defined scope: inference at the reasoning layer. Confidential tool calls remain an open engineering problem in the talk. Mo describes TEEs as useful for tool calls and says Venice also offers end-to-end encryption, then identifies the unresolved API challenge: returning entirely encrypted tool calls while preserving an ergonomic experience for developers. Protecting model execution does not automatically solve how external actions and their data move through the rest of an agent.
One proposed direction is for providers to own the compute layer where tool calls run. That would extend their role from inference into tool execution. Mo leaves the proposal undeveloped and closes with confidential tool calls still unfinished. The ending preserves the distinction the architecture requires: verifiable private reasoning is one achievement; carrying that protection through useful external actions requires further design.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Read the complete timestamped transcript
- 0:12
Hi, everyone. Uh, welcome to my talk. I'm Joshua Mo. And today, we're going to be talking about some of the challenges of building a privacy-respecting AI system. So just for an introduction as to who I am, I'm currently the lead developer relations engineer at Venice AI. So currently, I handle partnerships, documentation, talking to developers, um, some engineering,
- 0:42
and other things, other things as well.
- 0:48
Uh, so prior to this, I was actually the lead maintainer for Rig, which is the leading AI framework in Rust. Um, so what is Venice? So Venice is a private and unrestricted AI platform. Uh, so we host all the latest open source models. We're a hundred percent OpenAI compatible. We don't store any prompts or
- 1:17
conversations, and we also have a web app that you can use to talk to our models as well. We also have a studio offering that is, um, a offering for our content creators. That includes video, audio, image generation, and transcription.
- 1:42
So why does privacy matter? So let's di- deep dive into this. So, as we know, data breaches are increasingly becoming an eventuality, which means that essentially, reputational risk matters. You only need one data breach before your users stop trusting you and then move on to another provider. Or, if you're enterprise, maybe you lose that enterprise customer who's been
- 2:12
fueling almost ninety percent of your annual revenue. So in this case, sensitive data can severely increase the impact of any security incident. As well as that, storing less use, uh, user data directly reduces the blast radius of a security, security incident that happens. So how does Venice do privacy today? Well, there's a few ways.
- 2:42
The first way is that we don't store any prompts or conversations, of course, as we mentioned previously, which means that any digital footprint incurred in your side is sa- is essentially reduced to zero. So when you use models like GLM five point two and Kimi two point seven, you can guarantee that none of your conversations will be saved on our side. It'll be all stored locally. We also have predefined tiers of privacy. So private by default means that the aforementioned,
- 3:12
uh, measures apply. For providers like OpenAI and Claude, uh, we anonymize your user footprint so that none of-- Well, they can't identify you from your, simply from your prompt or your request. We also have models hosted in trusted execution environments, meaning that you can verifiably prove that your inference
- 3:42
was run in a totally private process. As well as that, we also use a privacy-first software architecture. So for endpoints like video, uh, generation, where you have to poll the endpoints, we store the results in a totally encrypted payload using split key custody, meaning that attackers can't access your data, and neither can the team should their accounts be compromised. So I
- 4:12
just want to introduce you to the guide- the, uh, the idea that not all systems are made equal. So not all systems are made private. Some companies may claim their systems are private, but they're not really. Uh, you guys know how it goes. So while models are a very important part of an agentic system, in fact, they're a vital part of the system, um, they are still only one part. And in fact, to
- 4:42
create a complete system to accompany an LLM in doing a task, you have to build an entire process around it. However, these systems are also much more prone to data leakage and breaches. So the real question is: How can we optimize for this kind of problem?
- 5:02
So one example I just wanted to bring up was observability logging. So observability, as many of you know, is a core part of any serious production system today. So just to break it down quickly, observability can be broken down into metrics, logs, and traces. So traces are simply a span of work, uh, done over a unit of time. Metrics as, are simply numbers like counters, gauges, for instance. And logs are the actual messages.
- 5:33
So logging, as we all know, is actually pretty great for debugging, especially in production or deve- dev environments. But you can also accidentally store prompts and secrets by default if you're not careful, which can also hold personal information. However, when you use Venice, you don't need to worry about any of that. So let's talk about how secrets can flow through a regular agent. So for example, let's say you're, you're
- 6:02
using OpenClaw. You send a user request that gets sent to an agent runtime. The runtime might need a secret that it will fetch from a 1Password vault, for instance, or HashiCorp Vault. The secret will then get sent to the LLM reasoning layer.
- 6:24
Um, oh my God.
- 6:31
So the reasoning layer will then make an API request through a tool call, which will then return the answer back to the user. However, due to this, the way that it's architected, the secrets get fed directly to the LLM, which means it then may get used in training data. So in terms of privacy preserving patterns that work, one thing that we've used at Venice is storing hashes that rep-
- 7:01
represent IDs that we need, as opposed to direct data. So for example, if you need to identify a user for rate limiting purposes, you might not want to use their direct user footprint directly. You might want to, you know, store it as a hash, for instance, just so you can identify them without directly using any of their personal data. As well as that, we encrypt data and store it in an inaccessible enclave using split key custody.
- 7:31
So for context, encryption keys are stored in two halves, a read half and a write half. By splitting these, the attacker cannot access the data due to needing both halves for the decryption to complete. Additionally, you can use design patterns like handles that can be easily revoked. So a handle is simply an object that grants you access to something. So for example, let's
- 8:01
say you have a database pool using a handle. So when you have the handle, you can access the database pool. When you don't, then you can't access it. This is very important, especially when you're using authorization like JSON web tokens or session keys. That may be, uh, a core part of your authorization system. Other than that, of course, engineering your prompts with privacy in mind can be,
- 8:31
uh, very helpful for engineering teams who are using coding harnesses like Cursor and Claude Code. In terms of protecting agent handle data, so essentially ensuring narrow agent scope to re- to decrease risk. So for example, let's say you pull tools from an MCP server that may have fifteen, twenty different tools. You don't really want to use all of them. You actually only really need one. But you also need an API
- 9:01
key to handle that, uh, API call. So if you pull that API key in from, you know, your, your password vault, and then you send it as part of the agent, um, what can happen is that it gets exfiltrated through, through the agent, uh, on a tool call to whatever, you know, web server is being hosted by a bad actor, for instance. As well as that, um, we also have a
- 9:31
classification layer in front of an agent to check for prompt injection, for example. So there's a couple of ways you can do this. So you can either use an LLM-based, uh, layer to do this, or you can alternatively use a semantic, um, search layer. For example, using vector search, uh, via something like Qdrant, LanceDB, or additionally you can
- 10:01
use an in-memory process.
- 10:05
It should also be noted that tight context loops caused by narrow agent scope can also improve context window performance. So for example, because you don't have as many tools, uh, your agents may be able to perform much better. It's well known that when you fill your context window, by doing so, when you reach near the limit of your context window, performance can actually degrade, uh, considerably.
- 10:35
By keeping an empty or near empty context window, we can make sure that the agent is performing at its best.
- 10:46
So let's look at keeping secrets out of the model. So here we have a user request that gets sent to the reasoning layer. This reasoning layer is actually then, uh, formulating a series of non-deterministic inputs for an agentic runtime. So by tying the password vault to the agentic runtime instead of the reasoning layer, what happens then is that the run-- the agent runtime is
- 11:16
essentially turned into an execution layer, which then we can, you know, we can consider it deterministic. We can, uh, do all kinds of things that you normally wouldn't be able to do for privacy and security reasons at the LLM layer. Uh, so for example, let's say you want to use a, a browse use tool, and you need to log into a, into a website that requires your credentials. With this, uh,
- 11:46
software architecture, it's much more reasonable to say that it's private and secure rather than feeding your password se-- uh, directly into the reasoning layer. As well as, you know, calling external APIs. So in terms of sandboxing your agents, um, it, it's a very well-known fact that using sandboxes and VMs, for example, Firecracker, um, and, you
- 12:16
know, AWS EC2 in a VPC, can significantly upgrade the security of your agent. However, this doesn't act as a cure-all for your agentic systems. Uh, so for example, if you have internet access enabled for your agent, it can still accidentally exfiltrate data through external API access if you're not careful, and therefore monitoring permissions
- 12:46
is essential. So my personal suggestion is
- 12:55
building your own agentic runtime. So essentially building your own coding harnesses where you can add your own steps, your own observability, uh, your own tools, things of that nature, essentially owning the execution layer and reasoning layer.
- 13:13
So let's talk about privacy at the hardware level very quickly. So in terms of this, um, there's something I like to call confidential compute, where a trusted execution environment essentially, uh, at the hardware level enables the inference process to be run privately. So with Venice, how this works is that when you send a request, uh, you get a response, and then we actually return an attestation that you can then, um,
- 13:43
verify with our attestation verifier. So in terms of this, what this actually means is that you can fully isolate an agent at the reasoning layer cryptographically and prove it. In terms of, oh, confidential tool calls, there is still some work to be done. Um, TEEs are still great for tool calls. However, we also offer end-to-end encryption. Um, but there's also, you know, some work to be done there because of,
- 14:13
uh, the need to provide an ergonomic API experience while still allowing users to return entirely encrypted tool calls. This is still kind of an ongoing challenge. Um, in terms of this, a very reliable solution would be to allow providers to own the compute layer at which your tool call's being run. However, um, ooh.
- 14:40
Sorry, just give me a sec.
- 14:51
So in terms of confidential tool calls, um, yeah, there's definitely some work to be done.
- 15:02
Um, yeah, that's it. Thank you for listening.