AI Engineer World's Fair 2025
How we hacked YC Spring 2025 batch’s AI agents
Read the talk
How AI agents turn ordinary security bugs into infrastructure breaches
Three attacks on YC startups trace how agent tools can expose other users’ records, undermine a code sandbox, and send private repository credentials to an attacker.
From a talk by Rene Brandel
Before you start: Basic familiarity with API requests, authentication tokens, and containers will help you follow the attack paths.
A blog built by voice, across three clouds
Ask a computer to build a blog post, then have it generate the site and upload pictures from San Francisco. Rene Brandel was building that workflow roughly a decade before this talk, using voice commands and APIs that made the demonstration visibly slow. He reports that the voice-to-code project won Europe’s largest hackathon. Brandel introduces himself as CEO of Casco, a YC company focused on red teaming AI agents and applications, after previously working on agents at AWS.
The old project worked, but assembling it required substantial plumbing before modern generative AI made this kind of interface straightforward. Two months before the talk, Brandel had left AWS to work with his co-founder in a garage; they subsequently joined Y Combinator. Looking back at his hackathon architecture, he shows three cloud providers, including IBM Watson and Microsoft LUIS. The diagram connects a React application to speech recognition and several language services: much of the engineering was making separate components cooperate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Every connection is part of the security boundary
The contemporary stack is more recognizable: a front end talks to an API server, the server talks to an LLM, and the model works through tools connected to data sources. That consistency makes applications easier to build. It also gives security reviews a concrete structure: examine each connection and the authority that crosses it.
Agent security extends beyond model behavior. Prompt injection and harmful content matter, but neither captures everything a tool can do with a database credential, a writable filesystem, or access to an internal network. The consequential failure may sit several connections downstream of the LLM. Brandel’s focus is the arrows in the architecture diagram: where requests travel, which permissions accompany them, and what those permissions allow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Thirty minutes per agent
The exercise began with a marketing objective: produce a memorable internal Y Combinator launch. Brandel says the resulting post became YC’s second-highest-upvoted internal launch, ahead of Rippling. To find material for it, the team selected agents that were already live and imposed a short testing budget.
Their sequence was straightforward:
- Set a thirty-minute timer for an agent.
- Try to extract its system prompt to understand its intended behavior.
- Inspect tool definitions, especially capabilities that access data or execute code.
- Attempt to turn those capabilities into unauthorized actions.
Brandel reports compromising seven of sixteen launched agents within a thirty-minute testing limit per agent. This was a bounded exercise on that group of applications; the talk does not specify a standardized compromise criterion. The findings organize the rest of the demonstration into three recurring issues.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A valid token does not authorize every record
The first issue was cross-user data access. Brandel introduces the examples as having been remediated, joking that investors could now consider the companies secure. In this case, an extracted system prompt exposed tools for looking up user information and documents by ID. Those interfaces suggested a familiar vulnerability: Insecure Direct Object Reference, or IDOR. The application could validate a token yet fail to check whether its holder was allowed to access the requested object.
An opaque ID does not fix that missing check. The team found a user ID in the URL bar of a publicly recorded product demonstration and supplied it to the lookup tool. Brandel reports receiving personal information, including an email address and nickname. The ID displayed during his talk was a replacement belonging to his co-founder.
The lookup also exposed relationships. A user record led to a chat ID and a document ID; those identifiers could become inputs to further tools. A single unauthorized lookup therefore provided routes through connected records. The failure was not simply that the model revealed one answer—it was that the backend accepted identifiers without enforcing the caller’s access rights.
The fix requires two distinct checks:
| Check | Question |
|---|---|
| Authentication | Is this token valid, and who is calling? |
| Authorization | May this caller access this particular object? |
Brandel points to access-control rules and Supabase row-level security as ways to enforce the second check. RLS must run under an appropriately restricted role; an administrative role that bypasses RLS defeats that protection. The important boundary is downstream, where a tool actually reads the data.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Treat the agent as a user
Why would developers who know how to build a web application omit these checks in an agent? In discussions with YC companies, Brandel encountered a recurring assumption: the agent runs on a server, so it should behave like a service and receive service-level permissions. Hosting location had become a substitute for deciding whose authority the operation should use.
The model can choose an operation; it should not decide whether the caller is authorized to perform it. Treating an agent as a user means keeping that decision in trusted application code and avoiding broad service permissions for user-directed work. A small Python example makes the separation concrete: caller_id comes from verified authentication, while the model supplies only the requested document identifier.
python
from dataclasses import dataclass
@dataclass(frozen=True)
class Document:
id: str
owner_id: str
text: str
DOCUMENTS = {
"doc-17": Document("doc-17", "user-b", "Private planning notes")
}
def read_document(*, caller_id: str, document_id: str) -> str:
document = DOCUMENTS.get(document_id)
if document is None or document.owner_id != caller_id:
raise PermissionError("Document unavailable")
return document.text
With authenticated caller user-a and requested document doc-17, this function denies access. Knowing another user’s document ID does not grant permission to read it. This owner-only example illustrates the boundary; applications with sharing would enforce their access-control rules at the same point.
The same treatment extends to tool inputs and outputs. Validate and sanitize what enters an operation, and handle its output safely at the destination. Brandel connects this to the preceding OAuth talk: the security practices developed for users and web applications still apply when an LLM chooses the next request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
When the sandbox can rewrite its own protections
The second issue involves code execution, which Brandel describes as less common but potentially more damaging. He introduces it with an Anthropic occupational-usage comparison for Claude. The matching Anthropic Economic Index report compares computer and mathematical occupations’ share of U.S. workers with their associated tasks’ share of sampled Claude.ai conversations—not programmers’ share of the world population or all Claude usage.
Code tools are useful beyond coding assistants. An agent might generate a calculator on demand to answer a numerical question. But the tool-to-container connection gives the application access to arbitrary compute, making the execution environment a critical security boundary.
The team again began with prompt extraction. The prompt itself was reconnaissance: it revealed what the developer wanted to prevent. Instructions to keep generated code hidden suggested asking to see it; instructions to execute at most once suggested trying repeated execution. The goal was to discover which restrictions were merely instructions and which were enforced by the runtime.
Initially, the environment appeared constrained. It accepted Python, blocked dangerous function calls, and restricted which Python files could run. Two remaining capabilities nevertheless mattered: it could write a Python file and read files. The team asked for filesystem-tree functionality and used the resulting listing to discover app.py.
Reading app.py exposed the service’s write-file and execute-file endpoints. Those endpoints were behind a VPC and could not be reached directly from outside, but the file also contained the code-execution security checks. The existing write capability could modify that same file. Brandel reports overwriting the checks with empty strings, removing the protections that were supposed to constrain execution. His Bitcoin-mining remark illustrates possible compute abuse, rather than a reported mining operation. The decisive mistake was allowing untrusted work to modify the code enforcing its restrictions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From code execution to BigQuery
Control of the container opened another path: discovering the surrounding infrastructure. Brandel describes using service endpoint and metadata discovery to identify network resources, then retrieving a service token, the project name, and the token’s scopes. The token’s permissions determined how far the compromise could extend beyond the execution environment.
Those permissions were too broad. Brandel reports that the team queried BigQuery and gained access to all of the affected company’s customer data. The full sequence runs from writable execution controls to arbitrary code, then through cloud metadata and service credentials to a data service. A container compromise became a customer-data breach because the surrounding identity and network boundaries allowed lateral movement.
Brandel’s recommendation is analogous to avoiding homemade authentication: use an established code sandbox instead of building the execution boundary as a small Python service. He names E2B and praises an unnamed company in his YC batch for built-in observability, quick startup, and an MCP server that makes it easy for agents to connect. These are reasons to evaluate a sandbox service, not evidence that a particular integration has been configured securely.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A repository string redirects credentials
The third issue was server-side request forgery, or SSRF: inducing a service to contact an endpoint it was not intended to contact. Here, the extracted system prompt revealed a database-creation tool that fetched a schema from a private GitHub repository. Private access implied that the fetch had credentials available to it.
The repository location was supplied as a string. The team substituted an attacker-controlled repository address and observed the request arriving there with Git credentials. Brandel reports that those credentials then enabled downloading the private codebase. The dangerous combination was control over the outbound destination plus credentials that followed the request to that destination.
Brandel says the team immediately notified the affected batch mates, who reported that they had fixed the issue. The broader implementation lesson is to inspect what each tool actually does with its inputs: a repository parameter can determine both where a server connects and where privileged information travels. Fast development, including vibe coding, does not remove the need to inspect those connections and apply established input and output validation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Testing beyond the prompt
Together, the cases make the security review larger than an LLM evaluation. User-scoped authorization must survive tool calls, and untrusted code must remain contained even when the model ignores instructions. Brandel warns that a sandbox which starts as an intern-sized project can become an infrastructure problem. He presents these vulnerabilities as basic examples, not an exhaustive inventory, and closes the prepared talk with Casco’s approach: an AI agent that attacks other agents and reports where they break, available through a demo.
The first audience question returns to the starting point of each investigation: how did the team obtain system prompts? Brandel points to publicly available techniques and recommends HiddenLayer’s work. The relevant published resource is Policy Puppetry, which addresses system-prompt leakage. He does not provide an extraction recipe in the Q&A.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Approvals and isolation solve different problems
A second audience question challenges command allowlisting: coding agents can find creative ways around permitted commands, so how can execution be controlled? The question becomes especially consequential locally, where the agent may have access to the credentials of the user running it.
Brandel distinguishes the local approval decision from server-side containment:
- Local execution: He characterizes the choices at the time as unrestricted execution or asking permission every time, referring to Cursor’s then-used YOLO terminology.
- Server execution: He recommends a code sandbox that constrains access to internal networks and limits how long the environment can live.
An approval policy governs when execution is allowed. A sandbox constrains what the resulting process can reach and how long it can persist.
The final follow-up asks whether those sandboxes use virtual machines. Brandel says they typically use Firecracker underneath and ends with a forceful warning against relying on containers for isolation. The technical distinction needs precision: containers do provide isolation mechanisms, including namespaces, but shared-kernel risks, capabilities, and mounts can weaken that boundary. Firecracker adds hardware virtualization through KVM microVMs, while still depending on correct host configuration. For untrusted generated code, placing a process in a container does not by itself establish a sufficient execution boundary.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
HiddenLayer's research on Policy Puppetry, safety bypasses and system-prompt leakage.
Current guidance for database access policies and the administrative roles that can bypass them.
Current quickstarts and documentation for E2B sandboxes, SDKs, networking and commands.
KVM-based microVM technology with architecture, setup and security guidance for isolated workloads.
Casco's security-testing service and demo request destination.
Further reading
The February 2025 analysis of Claude usage by occupational task, including the computer-and-mathematical usage and workforce comparison.
Explains container isolation mechanisms, capability restrictions and configuration risks.
Read the complete timestamped transcript
- 0:00
[upbeat music] So yeah, who's ready to hack some agents?
- 0:17
Yeah? Oh, wow. All right. So [chuckles] let me first introduce myself a little bit. I'm Rene. I'm the CEO of Casco. We're a YC company, and we specialize in red teaming AI agents and apps.
- 0:28
And so we spent, uh, I, I spent my previous time at AWS working on AI agents, but I've always really loved working on AI. In fact, there's a video of me ten years ago building voice-to-code, and I won Europe's largest hackathon by doing that.
- 0:44
And so I would talk to it, say, "Build me a blog post," and it would generate the sites. And it was actually-- It was kinda fun. Like, it did, uh, things like, um, yeah, uploading pictures from San Francisco, and you can see how horribly slow the APIs were back then.
- 0:58
And I'm gonna-- about to give you a nightmare by showing you the architecture diagram of that thing. Um, but yeah, it kinda did the job, and this was, like, ten years ago.
- 1:06
Obviously, back then was no generative AI, and these things were extremely difficult to do. Um, but it is-- it really gave me a glimpse of what the future could look like even back then as technology gets better, right?
- 1:18
So obviously, many things have changed. Two months ago, I quit AWS and worked out of, uh, the garage with my co-founder, and, uh, we got into Y Combinator. So yay!
- 1:27
That's awesome. And so from there, we also looked into how else have things evolved. Well, this was my, um, architecture diagram from back then. You could see there was three different cloud providers, including IBM Watson, which was, like, forefront at the time.
- 1:41
No, it was-- It's true. And, uh, w- before it was like, uh, Microsoft LUIS, which was like some natural language understanding things. And you can see it was just a lot of, like, piecing things together, and that was already kind of difficult to do.
- 1:54
But nowadays, we see the stacks normalize significantly more, right? I think this is probably what the average agent stack looks like these days. Got some sort of front end.
- 2:05
You talk to an API server that talks to an LLM, connects up with tools, and then you have a bunch of data sources associated to it. So this kind of normalization of agent stacks are actually really good.
- 2:15
That, like, makes many things easier. Definitely better than my hackathon project ten years ago. Um, but we need to think about the security posture around these systems. And my general impression over the last, uh, last few years is like primary discussions around LLM security, really like, hey, is it, um, can you do prompt injection?
- 2:34
Can we get it to do harmful content? Um, which is all really important. But the reality with security is you need to look at all the different arrows in your system.
- 2:45
And that is typically where real damage happens, right? And so this is really agent security, and that is what I wanna talk about today. Now, one thing is, like, why did we even hack a bunch of agents?
- 2:59
That's kind of a weird thing to do. Um, the answer is, quite frankly, you know, we wanted to launch internally at Y Combinator and wanted a splashy headline. [laughs] And so we're like, "Uh-oh, what do we do?"
- 3:10
And fun fact, we have the second highest upvoted launch post inside Y Combinator of all time. So higher than Rippling. Yes. Okay. [chuckles] Um, so, uh, we, we b- we did basically this approach.
- 3:22
At the time, we were looking at, oh, which agents are already live? And then let's just set a timer for thirty minutes. We don't wanna waste too much time on this.
- 3:29
And then, you know, let's, let's figure out what their system prompts are and just kinda understand how they're working. And I, I have a feeling when I was creating this meme that this could be true, but it turns out it is true.
- 3:40
And then we looked at, oh, what kind of tool definitions do they have, right? Like, you know, what is it supposed to do? Is it supposed to access data?
- 3:48
Supposed to run code, right? And then we just, uh, try to exploit them and see what's, what's going on. Uh, and it was really fun because we hacked, uh, out of sixteen agents that were launched, within thirty minutes each, we were hacked, uh, we hacked, we hacked seven of them.
- 4:02
And there are three common issues we see across all of these ones. So I hope that we all learn today what the most common issues are so you don't make the same mistakes.
- 4:11
And also, this is gonna be the best investment if you're a VC this batch because they're all secure now. So first issue, cross-user data access. I mean, you guys were just here at the OAuth talk.
- 4:23
You know where this is gonna head into, right? Um, [clears throat] so we first leaked this company's, uh, system prompt, and we saw, huh, it has a bunch of interesting tools attached to it, including looking up user info by ID, suspicious, uh, document by ID, and a bunch of other things.
- 4:41
And then, you know, like, when you see this, you just wanna like, oh yeah, there's this thing called IDOR, like Insecure Direct Object Reference. It's basically when you make a request and you validate that, hey, the token's valid, and then you just let the request through, right?
- 4:55
And you're kinda betting on the fact that the ID cannot be guessed. Well, that's obviously not good. Um, so yeah, we looked up a product demo video that they recorded, and we found the user ID in the URL bar and just, like, tried to plug it in.
- 5:10
Uh, this is a different ID, by the way. Don't worry, guys. This is my co-founder's ID now. And, uh, yeah, we were able to find their personal information, including their email, nickname, whatever.
- 5:19
Um, but it gets better because these things are also interconnected. So you had not only the user ID, but you also had, like, oh, the chat ID. Uh-oh. [chuckles] And their document ID.
- 5:31
And then these things ultimately linked up together and allows you to traverse the entire system, right?
- 5:38
It's not good. So [chuckles] what's the fix for that? There was a really comprehensive talk literally right before this. Sorry for the folks that missed it, but this is the basic fix for it, right?
- 5:48
You need to think about how do you authenticate but also authorize the request. It's really two checks, right? Make sure your, your token is valid. Good job, team. You got that.
- 5:56
And then the second thing is, like, this is what we see in the Superbase era with role-level security. Just make sure that you have some sort of access control matrix somewhere that checks that it matches up with whoever's making the request.
- 6:09
Okay? Super, super important. Authenticate and authorize. Now, you can see this was actually, you know, an issue that was kinda there, right? It's, it's not like around the LLM and the API server, it's really what is happening downstream.
- 6:23
And, um, yeah, there's a lot of arrows in this diagram. We're gonna look at all of them. So, the next thing is to remember, as you're thinking about these tools and how you're building it, like, agents actually act like users, um, not API servers.
- 6:38
When we were, like, debugging this issue, like we actually asked a bunch of Y Combinator companies, like, "Why, why did you build it this way?" Because clearly they can build a web app properly, right?
- 6:48
But it's just like, I think as developers, we have this natural pattern matching in our heads. It's like, "Oh yeah, this thing runs on a server, so it should be like a service, and then I'm gonna give it service-level permissions."
- 6:58
But actually, agents are like users, right? So everything that applies to users apply to agents, too. So make sure that, you know, your LLM should probably not determine your authorization pattern.
- 7:09
That, that, that's bad. That's a red flag. Uh, second thing is it should probably not act with service-level permission. Listen to your previous talk on OAuth, that's great. Um, and then just like users, you should make sure you, uh, don't just accept any input.
- 7:21
You should sanitize them. Same with outputs, right? A lot of these are like the traditional web application security things that you just need to like really, really internalize for this new world.
- 7:32
Now, that was interesting. And so the second one was even better. [laughs] Um, so this is not as common, but the damage is bigger. So it's... W- in pattern, we see, so there are a lot of code tools that agents use, and there's a, there's a, there's a Anthropic paper here.
- 7:49
It basically talks about what's the distribution of which industry and how much do they use Claude. And there's like this one outlier here. I'll zoom it in for you.
- 7:58
Um, yeah. W- so us nerds, we make up 3.4% of the world, but we're 37% of Claude's usage. Ooh, why is that? Because we love computers and we love coding, right?
- 8:08
And so we found immediately the value of it. But it's not just us that use agents with coding tools. In fact, many agents create code on demand to do some things, right?
- 8:19
Like, some agents just generate a calculator on demand to make a calculation, right? And so there's a lot of these code execution sandboxes out there that are interesting. And so if you, if you think about that, there's actually a critical path in your system because you've got a tool that talks to another container.
- 8:37
A container is arbitrary compute, and when you have arbitrary compute, many things can happen. Many bad things, many good things, right? But let's talk about the bad things today.
- 8:46
So we did the same script, did the system prompt. Again, the system prompt itself, great. I mean, doesn't cause any damage. But as an attacker, you always think about the fact, uh, the things that are like, "Huh, that's kind of suspicious," right?
- 8:58
It's like, "Oh wait, it, it, it runs code and never output it to the user. Okay, let's output it to the user. Oh yeah, and, and m- most- mostly run, run it mostly at most once.
- 9:08
Let's run it all the time." And so you try to basically invert what the system prompt is saying because that is exactly what the developer didn't want you to do.
- 9:16
And that is how bad actors think, right? So we figured out, oh, this thing does have a code tool, and so, you know, we tried, we tried running something.
- 9:24
It's like, ah, mm, it only allows me to write Python and, you know, I, I love JavaScript. And, um, yeah, and it doesn't allow me to run these really dangerous, you know, function calls.
- 9:33
Ah, okay, and it restricts like which Python files to run. That's also not good. So yeah. But we looked at what it could do, and it had two kinda innocent permissions: write a Python file and read some files.
- 9:49
You can do a lot with that. This is great because what if we just looked around the file system now, right? [laughs] We can read files. So we looked at, okay, build me a little tree functionality and, you know, return me the entire file system tree to see what's going on.
- 10:03
Oh my God, there's a app.py file. That's probably important. Um, and then we looked at, oh, it has two endpoints, write file and execute file. Ah, okay, these endpoints are hidden behind a VPC, so we cannot hit it directly.
- 10:14
That's okay. Um, but huh, we can write files. Huh, we can write files. There's a app.py file. Huh, let's look into that. Oh wait, that's where all the protections are for their code.
- 10:29
Uh, and so we can just override the app.py file with empty strings around the... all the security checks. And whoopsie, we got in. So now we can Bitcoin mine all day.
- 10:42
That's great, right? Yeah. No, it gets much worse. So the [laughs] thing with arbitrary code execution once you're inside a container is that you can do many things. Like, um, there's this thing called service endpoint discovery, metadata discovery.
- 10:56
Y'all heard of that? No? Okay. Basically allows you to discover what are other devices on the, uh, what are other devices on the network, what other resources are there on the network.
- 11:05
And, uh, you can also just, you know, fetch the user token, uh, sorry, the service token, you know, just see what's going on. What's the project name? Yeah, you know.
- 11:12
And you start looking around, it's like, oh, okay, yeah, okay, I, I, I can also fetch the scopes, so I can use, do many things with this token. That's awesome.
- 11:20
Um, who has really, really spent time configuring service-level tokens and their permissions in a granular manner and does it all the time and never forgets to set something wrong?
- 11:33
Okay, one guy, one guy there. [laughs] Okay. Whoopsie, we have access to all their customer data, so that's, uh... And we just queried BigQuery, which has a great interface for that.
- 11:41
Isn't that great? Yeah. So yeah, y- making sure you have code sandboxes correctly is very hard because you can move laterally across the infrastructure, and that is just very, very dangerous, okay?
- 11:53
And so kinda like don't roll your auth in the web world. Don't roll your own code sandboxes, please. Like, it's, it's just very hard. It's very, very hard. And so use a out-of-the-box solution.
- 12:04
There are many of them. E2B is, I think, a very popular one. Some, some folks have probably heard of it. Uh, there's one i- in our YC batch that I personally just genuinely really love.
- 12:13
They have observability built in. They boot up super quickly. And what I love about them is they have an MCP server that just is easy to plug into, right?
- 12:19
So just easier for your agents to work with. So please Do that. [laughs] Don't do, you know, your own Python app.py thing. Um, it's not good. Trust me. Um, so that leads into a third part of a attack vector around server-side request forgery.
- 12:37
This is... It's, it's a very long word, and it really bugs me that the SSRF didn't fit on the previous line. Just really triggers me. Um, yeah, I know.
- 12:45
So, um, this is what happens when you can kinda co- can kinda get a tool to call another endpoint that you didn't, and, you know, that the service itself didn't intend you to call, and you can pull out a lot of information just through that workflow.
- 13:01
So let me give you an example. So this is ex- exactly, extracted system prompt. Great. Oh, this thing can create databases. That sounds exciting. Um, and then you look into it, it's like, huh, it pulls a database schema from a private GitHub repository.
- 13:18
Isn't that great? That means whatever request goes to that private GitHub repo- repository must have the Git credentials, right? Otherwise, how can it pull that from a private repository?
- 13:28
So, um, yeah, and it's just a string, so I guess I can just put in whatever string I want and coerce it into providing that. So let's set up a badactor.com_test.gitrepo and just see what credentials come through.
- 13:41
And, yep, it comes across with the Git credentials. And so now you can actually take those Git credentials and just download their entire code base that was behind a private repo.
- 13:51
Isn't that crazy? Isn't that crazy? Yeah. This is... I mean, it's awesome for me to do this, right? [laughs] It's like you, you get paid to do th- oh, come on, it's amazing.
- 14:01
Now, um, we told our batch mates immediately, and they told us, "Don't worry, bro. It's already fixed. It's okay, guys." [laughs] That, that company's secure if you're a VC listening in.
- 14:09
Um, so, so but with that, though, it is really important to think about the implications of what your system is doing, right? I, I love vibe coding, not gonna lie, but, like, you gotta really think about where all these arrows are and if you've configured those things corre- correctly.
- 14:26
So with that, always sanitize your inputs and outputs. This could be like a web dev conference from 20 years ago. Um, [laughs]
- 14:34
but it, but it applies to agents too, right? Like, we just need to make sure we keep those good security practices that have, that we have learned to love, hopefully, [laughs] over the years to k- take it forward to a new technology paradigm.
- 14:47
And then ultimately, I want you to take away three things. So first thing is agent security is bigger than just LLM security. Make sure you understand how these threat vectors apply inside your overall system.
- 15:00
Second thing is treat agents as users, and that applies to authentication, to sanitization of user inputs, and many of the other things. And f- last thing, definitely don't roll your own code sandbox.
- 15:11
That is just so dangerous, and, you know, it, it, it very quickly turns from, like, an intern project into, like, a nightmare. So it... Be very, very careful of that.
- 15:20
And these are the most basic ones that we've seen come across, right? There's obviously many more security issues. And if you don't know exactly how your agent security posture is, you can go to Casco.com, you can book a demo with us.
- 15:33
We built an AI agent that actively attacks other AI agents and tells you where they break. Isn't that great? Um, and yeah, feel free to connect with me on LinkedIn or on Twitter.
- 15:43
And I have, uh, every now and then, some good stuff to post. Yeah. [clapping]
- 15:50
Awesome. Thanks, Rene. Does anyone have any questions? We have time for, like, one or two quick questions if you're in, if you're game for it. Sure.
- 15:58
How do you look at the system prompts?
- 16:00
Um, how do I look at system prompts? There's a lot of just, like, open techniques. The, the best one that I've seen is, uh, from hiddenlayer.com. Have you guys checked that, those guys out?
- 16:08
Yeah.
- 16:08
They have a great blog post on, like, um, the... It's a policy puppeteering attack. Yeah.
- 16:13
Mm.
- 16:14
It's great.
- 16:16
Very well.
- 16:17
Cool.
- 16:17
Awesome.
- 16:18
Oh.
- 16:18
Yeah.
- 16:20
Interesting, like, coding agents. Like, how do you make sure... Because a coding agent can recognize. Like, how do you make sure that it's actually not running, like, running the proper commands?
- 16:29
Because it's a super tough thing to do. Like, if you try whitelisting, like, there's so many creative ways that events can get around you- them. But, like, how, how would you-
- 16:38
Yeah. Are, are you talking about it locally or server side?
- 16:41
Um, they're on both. Like-
- 16:43
Yeah
- 16:43
... I mean, locally is even more dangerous because they have-
- 16:46
Yeah
- 16:46
... credentials of the user running them. [laughs]
- 16:49
Yeah, no, ve- very much so. So locally, uh, I think right now the industry is either you go full YOLO mode or you ask every time, right? Um, I mean, I'm not joking.
- 16:58
Cursor's thing is called YOLO mode, right? [laughs] Um, and then on server side, use a code sandbox, because ultimately they have constraints, uh, around the internal networks, but also they have constraints around, um, how long they can live as a sandbox.
- 17:10
Yeah.
- 17:12
Okay, so sandboxes that use VM is actually-
- 17:14
Um, yeah. So they, they typically use something called Firecracker under the hood-
- 17:18
Ah
- 17:18
... which is a better isolation layer. Yeah, if you just use containers, by the way, that's not an isolation layer, in case anybody's wondering.
- 17:23
That's why I asked.
- 17:23
Yeah. Yeah, don't use containers for isolation. Yeah. [outro music]