# Your AI Isn’t the Problem. Your Infrastructure Is. Jiachen Jiang · AI Engineer NYC 2026 · October 14 **Livestream opening excerpt only. Full talk: https://www.youtube.com/watch?v=Wb2v9Phhnns** [Watch recording](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27389s) [07:36:29](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27389s) My name is Jochen Jiang, and I'm a staff product manager at a company called Coder. [07:36:35](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27395s) We work on open source, self-hosted dev environments for agents and humans to collaborate. [07:36:41](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27401s) Many of our customers are in highly regulated industries, from federal government agencies to banks and investment management. [07:36:51](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27411s) My job is to build the AI governance that they need in order to get the productivity benefits of AI without leaving themselves wide open for security exploitation. [07:37:02](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27422s) I'm here today to tell you some scary stories. [07:37:06](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27426s) real incidents of AI usage gone wrong that happened at real companies over the past year. [07:37:13](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27433s) Some of these stories are public, while others are anonymized accounts that have happened to our own customers. [07:37:22](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27442s) For each incident, I will break it down step by step so you can understand how exactly they happened. [07:37:28](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27448s) No technical jargon necessary. [07:37:32](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27452s) Then I'll get to the most important part. [07:37:34](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27454s) I am going to tell you exactly how you can stop these scary stories from happening to you and your company. [07:37:42](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27462s) Because the good news is, everything I'm about to show you was totally preventable. [07:37:47](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27467s) Not by paying five times more per token for a smarter frontier model that is slightly better at reasoning its way out of making these mistakes. [07:37:56](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27476s) But with infrastructure controls, many of which you already own today, you just haven't gotten around to setting them up for the AI world. [07:38:06](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27486s) Let me set the scene here. [07:38:08](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27488s) You clock into work in the morning. [07:38:11](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27491s) You prompt your coding agent to take a look at your issues. [07:38:15](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27495s) Don't do anything yet, though. [07:38:17](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27497s) You tell the agent to consider this command an active code freeze. [07:38:21](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27501s) I want you to just investigate. [07:38:24](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27504s) Then you step away. [07:38:25](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27505s) You grab a coffee. [07:38:27](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27507s) And when you come back, this is the message waiting for you. [07:38:31](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27511s) So I know you said not to change anything, but, well, I destroyed a month of your work in seconds. [07:38:38](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27518s) Now what you see on the slide is a direct quote from the agent that stars in the first scary story I'm about to tell you. [07:38:46](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27526s) And I want to point something out here. [07:38:48](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27528s) This did not happen because the agent got taken over by some external malicious actor. [07:38:56](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27536s) This agent was just trying to follow instructions. [07:38:59](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27539s) It did exactly what the user gave it permission to do. [07:39:03](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27543s) So as we go through the rest of this talk, I want you to be asking yourself one question. [07:39:09](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27549s) What is stopping any of this from happening to me? [07:39:12](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27552s) Are these actions ones that my agents have permissions to take? [07:39:18](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27558s) Because the thing is, you cannot rely on anthropic or open AI to stop something like this from happening to you. [07:39:26](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27566s) They can make smarter models, but at the end of the day, [07:39:30](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27570s) large language models have two qualities that make them impossible to fully trust. [07:39:35](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27575s) They are non-deterministic, which means that there is always some randomness [07:39:39](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27579s) or chance associated with what they do. [07:39:41](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27581s) In other words, you can ask it to do the same thing 10 times, [07:39:45](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27585s) and it could do it 10 different ways. [07:39:48](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27588s) The second part is they have no structural separation between instructions and context. [07:39:54](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27594s) In other words, you can tell it to do something and give it some context to base [07:40:00](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27600s) its decisions off of. [07:40:01](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27601s) But if there's something in the context that conflicts with your instructions, the model isn't going to inherently prioritize your instructions because, from its point of view, all the information is coming through the same door. [07:40:16](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27616s) What you need to control is the infrastructure that your AI is running on, the environment that it is working inside. [07:40:23](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27623s) It is the difference between telling someone don't leave a room versus putting a lock on the door and actually using a key. [07:40:32](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27632s) The good thing about infrastructure being the problem and not the model is that you have far more control about preventing bad things from happening. [07:40:43](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27643s) So I'm not here to waste your time with vague hypotheticals or advice you can't actually act on. [07:40:50](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27650s) like, yeah, just put the genie back in the bottle, just stop using AI. Because that's not the world [07:40:56](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27656s) we live in, right? The genie is out of the bottle. And what we have to do is to mitigate risk and [07:41:03](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27663s) harm to the best of our ability. This talk is about how you do that. Not the things that you [07:41:09](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27669s) should keep an eye out for down the line, but steps that you can take right now as soon as this week [07:41:16](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27676s) to protect yourself and your company. Before I get into the stories, though, [07:41:21](https://www.youtube.com/watch?v=jiUC6IUMdvc&t=27681s) let me set up a little bit more context.