AI Engineer World's Fair 2026
The AI bugpocalypse is here. Now what?
Read the talk
The AI bugpocalypse is here. Build defenses that outlast the next model.
AI is expanding both software production and vulnerability discovery. Jack Cable explains how memory safety, contextual guardrails and defender access can turn that acceleration toward more resilient systems.
From a talk by Jack Cable
More vulnerabilities in the software everyone depends on
What happens when models can discover and exploit vulnerabilities faster, while simultaneously writing more of the software that needs defending? The problem begins in the open-source libraries beneath everyday applications. More discoveries in those shared dependencies mean more exposure across the systems that rely on them—the vulnerability surge Jack Cable calls the AI bugpocalypse.
Cable approaches the problem from both discovery and prevention. He co-founded Corridor to secure AI coding roughly 18 months before this talk, after working as a senior technical advisor at CISA on helping software companies build products that are secure by design. His background also includes reaching HackerOne’s top hundred hackers in high school and studying computer science at Stanford. That experience makes recurring, preventable vulnerability classes central to his response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Coding and attacks become more autonomous together
The growth of Cursor and Claude Code makes the first change visible: AI coding tools are becoming part of ordinary development. At the same time, frontier models are improving at finding and exploiting vulnerabilities. These changes compound one another. More generated software expands the attack surface, while better discovery makes that surface easier to search.
Cable cites the prior year’s Stack Overflow survey to establish the scale of adoption. Its 84% figure covers respondents using or planning to use AI tools in development, rather than current coding-agent users alone. Cable also recalls 30–40% of companies encouraging AI coding assistance, although that figure is not established by the inspected survey results. His expectation of still higher adoption in the current year is a forecast, not another survey measurement.
The more consequential shift is how developers delegate. Autocomplete leaves a person continuously selecting the next change. Synchronous work in an editor gives an agent a larger task but keeps the developer nearby. Cable describes his team moving beyond that pattern: launching multiple agents from Slack and letting them work in the background. More code can now be produced without a person following every step.
Attackers gain a similar expansion in scope. Cable says models can already outperform his own vulnerability-discovery work, despite his experience reporting hundreds of flaws. He points to an Anthropic chart comparing Mythos with other models, then widens the concern from discovery to exploitation and autonomous attack chains. A defender has to account for automation across the attack process, not just a faster search for suspicious lines of code.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A new discovery need not be a new vulnerability class
That broader attack capability brings Cable back to Secure by Design: make systems resilient to the attacks they will face, rather than depend entirely on finding each flaw first. Simple vulnerability classes were already recurring before AI. Models make their discovery easier, so defenders need to use the same capabilities to harden the underlying systems.
There is a useful distinction between finding a previously unknown flaw in a particular file and discovering a new kind of flaw. Cable’s assessment is that nearly all the vulnerabilities frontier models are finding belong to familiar classes. That creates a defensive opportunity: a control that prevents an entire class can protect against discoveries that have not happened yet. The question becomes how to apply AI to strengthen those controls as software production changes.
Cable traces this approach to a government Secure by Design paper he recalls publishing in March 2023, when coding LLMs were largely autocomplete tools. The premise does not require perfect security. It requires choosing designs that are substantially more resilient to known failure modes. On the MITRE list he presents, drawn from CISA’s Known Exploited Vulnerabilities Catalog, the leading classes are problems developers have known about—and known how to prevent at scale—for decades.
Buffer overflows appear second on the displayed list. Cable recalls documented examples going back more than 30 years and identifies them as the same established kinds of flaws that models such as Mythos now find. Rust and Go offer a different starting point from C and C++: language and runtime guarantees prevent ordinary code from performing many unsafe memory operations. Rust’s guarantee needs a boundary, however: unsafe operations leave additional correctness obligations with the programmer. Memory safety is a powerful class-level defense, not a claim that every program written in a safer language is secure.
A small Rust example illustrates the mechanism. For a three-byte packet, an attempted read at index 3 produces None rather than reading beyond the buffer:
rust
fn read_byte(packet: &[u8], index: usize) -> Option<u8> {
packet.get(index).copied()
}
fn main() {
let packet = [0x41, 0x42, 0x43];
assert_eq!(read_byte(&packet, 2), Some(0x43));
assert_eq!(read_byte(&packet, 3), None);
}
The caller still has to decide what a missing byte means for the application. But that decision happens without an out-of-bounds memory read. This is the kind of structural constraint Cable wants defenders to exploit.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Safer new code and the case for structural repair
Cable cites CVE-based estimates that memory-safe languages could prevent approximately 60–70% of vulnerabilities in products written in memory-unsafe languages. That is an estimate tied to those products’ vulnerability mix, not a universal prevention rate. He points to adoption at Google, Microsoft and Amazon, along with Rust work in parts of the Linux kernel, as evidence that this is a practical engineering direction.
Android provides the concrete example. Cable emphasizes that the improvement did not require rewriting the entire operating system: Google was writing more new code in memory-safe languages. He describes the chart approximately; Google’s published figures put memory-safety vulnerabilities at 76% of reported Android vulnerabilities in 2019 and 35% in 2022. Those figures cover critical- and high-severity vulnerabilities reported in Android security bulletins. The trend is consistent with safer new code helping, although Google cautions against assuming causation from the correlation alone.
This changes the investment question. Finding and patching individual vulnerabilities remains worthwhile. But repeated discoveries in a foundational library can also justify changing the implementation so the same class stops recurring.
| Investment | What it changes |
|---|---|
| Discover and patch a flaw | Removes a particular vulnerable behavior |
| Write new code in a safer language | Avoids adding many memory-safety flaws |
| Rewrite a critical library in Rust | Changes the memory-safety constraints on that library |
Cable contrasts spending millions on repeated discovery and patching with a one-time rewrite that can keep paying off. The attraction is durability: stronger attacking models do not erase the language’s safety guarantees. Within their supported boundaries, those guarantees continue to constrain which vulnerabilities can exist.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Intelligence does not supply the missing threat model
Safer memory access addresses one family of problems. It does not ensure that a capable model understands the security meaning of the code it writes. Cable recounts a smart-contract incident in which Opus 4.6 allegedly introduced a vulnerability followed by a couple of million dollars being stolen. He uses the account to illustrate a contextual failure: a model can be highly capable and still lack the information needed to recognize that a change violates a security requirement.
Cable reports a roughly 20–40% vulnerability rate for even the best models on the code-writing benchmark BaxBench, from researchers at ETH Zurich and UC Berkeley. Interpreting that range requires care: BaxBench evaluates backend tasks, distinguishes insecurity among functionally correct solutions and varies security prompting. The exact historical model rows and conditions behind Cable’s range are not established here, so it should not be read as the probability that arbitrary AI-generated code is vulnerable.
Two mechanisms help explain the continuing failures. First, models learn from human-written code, which already contains insecure patterns. Second, Cable’s customer observations increasingly involve contextual issues rather than obvious one-line mistakes. An authorization check may look reasonable locally while violating a company’s business rules. Knowing the programming language does not tell the model which user may access which resource, how proprietary workflows work or what the organization’s threat model assumes. Those requirements have to reach the development process somehow.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put security into the work before the pull request
As development becomes more autonomous, the question is how to make AI capable of producing Secure by Design software. The progression runs from autocomplete, through synchronous agents in Cursor or Claude Code, to agents working for an hour or several hours and producing large changes. Each step increases the amount of work waiting for review.
Cable forecasts that, within six to 12 months of the talk, most shipped code will be reviewed by AI rather than humans. This is Corridor’s expectation, driven by code review becoming a bottleneck as code generation accelerates. It is a prediction about how teams will respond to review demand, not a reported adoption result.
Corridor’s stated approach has two connected parts: prevent vulnerabilities before the pull request, and give security teams visibility into how AI coding tools are used. Cable expects pressure for faster development to prevail, so he frames the security decision around enabling agents with guardrails rather than withholding access.
Those guardrails become more consequential when an agent’s changes can be reviewed by another agent and merged with less human oversight. Security teams need enough assurance about that process to permit greater autonomy. Prevention and visibility therefore support the same goal: making faster development acceptable without requiring a person to inspect every action as it happens.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Defender access to powerful models
Cable then turns from development practices to model access. He describes export controls affecting Mythos and Fable, and says he participated in an Alex Stamos-led letter urging the White House to lift them. His position is that the defensive benefits outweigh the misuse risks. These are dual-use capabilities: the same model can help secure a system or exploit it. Cable credits Anthropic with safeguards around Fable intended to favor defenders.
His urgency comes from the narrowing gap between closed-weight and open-weight capabilities. In the distillation attacks he describes, a provider trains on outputs from a closed-weight model to improve another model. Cable argues that this shortens the interval before comparable capabilities become more widely available. Because adversaries already have powerful models, he sees rapid access for defenders as a necessary part of the response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
New code, shared foundations and independent customization
Cable closes with recommendations from his June 4, 2026 congressional testimony, which he describes as occurring a couple of weeks before the talk. The first is to prevent vulnerabilities in new code. For companies and government alike, accelerating development should include strengthening the process that keeps insecure changes out.
The second is to harden the open-source foundation. Public source gives adversaries an accessible proving ground: they can point a capable model at a library and search for previously undiscovered flaws. Cable assigns responsibility to both government and private companies to help secure these shared dependencies. He again emphasizes systemic work, including rewrites that reduce whole classes of future vulnerabilities, alongside individual discoveries and patches.
The third is to support an ecosystem of American-made frontier open-weight models. Cable presents access to weights as important to competitiveness and to companies that want to customize models independently. His fine-tuning example needs a narrower technical distinction: open weights permit direct control over customization, but fine-tuning is not exclusive to them. OpenAI’s historical GPT-4o fine-tuning announcement documents a provider-hosted closed-weight option; that offering later received a wind-down notice. The durable distinction is independent access to weights versus customization controlled by a provider.
Cable situates these recommendations amid rapidly changing model releases and export-control actions. His written testimony already discusses Mythos, so the hearing should not be understood as preceding every Mythos release. The closing engineering priority is less dependent on that chronology: use increasingly capable models to strengthen fundamental controls and make systems more resilient, including against vulnerabilities that future models will discover.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Benchmark of 392 backend implementation tasks evaluated with functional tests and security exploits, with links to code and datasets.
Research paper describing BaxBench's evaluation of backend correctness and security.
Developer survey results covering AI adoption, usage and attitudes.
June 4, 2026 written testimony on AI coding security, open-source hardening and American open-weight models; Cable's statement begins on PDF page 18.
Further reading
Official explanation of unsafe operations and the responsibilities needed to preserve Rust's safety guarantees.
Read the complete timestamped transcript
- 0:01
Hey there. I'm Jack Cable, and today I'm going to be talking about the effects of the AI bugpocalypse. As you may have seen, frontier models are getting better than ever before at discovering and exploiting vulnerabilities in our software, right?
- 0:14
This is leading to what many are calling a bugpocalypse, where we're finding more and more vulnerabilities, particularly in the open source libraries that power, um, all of the software we rely upon.
- 0:27
Right, so today I want to break down what exactly is happening and how defenders can get ahead of the exploitation that is occurring. As far as my background, right now I'm the co-founder and CEO at Corridor, a company I started about 18 months ago focused on securing AI coding.
- 0:45
Before this, I served as a senior technical advisor in government at CISA, the Cybersecurity and Infrastructure Security Agency, where I worked with top software companies to help them build their products to be more secure by design.
- 0:59
I'm also an ethical hacker. I got into the top hundred rank of hackers on HackerOne when I was in high school and studied computer science at Stanford. So I've seen firsthand how these simple, you know, repeat classes of vulnerabilities can be introduced and exploited, and have been, um, a close participant, right, in many of the, the most
- 1:19
recent, um, advancements and seeing just what this means, uh, for both our adversaries as well as defenders.
- 1:28
Just to set the stage, right, as everyone here knows, I imagine, AI coding tools are scaling faster than any software category in history. We've seen Cursor, Cloud Code, um, grow exponentially, and with that, right, also comes these improvements in how frontier models can find and exploit vulnerabilities.
- 1:48
Um, so we're seeing, right, both ends of the equation shifting. On one hand, um, models can do a better job finding vulnerabilities. On the other hand, our attack surfaces are growing immensely, right, as AI becomes the default code writer.
- 2:03
Um, so what I want to explore in this talk, right, is how do we balance that? How do we make sure that we're not going to have immensely more vulnerabilities than we've ever had before, right?
- 2:12
And just to, to give some sense, here I'll, uh, move myself here, of, um, some of the statistics, right? Um, pulled some from last year where about eighty-four percent of developers were using AI coding tools, thirty to forty percent of companies encouraging use of AI coding assistance.
- 2:29
That was from Stack Overflow, right? I don't-- I haven't seen the latest numbers this year, but what I would expect once those come out, right, is that is the vast, vast majority of developers and companies who are using coding agents, right?
- 2:42
And part of this is the increasing level of autonomy by which these coding agents are being used. It's no longer, you know, autocomplete to opt-ins, not even a developer synchronously within Cursor.
- 2:54
Um, right, when we do our own development right now, it's, um, spinning up agents from within Slack or wherever folks are working and having many agents run at once in the background.
- 3:06
Um, so this is a tremendous shift in how software is being built, um, and at the same time, right, like I mentioned, the frontier models are getting significantly better.
- 3:15
Um, and you can look at it from, you know, pretty much any part of the cyber attack chain, ranging from finding vulnerabilities where models can, you know, now do, do, you know, significantly better than even I could, and, you know, I've, I've reported hundreds of vulnerabilities to various companies.
- 3:32
Um, so everything from finding vulnerabilities to exploiting them. Uh, this is a chart here that comes from Anthropic, right, showing Mythos compared to a number of, um, other models that they and others have put out.
- 3:46
Um, and we can see that we're, we're seeing quite rapid advancements in models' capabilities, right, and, and particularly to execute, um, more kind of autonomous attack chains. So as we think about adversaries who are using these models, right, they're not just gonna be discovering vulnerabilities, but they're gonna be automating every part of the attack process.
- 4:09
So it's our job, right, as defenders to understand, okay, what are the points where we can make software systems more resilient to all of these attacks? Right, and to me, this brings back a lot of the work that I was doing in government around the, the Secure by Design initiative, right?
- 4:26
Um, and so, so the overall question, right, that I'm worried about is how can we make sure that frontier AI models aren't introducing exponentially more vulnerabilities over time, right?
- 4:37
Um, even pre-AI, we've had this, you know, uh, heavy increase in common, relatively simple classes of vulnerabilities that are being exploited by adversaries. AI is making this significantly easier, right?
- 4:52
So I think the only way that we're going to, to win as defenders is if we use the, the same techniques, right, to harden our systems. Um, and I would say that there is good news here, right?
- 5:02
That a lot of the vulnerabilities, pretty much all of the vulnerabilities that even frontier AI models are finding aren't anything new. Yes, it's new that a given vulnerability was found in a, you know, specific file with- within a piece of software, but that vulnerability class, um, isn't necessarily novel.
- 5:21
Um, and, and we can actually use that to our advantage, and I'll, I'll get into that. Um, right, so, so overall, uh, the, the thesis here is that, um, right, we are seeing both attackers get more tools in their toolkit.
- 5:35
At the same time, the way in which software is being built is fundamentally changing. Um, so, so really the question then becomes how can we apply AI to shore up, uh, these software systems?
- 5:48
Um, right, and I want to, to take a Quick detour, um, to some of the Secure by Design work that, that I, I kicked off with, with others in government.
- 5:57
Um, right, this is a paper that we put out in March of 2023, so just as, you know, LLMs were starting to become more readily available, but, um, right, their application in coding at that time wasn't m- much more than, you know, auto-complete.
- 6:13
And while that's useful, it wasn't necessarily the step change that we have now. Um, and what we focused on kind of laying out with this vision, right, was this idea that it, it isn't really rocket science when it comes to preventing vulnerabilities in software, right?
- 6:29
While it's true that it's hard to build a perfectly secure system, we do know how to build systems that are fundamentally more resilient to common classes of vulnerabilities. Um, and, and just to, you know, make this concrete, um, this is a set of vulnerability classes coming from MITRE.
- 6:48
It's the, uh, top classes that are, um, exploited in CISA's Known Exploited Vulnerabilities Catalog, right? And if you go down this list, you'll-- you might notice, right, that pretty much all of these are basic types of vulnerabilities that not only have we known about for decades, but we've known how to prevent at scale for decades, right?
- 7:10
Uh, take buffer overflows, right, number two on that list. Um, and by the way, these are the same vulnerabilities that models like Mythos are finding in software. Um, and buffer overflows were first documented, um, about, um, I believe 30-plus years ago, right?
- 7:27
Um, s-so we've had, uh, documented instances of how to find and exploit these vulnerabilities, and we also now have languages that are memory-safe, right? Languages like Rust, Go, pretty much any language, um, that, that's not C or C++, um, is built in a way such that it's impossible to introduce memory safety vulnerabilities, right?
- 7:50
They have guarantees, uh, that prevent those from being introduced. Um, so we have techniques by which we can pre-prevent them, and yet they continue getting introduced over and over again, right?
- 8:02
Um, so, so let's look at memory safety, for instance. There's, you know, the statistics range, but approximately 60 to 70% of vulnerabilities in products written in memory-unsafe languages, um, can be completely prevented using memory-safe languages, right?
- 8:20
Um, and, and this is, um, you know, based on CVE data out there. And not only that, right, we've seen a lot of companies, Google, Microsoft, Amazon, um, even, you know, open source software, um, the Linux kernel is being re-rewritten in parts in Rust.
- 8:36
Uh, we've seen real evidence that by shifting to memory-safe languages, you can reduce overall vulnerabilities, right? Um, on the right here is a chart from Google showing the rate of memory safety vulnerabilities over time in the Android operating system.
- 8:52
And what's interesting, right, is that this isn't even, you know, they're, they're not even necessarily rewriting code in a memory-safe language. They're just writing new code in a memory-safe language.
- 9:02
And even then, right, the, uh, percent of memory safety vulnerabilities has dropped quite dramatically from, um, you know, about 75% in 2019 to maybe 30% in 2022. Um, so to me, that's personally quite exciting, right?
- 9:17
Because it means that it's not a given that we're going to continue having these basic vulnerabilities over and over again, right? And, and part of the, you know, high level policy conversation, I think, as a result, has to be not just how can we deploy these frontier models to find one-off vulnerabilities in software.
- 9:35
That is something that we should be doing. But at the same time, right, I don't want to miss out on opportunities to make our software fundamentally more secure, right?
- 9:43
We could pour millions of dollars into, um, essentially playing whack-a-mole with vulnerabilities and patching them one off in some of the, the open source libraries that we all rely on, or we could do a one-time rewrite, for instance, to move some of these critical libraries, um, into, um, a language like Rust, right?
- 10:03
And then that will pay dividends for years to come. Um, s-so this is really at the core of how I'm thinking about this, right, is what are some of the fundamental changes that companies, that open source developers can be making that can reduce exploitation both by models today, right, and models to come.
- 10:22
'Cause the advantage, right, of doing a rewrite, for instance, is that if you have some of these fundamental guarantees, then even if the models get smarter, right, Rust has programmatic, uh, guarantees such that we know that memory safety vulnerabilities in most circumstances won't be possible, uh, to be introduced or discovered.
- 10:42
Um, right, and all this comes, um, in the context too of the fact that AI is increasingly capable at, of course, both writing code and then introducing vulnerabilities. Uh, you might have seen a couple months ago one example where Opus 4.6, right, by all accounts a very smart model, introduced a vulnerability in a smart contract that led
- 11:03
to a couple million dollars being stolen, right? Um, so, so while the models are very smart and capable, oftentimes security is very contextual, and the model just might not have the context in order to know that it's introducing a vulnerability, right?
- 11:19
And this is reflected in, um, academic benchmarks. Uh, one, uh, for instance here, BaxBench, um, you can find that at baxbench.com by researchers at ETH Zurich, um, UC Berkeley, uh, finds that even the best models introduce vulnerabilities about 20 to 40% of the time when writing code, right?
- 11:40
Um, and this shouldn't necessarily come as a surprise. Um, for one, right, models are trained on all of the world's existing code, and humans haven't been great at not introducing vulnerabilities in code in the past.
- 11:51
But two, writing increasingly, and this kind of lines up with some of what we're seeing among our customers, is that the vulnerabilities being introduced are often less so the basic one-liner vulnerabilities and more so contextual issues, right?
- 12:06
Things like authorization bugs that require an in-depth understanding of a company's business logic. Um, and that's something, right, even if the model is very smart, it's not being trained on your company's proprietary information or how your own, um, kind of, you know, threat model works.
- 12:24
Um, and, and that is why, you know, I believe we're still seeing quite a high rate of vulnerability introduction, um, even by, um, you know, by, by all accounts, very intelligent models.
- 12:37
Um, so, so let's now think about, okay, given that the vast majority of software development is d- being done with AI, how can we make sure that AI is capable of writing Secure by Design software, right?
- 12:48
And part of this is a shift we're seeing in the level of autonomy, um, that AI is, um, now given when it comes to software development tasks, right? We're kind of moving up this ladder that started with autocomplete to, you know, agents within Cursor, Claude Code that can synchronously produce code.
- 13:08
To now increasingly these autonomous agents that can work for an hour, hours at a time and produce quite large code changes. Of course, the next step then becomes agents that are reviewing code.
- 13:20
Um, and, and we, we at Corridor believe that, you know, within the next six to 12 months, the majority of code that is being shipped will be reviewed, uh, not by human, but by AI.
- 13:31
Um, I think that's just a kind of natural consequence of the rate at which companies need to move, given that code review is n- is now the bottleneck, and I don't think we're gonna accept that for very long.
- 13:44
Um, so really, right, our perspective at Corridor is around preventing vulnerabilities before the pull request, um, as well as giving visibility into how AI coding tools are being used.
- 13:54
Um, and I think that's really essential, right? Is that security cannot be the blocker when it comes to companies accelerating their development, right? Um, acceleration is always going to win out.
- 14:07
Um, so when we talk to security teams, the conversation is less around should you allow your, you know, development teams access to coding agents? The answer is obviously yes, right?
- 14:18
It's more around how can you do that with guardrails in place, right? Because what we're seeing is that without guardrails, yes, the coding agents can introduce vulnerabilities, um, and in order to get to a point where, you know, development can be more autonomous, that code can start to be reviewed, um, by AI and merged in without as
- 14:39
much human oversight, we really need to have tooling in place that allows, um, security teams to have that assurance, um, and to, to give the blessing to their, their engineering team to accelerate.
- 14:52
Um, I, I, I want to close with some of the policy perspective, right? And, um, this is in part tied to the recent export controls on Mythos and Fable models, right?
- 15:06
I was part of a letter, uh, led by my colleague Alex Stamos, um, where we urged the White House to lift the export controls on these models. And the perspective there is that the benefit to defenders far outweighs the risk, right?
- 15:25
These are very powerful, and let's face it, right, dual use models that can bo- both be used to secure systems and also, um, to exploit them. Uh, to Anthropic's credit, they have done a lot of work with the Fable release, right, to have some safeguards in place such that, um, they're more skewed towards defenders than, than adversaries.
- 15:47
Um, but this is also coming in the face of, right, increasingly powerful open weight models. Um, you've probably seen, you know, documented, um, distillation attacks, um, where open weight model providers can train on the output of, um, closed weight models, and that, um, as a result, is quickly shrinking the, you know, timeline between when a frontier closed
- 16:11
weight model comes out and when open weight models catch up to that. Um, so whether we like it or not, right, adversaries already have access to incredibly powerful models.
- 16:22
Um, they're already using them today to exploit systems, right? So to me, it becomes more a question of how can we rapidly, um, get the kind of capabilities in the hands of defenders, and I think that requires having these models be more widely available.
- 16:40
Um, one cool thing I had the opportunity of doing a couple weeks ago was testifying to, uh, the US Congress, um, on the, you know, risks of both, uh, frontier models as well as AI coding.
- 16:54
Um, my recommendations had a couple elements, right? One was to prevent vulnerabilities in new code going forward. I think this is something that both every company as well as the US government should be focusing on and making sure that as development accelerates, right, security isn't being left behind there.
- 17:10
Um, second is to harden the open source foundation. Um, this is incredibly important, right, especially since open source software is gonna be the kind of proving ground for a lot of adversaries, right, who want to test out these models and exploit vulnerabilities they find due to, right, the exact nature that's open source.
- 17:29
So, so you can just go and run a very smart model on them and in all likelihood find many novel vulnerabilities. Um, so, so I think both the US government but then private companies as well have a responsibility to help shore this up.
- 17:43
And like I mentioned, right, it's not just about one-off, um, you know, vulnerability discoveries or patches. It really has to be more systemic and start to get into, um, rewrites that can fundamentally reduce the risk of vulnerabilities that can be found, whether by models today or in the future.
- 18:02
Um, and then the last area of my recommendations were to foster an ecosystem of American-made open weight models, right? I think in order for us to stay competitive here, it can't just be closed weight models alone, right?
- 18:18
Um, there's a number of reasons here. You know, one of those is that for many companies, while there's a place for o- for closed weight models, you also might want to do things like fine-tuning models, um, and that is only possible with an open weight model.
- 18:33
Um, so, so I think it is really essential, right, that, um, we have frontier open weight models coming out of the United States. We haven't seen as much of that to date, um, but I think that's a critical element of American competitiveness when it comes to AI.
- 18:50
Um, so, so that-- those were my overall recommendations, right? And of course, all of this is in the context of these increasingly powerful models. The hearing was, I believe, a few days before, you know, um, Mythos and Fable came out, um, and then of course, all of the, um, export control, um, actions that were taken.
- 19:10
Um, so this is an incredibly rapidly evolving space, but I think that's why it's all the more important to go back to the basics, right? What are the fundamental controls that can protect against, um, any vulnerabilities that can be discovered by models today or in the future?
- 19:26
And I think that's where we really ought to be spending our time using these models, um, to make our systems more resilient. Um, so that's my talk. I'm happy to be reached.
- 19:38
My email is here, and thanks everyone for tuning in.