# Using LLMs to Secure Source Code

Eugene Yan, Anthropic

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/imFedndyXYQ-using-llms-secure-source-code) · [Video](https://www.youtube.com/watch?v=imFedndyXYQ)

## Using LLMs to Secure Source Code

A security agent needs more than source code: it needs system context, a safe testbed, independent verification, and a path from confirmed vulnerabilities to reviewed patches.

Basic familiarity with HTTP APIs, Python, SQL, and containers will help you follow the order-service example; no security-engineering background is required.

### What happens when finding vulnerabilities gets easier?

[0:00](https://www.youtube.com/watch?v=imFedndyXYQ&t=0s)

If a model can find vulnerabilities faster than engineers can fix them, what should a security team build next? Eugene Yan introduces this problem from his work as Anthropic technical staff, after several months helping security teams use **Claude** to find and repair vulnerabilities in code and systems. The lessons span dozens of organizations: improving discovery matters, but so does preparing the work that follows it.

**Task horizons** offer one way to understand the change in capability. Like the METR evaluations of knowledge work, the UK AI Security Institute’s [cybersecurity evaluation](https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber-capability-advancing) expresses task difficulty in terms of how long a human would take. Its tasks include identifying and exploiting weaknesses, reverse engineering, and web exploitation. The matching AISI evaluation measures 80% success with a 2.5-million-token budget per task; most human durations are expert estimates. These are narrow benchmark tasks, rather than a direct measure of success against defended production systems.

The chart shows models completing progressively longer cybersecurity tasks. Yan also points to recent results above the previous regression line: a step in capability, though that alone does not establish a new sustained growth rate. The operational question is what that additional capability produces—and who handles the output.

![Slide with an AISI cybersecurity time-horizon chart showing model results and rising trend lines.](https://ai.engineer/api/talk-essay-media/dcb75bacd511d4d8b5651d9cbbdeb8aee7d526b6f88cedfc13169db6d0afa145)

AISI CTF time horizons at 80% reliability, with a fitted frontier trend of roughly five months per doubling.

### From more findings to more unfinished work

[1:58](https://www.youtube.com/watch?v=imFedndyXYQ&t=118s)

Firefox makes the change tangible. Yan reports roughly 20 security fixes per month in 2025, roughly 60 and 70 in February and March, and roughly 400 in April—about 20 times his rounded prior-year baseline. [Mozilla’s account](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/) gives 423 security bugs fixed across April releases, including 271 **Mythos Preview** findings fixed in Firefox 150. Those are bug counts, not CVE counts; additional Mythos findings also shipped in other releases. The talk describes the April jump as sevenfold, but its rounded monthly figures do not establish that comparison precisely.

The stakes are familiar from shared infrastructure. With Log4Shell, an attacker-controlled string passed to a Java logging library could lead to code execution on the system logging it. Yan cites a Belgian Defense Ministry breach within days and an unnamed fintech incident affecting two million users. Heartbleed, in OpenSSL, supplies the earlier example: a flaw in a widely used dependency can spread risk across many otherwise unrelated systems.

Anthropic’s [initial Glasswing update](https://www.anthropic.com/research/glasswing-initial-update), dated May 22, 2026, shows the downstream workload. Yan rounds the snapshot to more than 1,000 open-source repositories, 23,000 candidate findings, 6,200 initially model-rated high or critical, 1,600 disclosures to maintainers, and about 100 upstream patches. The severity label is provisional: in the published assessed subset, 1,094 of 1,752 candidates retained high or critical severity. Candidates, verified severity, disclosures, and patches are different stages of work.

![Flow diagram tracks vulnerability findings through triage, disclosure, and remediation, beside a quotation about human capacity to deploy patches.](https://ai.engineer/api/talk-essay-media/11c55e08eb673794147e78a1578e0ba5575ee925fac9283e985c1df262e21126)

The bottleneck shifts from finding vulnerabilities to triage and patching.

**Discovery is becoming easier; verification, triage, and patching now constrain the useful output.** A scanner’s queue is not a list of repaired systems. The harness must carry findings far enough that maintainers can act on them.

### Build the setup once, then run the loop

[3:48](https://www.youtube.com/watch?v=imFedndyXYQ&t=228s)

Mozilla’s earlier experience, before its Mythos Preview work, explains why the **agentic harness** matters. Initial experiments showed promise, but false positives made them impractical to scale. A model equipped to investigate findings can distinguish real bugs from speculation that does not reproduce. The model and its surrounding tools do the work together.

Teams tend to converge on six steps. The first two are investments in a particular codebase; the remaining four form the operating loop.

| Phase | Step | Purpose |
|---|---|---|
| Setup | Threat model | Define the risks that matter |
| Setup | Sandbox | Isolate and reproduce execution |
| Loop | Discovery | Find candidate vulnerabilities |
| Loop | Verification | Establish exploitability |
| Loop | Triage | Prioritize engineering attention |
| Loop | Patching | Repair and validate |

The sandbox lets an agent deliberately execute a **proof of concept**, or PoC, to test whether a suspected exploit works. Triage then prevents a different failure: handing a developer a hundred findings when the useful assignment is the ten or twenty most consequential high and critical issues.

### Give the model the system, not just the code

[5:30](https://www.youtube.com/watch?v=imFedndyXYQ&t=330s)

The running example is a hypothetical **order service**: submit an ID, retrieve an order. Before asking an agent to find vulnerabilities, define which failures matter. Yan reports teams reaching a 90% true-positive rate with well-documented threat models; the companion guide grounds the 90% figure in one team’s experience, not a controlled comparison. He suggests above 75% as a practical target. These figures concern the usefulness of reported findings, not the fraction of all vulnerabilities discovered.

The underlying gap is between code context and system context. A model can read a handler and understand its control flow. It cannot recover an unwritten design decision, an on-call engineer’s history of fixes, or the reason a service was deployed behind a particular boundary. That knowledge has to become a document the agent can use.

A threat model can be built collaboratively:

1. **Bootstrap from existing evidence.** Give the model documentation, source code, comments, historical patches, and the CVEs those patches addressed. Ask it to identify potentially related issues that remain unpatched.
2. **Interview system experts.** Have the model ask what could happen outside the intended design and which apparent risks already have compensating controls.
3. **Record the deployment assumptions.** VPN-only access or a control elsewhere in the system may change the interpretation of a finding, even when neither appears in the repository.

The model can organize this material, but people must supply the knowledge that was never written down.

For the order service, the key asset is customer personally identifiable information, or **PII**, and the entry point is the orders API. The initial threat model identifies SQL injection and calls without authentication as possible threat vectors. This small document gives discovery a concrete target: protect sensitive order data across the API’s trust boundary.

### Make exploit testing isolated and reproducible

[7:42](https://www.youtube.com/watch?v=imFedndyXYQ&t=462s)

The sandbox has two jobs: **isolation** and **reproducibility**. An agent may run untrusted code or deliberately construct an exploit. Yan recommends a VM without egress and without cloud credentials, limiting the opportunity to exfiltrate data or damage production systems. The ability to execute code is useful only if its effects stay inside the intended test environment.

Reproducibility requires a consistent baseline. A finding may depend on an old dependency rather than the version actually deployed. Starting agents from the same baseline container makes that distinction inspectable and gives the pipeline a repeatable environment. Representative testbeds take work to build, but one team identified live-system sandboxes, where agents could execute PoCs and confirm true positives, as its biggest lever.

The order-service testbed contains three Docker images: the application, a Postgres database, and a Redis cache. The security agent sits outside the target boundary and probes the application over HTTP. That arrangement preserves the distinction between the service under test and the actor attempting to exploit it.

### Discover with context, simpler prompts, and tools

[9:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=553s)

With the setup in place, discovery depends on three ingredients. First comes **context engineering**: give the agent the documented system knowledge that improves coverage and helps rule out false positives. Second comes simpler prompting. Yan says he often cuts prompt size by roughly 50% after a model capability jump, removing increasingly unnecessary prescriptions. Instead of enumerating vulnerability classes, a stronger model can work from an instruction such as “Look for where untrusted data hits the trust boundary.”

Third, give the agent the tools a security engineer would use: API requests, response inspection, logs, and source-code access. Reading code alone cannot resolve every question about runtime behavior. Yan reports that one penetration-testing team reached an almost 100% true-positive rate after enabling tools and verification in the loop. No sample size or evaluation protocol is supplied for that team’s result.

The order-service demonstration reduces discovery to a five-line Python GET handler. Its fourth line constructs SQL using string interpolation, allowing the caller’s input to flow into the query itself. The vulnerable pattern can be expressed as:

```python
def get_order(cursor, order_id: str):
    query = f"SELECT * FROM orders WHERE id = '{order_id}'"
    cursor.execute(query)
    return cursor.fetchall()
```

The problem is the boundary between **SQL syntax** and **data**. The database receives one assembled string, with no separate indication that the supplied ID must remain a value. An attacker can alter the query’s meaning and potentially retrieve data beyond the intended order. This is deliberately ordinary code—the kind a non-security specialist might write to get a lookup working—and current models are good at spotting it.

![Python order-service code with the interpolated SQL query highlighted and a finding box explaining how unsanitized input enables SQL injection.](https://ai.engineer/api/talk-essay-media/5c1964acafd59ef140ef7d081e88d69ed1fe81ca9dce0018194f71801fd107c1)

The discovery agent flags SQL injection in the order service.

### Let a separate agent try to disprove the finding

[11:34](https://www.youtube.com/watch?v=imFedndyXYQ&t=694s)

Discovery and verification have different objectives:

| Stage | Optimize for | Question |
|---|---|---|
| Discovery | Recall | What might be vulnerable? |
| Verification | Precision | Which findings are exploitable? |

Tool-assisted investigation can improve discovery, but forcing the discovery agent to debate every candidate with itself can cause self-censorship and reduce recall. Separating the stages lets discovery search broadly without making every suspicion pass its own final evidentiary test.

The verifier should be **independent and adversarial**. It receives the candidate vulnerability without the discovery agent’s reasoning traces or work history. It starts from the assumption that the report may be false and tries to establish whether it actually works. In a representative sandbox, that means building a PoC and executing it, rather than merely producing another persuasive explanation of the same code.

For the order service, the verifier starts in a fresh container and receives the location of the suspected SQL injection: line four. It then sends a request using `curl`. In Yan’s example, the response exposes customer PII, confirming that the query can be exploited. The important output is reproducible evidence that crosses the intended data boundary, not agreement between two agents.

### A real vulnerability still needs business context

[13:26](https://www.youtube.com/watch?v=imFedndyXYQ&t=806s)

A confirmed finding is not automatically the next thing a product engineer should fix. Some findings are correctness issues; others have little business impact. Teams can lose engineers’ trust by forwarding every true finding, including a volume of medium and low severity work they cannot absorb. **Engineering attention is the scarce resource.**

Triage first removes duplicates, using inexpensive heuristics or model-assisted clustering. It then considers two separate dimensions:

- **Impact:** What happens if the exploit succeeds? Yan uses ring-zero versus ring-one consequences as an example of differing magnitude. Data sensitivity matters too: customer PII or healthcare records can make an otherwise moderate finding more consequential.
- **Likelihood:** How many barriers must an attacker cross? Deployment boundaries and compensating controls can reduce exposure, even when the underlying defect remains.

This is why the threat model remains useful after discovery. A firewall can lower the assessed severity; sensitive database contents can raise it.

In the order-service example, the agent initially assigns high impact and high likelihood. Human review keeps the impact assessment but lowers likelihood: an application firewall blocks SQL injection, and the service is used only for internal warehousing or logistics, never exposed to the internet. The review changes the risk assessment; it does not patch the vulnerable query. Those deployment facts may live in a system design, documentation, or Confluence rather than the source code. Writing them into the context lets the next agent make a better-informed assessment.

The same finding after deployment context is added
New context changes exploit likelihood without removing the underlying vulnerability.
Constructed example: The comparison represents the absence of controls from the initial assessment as null; it does not imply the controls were newly deployed.
Finding under review — unchanged: SQL injection in the order-service orders API
Operation: Human review adds deployment controls to the agent's risk assessment; no code patch occurs in this step.
- Finding
  - Before: Agent assessment: SQL injection in the orders API
  - After: After human review: SQL injection in the orders API
- Impact
  - Before: Agent assessment: High
  - After: After human review: High
- Likelihood
  - Before: Agent assessment: High
  - After: After human review: Low
- Application firewall
  - Before: Agent assessment: Not present
  - After: After human review: Blocks SQL injection
- Deployment context
  - Before: Agent assessment: Not present
  - After: After human review: Internal warehousing/logistics; not internet-facing
New context changes exploit likelihood without removing the underlying vulnerability.

### Validate the patch and preserve what you learned

[15:52](https://www.youtube.com/watch?v=imFedndyXYQ&t=952s)

Patch validation follows a ladder:

1. **Replay the original PoC.** The exploit must stop working.
2. **Run the existing tests.** The repair must preserve legitimate behavior.
3. **Attack the patched code again.** A fresh discovery agent looks for incomplete fixes or another route to the same failure.
4. **Require human confirmation before merge.** The agent proposes and revises; a person approves the repair.

The repeated generation and verification give the patching agent concrete feedback. A failed exploit alone is insufficient if the patch also breaks the service, and passing existing tests alone is insufficient if a nearby exploit still works.

The order-service fix separates the parameter from SQL construction. Using a Postgres driver with `%s` parameter binding, the earlier Python example becomes:

```python
def get_order(cursor, order_id: str):
    cursor.execute(
        "SELECT * FROM orders WHERE id = %s",
        (order_id,),
    )
    return cursor.fetchall()
```

The query structure stays fixed while the driver handles `order_id` as a value. This expresses the mechanism of the one-line repair Yan describes: remove the user-supplied variable from Python’s SQL string interpolation.

The second diff updates the system’s knowledge, not just its implementation. It records the application firewall and internal-only deployment so later scans do not repeat the same mistaken assumptions. Yan frames this as turning repeated operational expense into an accumulating investment: each pass should leave behind better code, better context, or both.

### Scale the decisions around the harness

[17:14](https://www.youtube.com/watch?v=imFedndyXYQ&t=1034s)

The technical workflow resembles an ML pipeline: context and reproducibility provide the foundation, followed by recall, precision, ranking, and feedback. But increasing its input by one or two orders of magnitude exposes constraints that additional compute cannot resolve. More scanning can be purchased; more human attention is harder to obtain. Product and security engineers need written agreement on high and critical severity, and experts need to externalize the threat models held in their heads. Yan says he knows few companies using fully automated security patch review.

Three operating changes help the organization absorb the output:

- **Route by ownership.** Curated emails and manually assigned Jira tickets may work for a dozen findings a month, but not hundreds. Route directly to the code owner or service owner; this step does not need an LLM.
- **Calibrate severity together.** Product engineers, security engineers, red teams, and blue teams need a shared rubric grounded in the business. Discuss disagreements, document the rules, and give those rules to the agents.
- **Supply patches, not just reports.** Even with coding assistance, a bare vulnerability leaves substantial work for the recipient. Move toward AI-generated patches, attack them again, and retain human verification.

These changes connect a technically valid finding to an owner who understands its priority and can review a proposed repair.

### Start interactively before automating

[20:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=1205s)

Begin with open-source dependencies or another scope you understand and feel comfortable investigating. Work interactively in **Claude Code** or your preferred IDE before trying to automate the whole process. That hands-on pass reveals which context is missing, where precision falls short, and what the harness needs to make its output useful. Budget for verification, triage, patching, and the organizational work around them—not only the scan.

Yan points to [Claude Security](https://claude.com/blog/claude-security-public-beta) for scanning codebases, validating findings, and suggesting patches, alongside a [companion guide](https://claude.com/blog/using-llms-to-secure-source-code) and open-source materials with interactive skills and autonomous harnesses. The [Defending Code Reference Harness](https://github.com/anthropics/defending-code-reference-harness) provides a concrete starting point. The talk’s quick-start walkthrough places a customizable harness at step five; the current repository organizes customization under Step 3 and detection and response under Step 5. Follow its current instructions, then adapt the harness using what the interactive work taught you about your own system.

![“Start now. Here's how” slide with three adoption recommendations and four resources covering Claude Security, a blog, a reference harness, and code review.](https://ai.engineer/api/talk-essay-media/9e89dab1ccd7a778357233263e6eedf6ac73a2f2b2007b468146e6f2ce086cbc)

Start small, learn interactively, and budget for verification, triage, and patching.

## Resources

- [Using LLMs to secure source code](https://claude.com/blog/using-llms-to-secure-source-code) — Eugene Yan and Henna Dattani explain threat modeling, sandboxing, discovery, verification, triage and patching.
- [Defending Code Reference Harness](https://github.com/anthropics/defending-code-reference-harness) — Interactive security skills and an autonomous reference pipeline with sandboxing, independent verification and patch validation.
- [Behind the Scenes Hardening Firefox with Claude Mythos Preview](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/) — Mozilla details its vulnerability discovery pipeline, April 2026 fixes and the distinction between security bugs and rollup CVEs.
- [How fast is autonomous AI cyber capability advancing?](https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber-capability-advancing) — AISI explains cybersecurity task horizons, token budgets, human baselines and uncertainty in capability trends.
- [Project Glasswing: An initial update](https://www.anthropic.com/research/glasswing-initial-update) — Anthropic's May 2026 snapshot separates candidate findings, assessed vulnerabilities, disclosures and patches.
- [Claude Security is now in public beta](https://claude.com/blog/claude-security-public-beta) — The historical launch announcement describes code scanning, finding validation and proposed fixes for Enterprise customers.

## Slides

### Working with Models to Secure Source Code

Eugene Yan shares lessons from working with security teams and Claude to find and fix vulnerabilities in code and systems.

[0:19](https://www.youtube.com/watch?v=imFedndyXYQ&t=19s) [Title slide for Eugene Yan’s Anthropic talk at AI Engineer World’s Fair 2026.](https://assets.ai.engineer/sites/aiecode2025/sha256/a29af47d846392ce1acca0d30c3cc28b871eef8667edade0b98f1c1603f636f0.webp)

Working with Models
to Secure Source Code

Eugene Yan · Anthropic
AI Engineer World's Fair · 2026

ANTHROPIC

### What we’ll cover

Eugene Yan outlines trends in model capability, vulnerabilities found, and shifting bottlenecks, then shares lessons from dozens of organizations and resources to get started this week.

[0:43](https://www.youtube.com/watch?v=imFedndyXYQ&t=43s) [Agenda covering security trends, working with models, potential bottlenecks, and getting started.](https://assets.ai.engineer/sites/aiecode2025/sha256/56afdd65843156b863863f3d9e85b7a47f9728be17081f2c9eed2d00db89252d.webp)

What we’ll cover

• Why now: the trends we’re seeing
• How to work with models for security, and potential bottlenecks
• How to get started now + resources to help

ANTHROPIC

### Why now

This section examines three trends: model capability, vulnerabilities found, and the shifting bottleneck.

[1:07](https://www.youtube.com/watch?v=imFedndyXYQ&t=67s) [Section introduction naming model capability, vulnerabilities found, and the shifting bottleneck as three trends.](https://assets.ai.engineer/sites/aiecode2025/sha256/869d88ef601b95aef70c97a017554afdc39045c7287dd1d0c458b6925cf91196.webp)

Section I

Why now

Three trends: Model capability, Vulns found, Shifting bottleneck

ANTHROPIC

### Model capability on cyber tasks

At 80% reliability and a 2.5M-token cap, AISI’s cyber CTF time horizons rise; the pre-Mythos trend doubles about every 4.7 months. Mythos Preview and GPT-5.5 saturate the suite, making their estimates highly uncertain.

[1:11](https://www.youtube.com/watch?v=imFedndyXYQ&t=71s) [AISI chart shows increasing cybersecurity task time horizons and a roughly five-month doubling trend, with caveats about the benchmark.](https://assets.ai.engineer/sites/aiecode2025/sha256/93faa68ee9c08a71affeaf23f2dda1c72abebed4ff0e8953bbcf11976190ad60.webp)

Model capability on cyber tasks is doubling every ~5 months

AISI Cybersecurity Time Horizons
AISI CTF suite, 80% reliability, 2.5M token cap, reasoning models only

Previous AISI estimate (Nov 2025): ~8mo doubling
Frontier post-reasoning trend (pre-Mythos): ~4.7mo doubling

Caveats:
• Models evaluated under a 2.5M token cap; uncapped time horizons would be significantly higher
• Narrow cyber tasks only; no claim about generalization to other capability domains
• Mythos Preview (new) and GPT-5.5 saturate the task suite, resulting in highly uncertain time horizons

Y-axis: 80% Reliability Time Horizon (log scale)
Y-axis ticks: 6m, 1.0h, 10.0h, 4d
X-axis: Model Release Date
Timeline: 2025-01, 2025-04, 2025-07, 2025-10, 2026-01, 2026-04, 2026-07, 2026-10

Models: o1, Claude 3.7 Sonnet, Claude 4 Opus, GPT-5, Claude Sonnet 4.5, Claude Opus 4.5, Claude Opus 4.6*, GPT-5.3-Codex*, Mythos Preview (new), GPT-5.5
Provider: Anthropic, OpenAI
AISI 8mo estimate computed (Nov 2025)

* GPT-5.3-Codex and Claude Opus 4.6 share a release date (2026-02-05); offset here for clarity.

AISI | AI SECURITY INSTITUTE
ANTHROPIC
UK AISI — How fast is autonomous AI cyber capability advancing?

### Firefox security bug fixes by month

Firefox security bug fixes rose from roughly 20 a month in 2025 to 423 in April 2026. The speaker attributed about 271 of April’s fixes to Mythos Preview.

[1:57](https://www.youtube.com/watch?v=imFedndyXYQ&t=117s) [Bar chart shows Firefox security bug fixes rising from roughly 20 per month in 2025 to 423 in April 2026.](https://assets.ai.engineer/sites/aiecode2025/sha256/40605e105c91dda1e2c6c5eb9813537eb402037f0b2f41bd059e671f2c3d1103.webp)

Mozilla found more vulns in April 2026 than all of 2025

Firefox Security Bug Fixes by Month
All Sources • All Severities

Monthly values: JAN 2025: 21; FEB 2025: 20; MAR 2025: 26; APR 2025: 31; MAY 2025: 17; JUN 2025: 21; JUL 2025: 22; AUG 2025: 17; SEP 2025: 18; OCT 2025: 26; NOV 2025: 19; DEC 2025: 20; JAN 2026: 25; FEB 2026: 61; MAR 2026: 76; APR 2026: 423

ANTHROPIC
Mozilla Hacks — Behind the Scenes: Hardening Firefox

### Notable public vulnerabilities and exploits

Log4Shell let attackers run code through a logged string; Heartbleed let them read server memory, including passwords and private keys. Both affected widely used software.

[2:36](https://www.youtube.com/watch?v=imFedndyXYQ&t=156s) [Two panels describe Log4Shell and Heartbleed, their mechanisms, and their reported impact.](https://assets.ai.engineer/sites/aiecode2025/sha256/c6106b1a0ba9eeb28dc62671e02496d0b6375a8ccddb143d41211860788f0c89.webp)

What are some notable public vulns and exploits?

Log4Shell
CVE-2021-44228 · Dec 2021

A bug in a Java logging library let attackers run code on your server just by getting it to log a string. Minecraft, iCloud, Steam, AWS — all affected.

93% of enterprise cloud environments were vulnerable. Belgium’s Defence Ministry was breached within days; one fintech platform leaked 2M users’ data.

Heartbleed
CVE-2014-0160 · Apr 2014

A bug in OpenSSL — the encryption library behind most of the internet’s HTTPS traffic — let anyone read server memory, including passwords and private keys.

Half a million HTTPS servers vulnerable; 4.5M patient records stolen in one breach. Sat undetected in open-source code for two years.

ANTHROPIC

Log4Shell · Heartbleed — Wikipedia

### Six simple steps

Section II introduces six steps for building an agentic security harness for your own systems and codebases.

[3:46](https://www.youtube.com/watch?v=imFedndyXYQ&t=226s) [Section II introduces six steps for building an agentic security harness.](https://assets.ai.engineer/sites/aiecode2025/sha256/0725c9169fb97b7eb4b420840eba23ec7650e0e6bc9fbad91512c1559e6b79d3.webp)

Section II

Six simple steps

To building your own agentic harness for security

ANTHROPIC

### Meet order-service

A hypothetical order service looks up an order by ID. The example shows a delivered order with options to track the package, view the invoice, start a return, or report an issue.

[5:28](https://www.youtube.com/watch?v=imFedndyXYQ&t=328s) [A hypothetical order lookup interface displays an example delivered order and several action buttons.](https://assets.ai.engineer/sites/aiecode2025/sha256/f43d96b1819a01993221f9ddf9244e85faa1696f8be51e686b427098489e984e.webp)

RUNNING EXAMPLE
Meet order-service

Find your order
ORD-2412
Look up

Order #ORD-2412  Delivered
[redacted] · 2 items · $148.00
[redacted] · Delivered Tue, Jun 23

Track package
View invoice
Start a return
Report an issue

ANTHROPIC

Interface text:
shop.example.com/orders?id=ORD-2412

### Sandbox — target stack and security agent

The order service connects to Postgres and Redis inside the target boundary. A security agent outside that boundary probes the app over HTTP; the diagram specifies no agent egress.

[8:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=531s) [A security agent probes an order service connected to Postgres and Redis inside a target boundary.](https://assets.ai.engineer/sites/aiecode2025/sha256/c6a9a6980894aecebce279365572b23a167f6747c7d92e9f7f30ed39ff80949f.webp)

EXAMPLE · order-service

Sandbox — the target stack, plus a security agent probing it

target · network_mode: none

order-service
flask@a3f9c1...

postgres
16.2@sha256:...
seeded fake PII

redis
7.2@sha256:...

HTTP
←
probe

security agent
claude + tools
egress: none

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching

ANTHROPIC

### One vulnerability the discovery agent found

The order-service endpoint inserts user input directly into a SQL query. The discovery agent flagged the resulting SQL injection, which could expose tables accessible to the database role.

[11:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=665s) [The order-service code highlights an interpolated SQL query and identifies a SQL injection finding.](https://assets.ai.engineer/sites/aiecode2025/sha256/f5b1994641b79c00905e8d8da9f9d4bd942d14d902aacdfdca2409daea2e0257.webp)

EXAMPLE · order-service
One vulnerability the discovery agent found

app.py
@app.get("/orders")
def get_order():
    order_id = request.args["id"]
    row = db.execute(f"SELECT * FROM orders WHERE id = '{order_id}'")
    return jsonify(row)

FINDING · app.py:14 · CWE-89
order_id flows unsanitized into an f-string SQL query → SQL injection. A crafted id like ' UNION SELECT email,pw_hash FROM users-- can read any table the DB role can access.

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching

ANTHROPIC

### Triage findings by root cause and severity

Triage combines heuristic and model-based deduplication with impact, likelihood, and threat-model context. Prioritizing critical and high findings helps preserve engineers’ attention.

[13:25](https://www.youtube.com/watch?v=imFedndyXYQ&t=805s) [Triage guidance covers deduplication, severity assessment, and threat-model context.](https://assets.ai.engineer/sites/aiecode2025/sha256/f21cfccce09157c8cdfc89d9b3bc50fbd520b35e70865da114c1634416fb1bb5.webp)

Triage: Dedupe by root cause, rank by preconditions

Why

Multiple teams shared the same lesson: if you send product engineers a pile of findings where a majority are non-exploitable, they lose trust in the reports and give up. So they prioritize critical and high findings to avoid overwhelming the engineers downstream.

What and how

• Deduplicate findings
  – Cheap pass: same file, same category, within ~x lines likely means the same bug
  – Model pass: same root cause? same sink? would one patch fix both?
• Assess severity — one way is by considering potential impact × reachability/likelihood
• Provide the threat model during triage
  – Context on compensating controls can calibrate highs down to lows
  – Business rules about PII or health data can calibrate lows up to highs

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching

ANTHROPIC

### Compensating controls change triage

Human review keeps impact high but lowers likelihood and severity: a WAF blocks SQLi patterns, and the service is internal-VPN only.

[14:59](https://www.youtube.com/watch?v=imFedndyXYQ&t=899s) [Human review downgrades an illustrated finding from high to low severity after considering firewall and network controls.](https://assets.ai.engineer/sites/aiecode2025/sha256/6b30ec28e94d4f4d87c52d332e59599211181a0a0f42540c34480c9bfa75d04b.webp)

EXAMPLE · order-service
Triage — compensating controls downgrade High to Low

AGENT TRIAGE
Severity
HIGH
Impact
High — dumps customer PII (email, phone, address)
Likelihood
High — unauth, internet-reachable
Controls
(none in threat model)

AFTER HUMAN REVIEW
Severity
LOW
Impact
High (unchanged)
Likelihood
Low (downgraded)
Controls
• Web Application Firewall (WAF) blocks SQLi patterns upstream
• Service is internal-VPN only

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching
ANTHROPIC

### Patch and validate

Validate a patch by rebuilding, rerunning the original PoC and regression suite, then having a fresh discovery agent attack it. Feedback helps the agent iterate; a human confirms the patch and owns the merge.

[15:52](https://www.youtube.com/watch?v=imFedndyXYQ&t=952s) [Patching guidance covers building, retesting the proof of concept, regression testing, re-attack, and human review.](https://assets.ai.engineer/sites/aiecode2025/sha256/d6d7911fa283eeb43bfb1766283d3c7491126d5bc51d756fab9831a872f855db.webp)

Patching: Fix, validate, hunt variants, close the loop

Why
One pentester found their generated patches were inconsistent — until the harness told the model to validate patches by re-running the proof of concept against the patched code. Given feedback to iterate against, patch quality jumped, saving time on human review.

What and how
• Build — it compiles, and the new test passes
• Reproduce — the original PoC stops working
• Regress — the existing suite stays green
• Re-attack — a fresh discovery agent probes the patch
• Human in the loop — a person confirms the patch and owns the merge

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching

ANTHROPIC

### Patch the code and update the threat model

The patch passes order_id separately to the database. The updated threat model records a WAF on public routes and VPN-only service access for future scans.

[16:30](https://www.youtube.com/watch?v=imFedndyXYQ&t=990s) [Code and threat-model diffs show a parameterized query and newly documented security controls.](https://assets.ai.engineer/sites/aiecode2025/sha256/0aea5aa34cd4853cf72741f6061ebc4723f4e104202c955acc9b2a641758a92a.webp)

Patch the code — then close the loop on the threat model

EXAMPLE · order-service

app.py
- row = db.execute(f"SELECT * FROM orders WHERE id = '{order_id}'")
+ row = db.execute("SELECT * FROM orders WHERE id = %s", (order_id,))

No more building SQL with Python string interpolation — the value is passed to the database separately, as data, not SQL.

THREAT_MODEL.md → v2
- Controls: (none identified from code)
+ Controls:  • WAF (managed SQLi ruleset) fronts all public routes
+            • Service is internal-VPN only; not internet-reachable

1 · Threat model → 2 · Sandbox → 3 · Discovery → 4 · Verification → 5 · Triage → 6 · Patching

ANTHROPIC

### What each of the six steps focus on

The threat model and sandbox establish context and reproducibility. Discovery, verification and triage focus on recall, precision and ranking; patching feeds lessons back into the threat model.

[17:07](https://www.youtube.com/watch?v=imFedndyXYQ&t=1027s) [Six-step security workflow from threat modeling through patching, with feedback into the threat model.](https://assets.ai.engineer/sites/aiecode2025/sha256/2ca6d58419cb33bdab7c98956c92b30fe1f78e7d1848e61f6322b76a8518c644.webp)

What each of the six steps focus on

1. Threat model
Context. Provide context not available in the code, to raise the rate of true exploits

2. Sandbox
Isolation and reproducibility. Give the model a real environment to cook in

3. Discovery
Recall. Cast a wide net and identify as many potential vulnerabilities as possible

4. Verification
Precision. Independently confirm which vulns are exploitable and drop false positives

5. Triage
Rank. Prioritize by potential impact × likelihood to avoid overwhelming the org and devs

6. Patching
Fix and feedback. Fix the root cause, then re-run the PoC to validate. Update the harness with what you've learned

Feed back into the threat model so the next iteration is better informed

ANTHROPIC

### Yet another bottleneck—org processes

Section III turns from the technical pipeline to organizational processes as a non-technical bottleneck.

[17:41](https://www.youtube.com/watch?v=imFedndyXYQ&t=1061s) [Section III introduces organizational processes as a bottleneck.](https://assets.ai.engineer/sites/aiecode2025/sha256/adfc188cfa9329888781474bfde31bb08ce606e593d2576158ad2d02f94040c0.webp)

Section III

Yet another bottleneck—org processes

Non-technical problems are much harder than technical problems

ANTHROPIC

### What happens when you find one to two OOM more vulns?

The scanning harness can find more vulnerabilities with more engineering, compute and time. Human attention becomes a bottleneck: teams still need to agree on severity, document threat models and review patches.

[17:55](https://www.youtube.com/watch?v=imFedndyXYQ&t=1075s) [The scanning harness scales, while human attention and organizational processes can become bottlenecks.](https://assets.ai.engineer/sites/aiecode2025/sha256/be1ffc31aa76591b5f398c2b453de5c7d5a2c726cfcb73f56b263e1992640f76.webp)

What happens when you find one to two OOM more vulns?

• The harness scales — more engineering, more compute, more time → more vulns found
• Human attention doesn’t — threat models from implicit context, triage and severity calibration, patch review
• Another bottleneck you might hit is your org’s process

ANTHROPIC

### Organizational bottlenecks you might hit, and what to do

As findings grow, automate routing by service, code owner, and component; provide a shared threat model for severity calibration; and use AI-generated patches with human approval.

[18:53](https://www.youtube.com/watch?v=imFedndyXYQ&t=1133s) [Recommendations for vulnerability routing, severity calibration, and patching bandwidth.](https://assets.ai.engineer/sites/aiecode2025/sha256/08684a838a124ac679c9f9bb17bb4683b0432b38bc3ab45a17f840dc0eb4391d.webp)

Organizational bottlenecks you might hit, and what to do

• Vuln routing
  – Don't: manually assign findings to teams — it won't scale to hundreds/thousands of vulns
  – Do: automate routing by service, code owner, and component

• Severity calibration
  – Don't: rely solely on calibration meetings to discuss and align on severity
  – Do: provide your threat model to help the model calibrate better on impact × likelihood

• Patching bandwidth
  – Don't: rely on devs writing every fix from scratch — can't clear backlog this way
  – Do: move to AI-generated patches with human-in-the-loop approval

ANTHROPIC

### If you only remember one thing...

The closing section introduces three takeaways. The speaker recommends starting with open source dependencies and learning interactively before aiming for automation.

[20:01](https://www.youtube.com/watch?v=imFedndyXYQ&t=1201s) [Section IV introduces three closing takeaways.](https://assets.ai.engineer/sites/aiecode2025/sha256/27df5dc4164dda62d276983bd91cfaa4c40cc2805f49a1d202c7e02412991874.webp)

Section IV

If you only remember one thing...

Okay, maybe more like three things

ANTHROPIC

## Full transcript

[0:00](https://www.youtube.com/watch?v=imFedndyXYQ&t=0s) [upbeat music] Hi, I'm Eugene, a member of technical staff at Anthropic.

[0:15](https://www.youtube.com/watch?v=imFedndyXYQ&t=15s) Um, I've spent the last several months with working with security teams and Claude to find and fix vulnerabilities in codes and systems. So I want to share with you what we've learned as much as we can.

[0:24](https://www.youtube.com/watch?v=imFedndyXYQ&t=24s) Before I get started, quick show of hands, how many of you here, here are security engineers in your day job?

[0:31](https://www.youtube.com/watch?v=imFedndyXYQ&t=31s) Okay. Okay. How many of you here are engineers in your day job? Oh, sweet. Okay, great. I have a better understanding of how to give this talk now. Um, security engineers in the room, please don't boo me if I oversimplify things.

[0:42](https://www.youtube.com/watch?v=imFedndyXYQ&t=42s) I wanna get this message out to as many people as I can. So first, I wanna share with you three high-level trends that we are seeing, and those are model capabilities, right?

[0:50](https://www.youtube.com/watch?v=imFedndyXYQ&t=50s) Number of vulnerabilities found, um, and where the bottleneck is shifting to. Then I want to distill the lessons we've learned from working with dozens of, uh, organizations to improve their security posture, as well as the bottleneck that you might encounter and how you can try to address it and prep for it.

[1:04](https://www.youtube.com/watch?v=imFedndyXYQ&t=64s) And finally, I wanna share how you can get started this week with links to resources.

[1:09](https://www.youtube.com/watch?v=imFedndyXYQ&t=69s) So why do we care about so much about cybersecurity now? Um, time horizon benchmarks track the length of time an AI model can complete a task measured against an actual human doing the task.

[1:22](https://www.youtube.com/watch?v=imFedndyXYQ&t=82s) Um, so we all know about the METR eval task, you know, how, how much knowledge work one agent can do. The UK AI Security Institute has a cybersecurity version of this.

[1:32](https://www.youtube.com/watch?v=imFedndyXYQ&t=92s) You know, these tasks require identifying, exploiting security weaknesses in target systems, and, you know, testing skills like reverse engineering and web exploitation. So in this chart, we see that models are able to increasingly do longer cybersecurity task.

[1:45](https://www.youtube.com/watch?v=imFedndyXYQ&t=105s) Um, but what's also interesting is that you kind of see a step jump against the previous regression line. So these models are a step improvement in capability. And what does this step improvement in capability buy us?

[1:58](https://www.youtube.com/watch?v=imFedndyXYQ&t=118s) Recently, Mozilla Firefox published a number of security bug fixes they made each month. The average in twenty twenty-five was about twenty, twenty-ish. And then you can see in February and March, it kind of three X'd to about sixty and seventy.

[2:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=133s) And then in April, it seven X'd to four hundred. So what this means is that what's happened in April is twenty X of last year's average.

[2:24](https://www.youtube.com/watch?v=imFedndyXYQ&t=144s) Um, they attributed about two-thirds of this to Mythos Preview, about two hundred and seventy-one, which shows that frontier models can help defenders like yourself find and fix vulnerabilities at scale.

[2:37](https://www.youtube.com/watch?v=imFedndyXYQ&t=157s) We also-- What are some notable vulnerabilities? Hands up, who here remembers Log4Shell? Anyone? Anyone was here over the December holidays, like scrambling to patch this stuff, too? So Log4Shell, um, is a bug in a Java logging library.

[2:52](https://www.youtube.com/watch?v=imFedndyXYQ&t=172s) If an attacker sends a string, you log it, the attacker can run code on your system. The Belgium Defense Ministry was breached within days, and a fintech platform w- uh, leaked two million users data.

[3:03](https://www.youtube.com/watch?v=imFedndyXYQ&t=183s) And then before that, we had Heartbleed, which is an, a bug in OpenSSL. You know, so much of the li- of the Internet runs on OpenSSL, and we, we all know what impact it had.

[3:14](https://www.youtube.com/watch?v=imFedndyXYQ&t=194s) This is Anthropic sharing of our own work, scanning more than a thousand open source repos. Um, from twenty-three thousand candidates, uh, sixty-two hundred of them were rated as high or critical.

[3:25](https://www.youtube.com/watch?v=imFedndyXYQ&t=205s) And at the time of the update, sixteen hundred of them were reported to maintainers and about a hundred patched upstream. We shared our observation that finding vulnerabilities now is quite straightforward.

[3:39](https://www.youtube.com/watch?v=imFedndyXYQ&t=219s) The bottleneck has now shifted to verification, triage, and patching, and I wanna share with you everything we've learned about that so we can get ahead of it.

[3:48](https://www.youtube.com/watch?v=imFedndyXYQ&t=228s) So how do we do this in our own systems and code bases? Um, two words: agentic harnesses. I'm citing Mozilla again, and this was even before they worked with Mythos Preview.

[3:58](https://www.youtube.com/watch?v=imFedndyXYQ&t=238s) I'm gonna read it off the screen here. Early experiments showed that some promise, but the high rates of false positives made it impractical to scale. But the introduction of agentic harnesses that can reliably detect security issues has changed this, right?

[4:15](https://www.youtube.com/watch?v=imFedndyXYQ&t=255s) These models and harnesses together can find real bugs and dismiss non-reproducible speculation.

[4:23](https://www.youtube.com/watch?v=imFedndyXYQ&t=263s) So the harness and the model working together can get a lot of work done.

[4:29](https://www.youtube.com/watch?v=imFedndyXYQ&t=269s) We found that most teams converge in approximately these six steps, and a big chunk of my talk is gonna be about these six steps. Right, the first two steps are set up, right?

[4:38](https://www.youtube.com/watch?v=imFedndyXYQ&t=278s) You do a bit of upfront investment per code base. How many of you here know what a threat model is?

[4:44](https://www.youtube.com/watch?v=imFedndyXYQ&t=284s) Amazing. I love that. A threat model is some kind of like, kind of system design, but for your code base or system, what are threat vectors? So this helps you decide what are the vulnerabilities in your system that you care about.

[4:56](https://www.youtube.com/watch?v=imFedndyXYQ&t=296s) And then also, we also want an isolated sandbox where you let your model run possibly untrusted code, or you let your model deliberately detonate proof of concepts of exploits, I'm gonna call them PoCs, to confirm that they are true positives.

[5:10](https://www.youtube.com/watch?v=imFedndyXYQ&t=310s) And then the next four are loop, where, you know, we have discovery that identifies vulnerabilities, verification confirms that they are real. We have triage. You know, you can't give a developer a hundred vulnerabilities.

[5:21](https://www.youtube.com/watch?v=imFedndyXYQ&t=321s) You gotta identify the top ten or twenty critical and high, so they can focus their attention. And then you can patch, patch the bugs. So now I'm gonna talk about each step.

[5:30](https://www.youtube.com/watch?v=imFedndyXYQ&t=330s) Now, to make things concrete, I wanna share with you how they apply on a hypothetical system. So this is order service. Order service, you type in an ID, and then you look up an order.

[5:41](https://www.youtube.com/watch?v=imFedndyXYQ&t=341s) So what kind of vu- vulnerabilities might we find in this? Um, the first step is the threat model. So why does this matter? So several teams, if you look at a code, right, finding-- have found that having a well-documented threat model really increases your true positive rate to ninety percent.

[5:58](https://www.youtube.com/watch?v=imFedndyXYQ&t=358s) Ninety percent is outstanding, if you ask me. I think anything above seventy-five is, is great. That's where you wanna aim for. Now, to put it another way, one CISO said this, "The model has great context of the code-"

[6:09](https://www.youtube.com/watch?v=imFedndyXYQ&t=369s) But poor context of the system. And why does this, why does this happen? It's the be- whatever the model can read, the model will understand well. But there's so much information that the model cannot read.

[6:20](https://www.youtube.com/watch?v=imFedndyXYQ&t=380s) Why did the senior designer design the system this way? What are all the vulnera- vulnerabilities that on-call has been fixing that has not been written down? All of this is now implicit context in your head, but you need to write it down somewhere, maybe in a threat model, to help the model be a better security engineer.

[6:37](https://www.youtube.com/watch?v=imFedndyXYQ&t=397s) So we can work with models to build these threat models, and the threat model is really just a document, right? First, we can bootstrap a threat model, give the model access to all your documents and code, even past code mo- comments, past patches that were made and what CVEs they were patching.

[6:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=411s) And you can ask the model to infer, hey, what possible... what other CVEs have not been patched, right? So the model can do that. Then after the next step is that you should ask the model to interview the expert on the system, right?

[7:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=425s) What are some things that could happen that we didn't plan for? Or what are some things that we actually don't need to worry about for vulnerability because maybe this is an internal VPN-only app, or there's a compensating control elsewhere.

[7:17](https://www.youtube.com/watch?v=imFedndyXYQ&t=437s) A lot of this is not in the code, right? If you look at your code base, it's not there. But it needs to be written down somewhere, so the model can benefit for it.

[7:24](https://www.youtube.com/watch?v=imFedndyXYQ&t=444s) So here's an example threat model of order service, right? You know, the key asset is the data, you know, con- con- uh, contains customer PII, and the entry point is the orders API.

[7:34](https://www.youtube.com/watch?v=imFedndyXYQ&t=454s) We can also see that the model has suggested some possible threat vectors, like SQL injection, and that the fact that you could call this API without any form of authentication.

[7:42](https://www.youtube.com/watch?v=imFedndyXYQ&t=462s) Then the second step is the sandbox. So isolation and reproducibility.

[7:49](https://www.youtube.com/watch?v=imFedndyXYQ&t=469s) In some cases, we may be running untrusted code. In some cases, we may be asking the model to deliberately create exploits. So we want some isolation, um, to prevent the model from doing bad things.

[8:00](https://www.youtube.com/watch?v=imFedndyXYQ&t=480s) So this isolation, doing bad things could be data exfiltration or, you know, dropping things in production. And of course, uh, you, you want all of this to be running in a VM without egress and of, of course, without your cloud credentials.

[8:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=493s) The second thing we want is reproducibility. So a lot of times, uh, you may find that the model may find a s- a bug because you were using a old dependency.

[8:22](https://www.youtube.com/watch?v=imFedndyXYQ&t=502s) Or you want, you want all of your agents to be starting from the same baseline container so that your agents can very f- can, can, can run everything in a harness in the pipeline.

[8:32](https://www.youtube.com/watch?v=imFedndyXYQ&t=512s) So now building a reprodu-- uh, representative sandbox of your system is actually quite difficult, but the capability is quite useful. I think, here's one team we worked with. They quoted, "The biggest f- the biggest lever we had is having the model testbeds, essentially sandboxes with live systems and where they can run and detonate the PoCs to confirm

[8:50](https://www.youtube.com/watch?v=imFedndyXYQ&t=530s) that they are true positives." So how would a sandbox for order service look like? Well, y- this is a simple sandbox. Uh, three Docker images connected together. One for the app, one for the Postgres database, and one for the Redis cache.

[9:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=545s) And on the right, we have the security agent, uh, which sits outside the target boundary trying to probe the app via HTTP.

[9:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=553s) Now that we have the setup done, we can now move on to doing the work of discovery, finding the vulnerabilities. So the first is no-- So what are some...

[9:24](https://www.youtube.com/watch?v=imFedndyXYQ&t=564s) There are three things that matter a lot, and it's no surprises here. The first thing is really context engineering. You're gonna give the model all the context, as much context as you can.

[9:33](https://www.youtube.com/watch?v=imFedndyXYQ&t=573s) If the, if it's written down and if you share it with the model, the model can find it, the model will be a better security engineer. You improve your vulnerability coverage.

[9:41](https://www.youtube.com/watch?v=imFedndyXYQ&t=581s) You reduce your false positive rate. The second is to have simple pro- simpler prompts. Um, how many of you here have found that as models get better, you actually need to remove from your prompts?

[9:54](https://www.youtube.com/watch?v=imFedndyXYQ&t=594s) That's what I found. With every new model version of step jump, I actually have to cut my prompt size by maybe about fifty percent. Right. So as models get better, your prompts need to be less and less specific, prescriptive even.

[10:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=605s) While, you know, previous, previous models, you might s- ask the model to look out for specific category classes. But for newer models, you can just probably say something like, "Look for where untrusted data hits the trust boundary," and the model is very good at inferring this.

[10:20](https://www.youtube.com/watch?v=imFedndyXYQ&t=620s) Finally, give the model tools. So one example, we had a pen testing team. They give the model tools to check the response, to, to query the API, check the response, read the logs, read the source code.

[10:32](https://www.youtube.com/watch?v=imFedndyXYQ&t=632s) Now, this is just like the tools that you give a regular security engineer, right? A lot of times you expect the model just read the code. That doesn't quite work.

[10:40](https://www.youtube.com/watch?v=imFedndyXYQ&t=640s) Allow the model to be dynamic, to read the tools, uh, to, to run the tools. And when they did this [chuckles], their true positive rate was almost a hundred percent because the model could actually verify in the loop.

[10:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=651s) So here is the GET API for the order service, and it's just five lines of code. Can anyone here spot the vulnerability?

[11:01](https://www.youtube.com/watch?v=imFedndyXYQ&t=661s) You can shout it out. SQL injection. I-- [chuckles] You guys, exactly. So this is what the discovery agent flag, right? You know, in line four, we build a SQL query via Python string interpolation.

[11:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=673s) So the user incu- input flows straight to the SQL query, right? And that means that someone can inject a SQL query that exfiltrates data from your database. This is code that I would have written, right?

[11:22](https://www.youtube.com/watch?v=imFedndyXYQ&t=682s) As me as a non-sec- security engineer, it's just a very... I just write the simplest code possible, and this is how it would look like. Current models are very good at detect-- spotting simple stuff like this.

[11:34](https://www.youtube.com/watch?v=imFedndyXYQ&t=694s) So after discovery, so we have maybe a thousand, let's say a hundred vulnerabilities, we may have not been able to confirm them. The next step is verification. So if you think about machine learning, discovery optimizes for recall, find as many vulnerabilities as possible.

[11:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=711s) Verification optimizes for precision. Of the hundred vulnerabilities we found, how many of them are truly optimi- uh, truly exploitable? So we have them both working separately. So we found that it's helpful to have these two steps separately, right?

[12:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=725s) Because when the discovery agent is trying to verify its own work in the loop, trying to debate against itself in the loop, it may actually self-censor, and this may actually hurt recall.

[12:15](https://www.youtube.com/watch?v=imFedndyXYQ&t=735s) Um, and when building the verification agent, it's helpful for the verification agent to be independent and adversarial. Independent means that the verification agent doesn't see the reasoning traces, doesn't see all the work that the discovery agent has done.

[12:28](https://www.youtube.com/watch?v=imFedndyXYQ&t=748s) All the verification agent gets is the vulnerability, and it approaches it from an adversarial point of view. Assume that this vulnerability is false, try to confirm it's false or confirm it's true.

[12:38](https://www.youtube.com/watch?v=imFedndyXYQ&t=758s) So this sets the-- this sets a very high bar for the vulnerability, which reduces the false positive rate. And also, if you have invested the time in building, building a representative sandbox, um, have the verification agent build the PoC, detonate the PoC to confirm if the verif-- the vulnerability is true.

[12:58](https://www.youtube.com/watch?v=imFedndyXYQ&t=778s) So in this case, back to order service, our verification agent is confirming the SQL injection, right? In a fresh container, and this is why we have everything in containers, the verification agent is only given line number four.

[13:10](https://www.youtube.com/watch?v=imFedndyXYQ&t=790s) Here's where SQL injection could happen. And the verification agent actually tries, just runs the curl command, and the verification agent can see all the customer PII that has been ex-exfiltrated.

[13:21](https://www.youtube.com/watch?v=imFedndyXYQ&t=801s) So now you have confirmation that a SQL injection can work.

[13:26](https://www.youtube.com/watch?v=imFedndyXYQ&t=806s) So after discovery and verification, we've maximized recall, we've optimized for precision, the next step is triage. So a lot of this may truly fire, but some of these are bug correctness issues.

[13:38](https://www.youtube.com/watch?v=imFedndyXYQ&t=818s) They have very low business impact. So across multiple teams, they shared that you can lose trust with product engineers by sending them all the vulnerabilities that are true, even those that are medium or low severity, because those engineers can't cope.

[13:53](https://www.youtube.com/watch?v=imFedndyXYQ&t=833s) And as we've seen so many times, the scarce resource now is engineer attention. So imagine you're a product engineer. What are the kinds of vulnerabilities you want to actually receive and fix?

[14:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=845s) So as part of this step, we can do deduplication, you know, cheap heuristic checks, or we can get a model to help you with clustering and deduplication. We should also consider the severity ex-- of the exploit, right?

[14:15](https://www.youtube.com/watch?v=imFedndyXYQ&t=855s) Uh, one thing to consider is the magnitude of the outcome, of the impact. What is the magnitude? Is it a ring zero magnitude or a ring one magnitude? Um, and also the likelihood of it occurring, right?

[14:25](https://www.youtube.com/watch?v=imFedndyXYQ&t=865s) How many hoops does an attacker have to jump through to actually make that exploit? And this is where the threat model is useful throughout. You can think of the threat model as just synonymous with context.

[14:36](https://www.youtube.com/watch?v=imFedndyXYQ&t=876s) Just give the model context about your business, right? So, for example, compensating controls. If you have a firewall, something that was initially high severity could actually be low. On the other hand, if you know that your database is mostly customer PII data or healthcare data, a model might have, have set that as medium severity, but it actually

[14:57](https://www.youtube.com/watch?v=imFedndyXYQ&t=897s) could be high because the, the contents of your data are so sensitive. So in this case, we have on the left

[15:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=905s) triage that the agent has done. The agent has said that this is high severity for these reasons, right? High impact, high likelihood. Now, on the right, after human review,

[15:16](https://www.youtube.com/watch?v=imFedndyXYQ&t=916s) the operator has confirmed that, you know, th- has downgraded this to low likelihood. And the reason why it's low likelihood is because we have an application firewall that blocks SQL injections, and also there's the fact that the service is internal only.

[15:28](https://www.youtube.com/watch?v=imFedndyXYQ&t=928s) It's only for internal i- warehousing or internal logistics. It's not, it's never internet facing. Therefore, the likelihood is low. Now, you can imagine, like, how would the model know this?

[15:37](https://www.youtube.com/watch?v=imFedndyXYQ&t=937s) This is not written in a code base. It may be in, available in a system design, it may be available in your documentation, it may be available in your Confluence, et cetera.

[15:45](https://www.youtube.com/watch?v=imFedndyXYQ&t=945s) So this is the kind of context you need to provide to the model so that the model can get better at this.

[15:52](https://www.youtube.com/watch?v=imFedndyXYQ&t=952s) Now, after that, we close the loop: patching. To validate the patch, you know, we can build a patch, and then we wanna validate. There's a ladder of sorts. First, the original PoC has to stop working.

[16:03](https://www.youtube.com/watch?v=imFedndyXYQ&t=963s) That's basic. Second, the existing test suite should stay green. No regression. And then we can go further, right? We can have a fresh discovery agent try to attack the patched code again.

[16:13](https://www.youtube.com/watch?v=imFedndyXYQ&t=973s) Is the patch comprehensive enough? And teams have found that by giving the patching agent such feedback, um, you can actually... The patch quality improves greatly, right? And you know, this is a generative verifier loop.

[16:23](https://www.youtube.com/watch?v=imFedndyXYQ&t=983s) Give your agents context, give your agents feedback, and they, they can do the, do their work. And finally, a human confirms the patch before it's been merged. So here's how the patch will look like.

[16:32](https://www.youtube.com/watch?v=imFedndyXYQ&t=992s) You know, in the first diff, we update the code. It's a one-line fix where we move the variable out of Python. Uh, and therefore, we reduce the likelihood of SQL injection.

[16:41](https://www.youtube.com/watch?v=imFedndyXYQ&t=1001s) And in the second diff, not only do we update the code, not only do we fix the system, we close the loop. We make sure that the s-- next time we run this loop, we run this scan, these issues don't happen again.

[16:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=1011s) And this is really important when you're building the, building harnesses, right? You're building loops, they're operational expense, but when you close the loop, they now become capital expense. You get better with each iteration you run.

[17:03](https://www.youtube.com/watch?v=imFedndyXYQ&t=1023s) So we make clear the compensating controls we have, the ap-application firewall, and the fact that it's internal only. Okay, so the quick recap. These are the six steps, and each step focuses on one thing.

[17:14](https://www.youtube.com/watch?v=imFedndyXYQ&t=1034s) Threat model provides context. Sandbox provides isolation and reproducibility. This is the baseline fundamentals that you need to have. This will carry throughout every scan you have. Context and reproducibility.

[17:27](https://www.youtube.com/watch?v=imFedndyXYQ&t=1047s) Then after that, the rest is really, if you look at it, it's... If you, if you squint, it actually looks like an ML pipeline. Recall, precision, ranking, and closing the loop.

[17:38](https://www.youtube.com/watch?v=imFedndyXYQ&t=1058s) It's a regular machine learning pipeline. Now, that's the technical challenge we have. There's another non-technical challenge we have, and as my previous director would say, non-technical problems are an order of magnitude harder than technical problems.

[17:54](https://www.youtube.com/watch?v=imFedndyXYQ&t=1074s) So what are some other bottlenecks you might have? So imagine you have a process. What happens when you have one or two orders of magnitude more input? What is gonna start to break?

[18:06](https://www.youtube.com/watch?v=imFedndyXYQ&t=1086s) In the case of the harness, the scanning harness, you just have more engineering, you spend more compute, you pay more money. Things that can be solved with money are not really problems.

[18:15](https://www.youtube.com/watch?v=imFedndyXYQ&t=1095s) But human attention doesn't scale. Your dev, your product engineers, and your security engineers, what if they don't agree on what high severity or cal- ca- uh, tr- critical severity is?

[18:27](https://www.youtube.com/watch?v=imFedndyXYQ&t=1107s) You need to put them in a room, you need to write down all these rules so that everyone agrees. And, you know, your threat models, they're all stuck in people's heads right now.

[18:35](https://www.youtube.com/watch?v=imFedndyXYQ&t=1115s) Someone needs... They need to be interviewed, maybe by a model or by someone else, and write all their threat model down so that now you can have agents help you.

[18:43](https://www.youtube.com/watch?v=imFedndyXYQ&t=1123s) And finally, you know, patching, patch, patch review. Um, I still don't know of many companies that have gone through fully automated patch review, especially when pi- uh, patching security issues.

[18:55](https://www.youtube.com/watch?v=imFedndyXYQ&t=1135s) So here are some organizational bottlenecks that you might hit and what you can do. The first thing is vulnerability routing. So if you maybe get a, a dozen vulnerabilities a month, you can probably curate them, you know, email them to specific teams, assign Jira tickets.

[19:08](https://www.youtube.com/watch?v=imFedndyXYQ&t=1148s) When you have hundreds, you can't do this. And this can really be as simple as, you know, sending it to the code owner, et cetera, um, or, or the service owner, and it doesn't really need an LLM in the loop.

[19:20](https://www.youtube.com/watch?v=imFedndyXYQ&t=1160s) The next, second thing is severity calibration. The, the one hard thing we found is that product engineers and security engineers, red team, blue team, they may not agree on what high severity is, and all this requires a lot of your business context.

[19:32](https://www.youtube.com/watch?v=imFedndyXYQ&t=1172s) I really think it helps to have everyone discuss and write it all down once and for all, agree upon that, that you can send that to agent. And finally, the patching bandwidth.

[19:42](https://www.youtube.com/watch?v=imFedndyXYQ&t=1182s) Um, I'm sure no one writes... Very few of you here write code by hand now, but it's also very challenging for you to just be given the vulnerability and try to implement a patch, even with LLM assistance.

[19:54](https://www.youtube.com/watch?v=imFedndyXYQ&t=1194s) You definitely wanna be moving towards AI-generated patches, and then have the human in the loop verify it. And, you know, generate the patch, reattack the patch. Okay.

[20:05](https://www.youtube.com/watch?v=imFedndyXYQ&t=1205s) If you only remember one thing, well, here are the three things you want-- I want you to remember. I would like you to start now. Please, start with open source dependencies.

[20:14](https://www.youtube.com/watch?v=imFedndyXYQ&t=1214s) Start with whatever you're comfortable with. Two, I wanna teach you how to climb the learning curve. Don't try to aim for automation immediately, right? Start interactively. Do it hands on the wheel with, uh, Claude Code or your favorite IDE, doesn't matter.

[20:29](https://www.youtube.com/watch?v=imFedndyXYQ&t=1229s) Learn where you get cut. Learn what kind of context you're missing. Learn where precision is low. Then third is don't just aim for scanning. Uh, scanning is not the bottleneck.

[20:41](https://www.youtube.com/watch?v=imFedndyXYQ&t=1241s) Verification, triage, and patching, and then all your organizational processes, those are the bottleneck. Over there, I have a couple of resources. We have Claude Security to scan your code base, validate findings, suggest patches.

[20:51](https://www.youtube.com/watch?v=imFedndyXYQ&t=1251s) We have a blog. We have open source repos for, with, that includes the interactive skills and autonomous harnesses. And all you have to do is to run these few steps to very quickly get hands on the wheel.

[21:03](https://www.youtube.com/watch?v=imFedndyXYQ&t=1263s) And in it, if you look at step five, you have the harness, and you can just customize it.

[21:09](https://www.youtube.com/watch?v=imFedndyXYQ&t=1269s) That's all I had. Thank you. [audience applauding] [outro jingle]
