# Your KV Cache Can Spot Attacks on Your AI Agents - Ari Azarafrooz, Intrinsec AI Ari Azarafrooz, Intrinsec AI [Talk page](https://ai.engineer/talks/1NWnSwCB4NM-your-kv-cache-spot-attacks-on) · [Video](https://www.youtube.com/watch?v=1NWnSwCB4NM) ## Your KV Cache Can Spot Attacks on Your AI Agents A hidden instruction in a voicemail gives a text filter nothing to inspect. Ari Azarafrooz of Intrinsec AI shows how the key-value pages a voice agent writes during inference can point to the passage that deserves a closer look. ### What the engine knows before a tool call [0:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=0s) An inference engine keeps the keys and values it computes for each token so it can reuse them as it generates a response. For a team running its own inference stack, that key-value (KV) cache offers a second use. Before the agent generates a response that might become a tool call, the engine has already recorded part of the computation that led to it. A judge model can check prompts and responses against a policy. A team can also train a classifier for each failure it knows to look for, from prompt injection to sensitive data exposure. The cache gives those guardrails another source of evidence. An input that slips through a filter still affects the model’s computation. A detector can examine that computation in place, even when the team cannot read raw inputs or outputs. The unit of comparison is a *page*, a block of cached keys and values covering a span of tokens. The detector first learns what pages look like in ordinary traffic for a particular workload. It then flags pages that are rare against that baseline. Rarity tells an engineer where to look. A classifier or explainer can take the further step of describing what happened there. ### Two places to read a KV page [2:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=145s) Caching makes inference stateful in a practical sense. A page can outlive the computation that produced it, and its location matters for the next request. LMCache can store pages across tiers extending from GPU memory to remote storage. Cache-aware routing systems such as llm-d and NVIDIA Dynamo can direct a new request toward a worker that already holds a relevant page. Where should a detector read those pages? The choice depends on when someone needs the result. - **Inside the inference engine.** A detector integrated with an engine such as vLLM or SGLang could raise an alert while the request is running. The alert could accompany a subsequent tool call, giving the guardrail that evaluates the call evidence about the computation that preceded it. - **At the cache manager.** A detector integrated with LMCache could scan stored pages for ones that stand out from ordinary traffic. This gives an investigator a way to discover suspicious activity across retained requests, including ones they had no prior reason to inspect. ### Finding the unusual passage in a voicemail [3:47](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=227s) The voice-agent experiment started with audio and no input text to filter. The team built its baseline from 72 ordinary clips spoken in four voices. It divided the cache into 16-token pages. Choosing short pages gave the detector more examples of ordinary computation from the same 72 recordings. Could the detector tell an engineer *where* to look within a recording? The test used different voices and compared a clean clip with a voicemail containing a hidden spoken instruction. In the voicemail, simple anomaly detection flagged an unusual KV page. Because that page covered a particular span of the request, the team could narrow its inspection to the part of the recording associated with the anomaly, without first transcribing the instruction. The team then trained a second model to use KV pages as features and assign descriptive tags. Although this tagger had received no audio training, Azarafrooz reports that it correctly identified categories including personal information, home address, location, travel information and destination in the example. That gave an investigator two clues from the same cache. The unusual page located the event, and the tags described the kind of information present. ### What turned up in the top ten text-agent alerts [5:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=309s) For text agents, the team examined traffic from a live 14-billion-parameter model. A comparison of attack traffic with ordinary traffic illustrated how the two tended to separate. A benchmark then posed a question an engineer could use to budget review time. If you inspect only the ten highest-ranked alerts, how often do you find something worth investigating? The benchmark supplied its own baseline of benign agent traces and eight kinds of agent failure, including injected tool descriptions and misinformation. Of the detector’s top ten alerts, about seven or eight pointed to activity the team judged meaningfully anomalous. That measures the yield at the top of the review queue. Estimating what fraction of all failures the detector caught would also require counting failures outside those ten alerts. ### Changing models and the cost of moving pages [6:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=363s) Each model writes keys and values in its own coordinates. A detector familiar with one model’s ordinary traffic therefore needs a way to interpret another model’s pages. Fine-tunes and new releases create this problem over time. Agents running different models, or a router choosing a specialist for each request, can create it within a single deployment. Relearning the baseline, classifier and explainer for every model can become computationally prohibitive. Azarafrooz proposes mapping their pages into a shared representation so the same analysis can apply across models. Real-time analysis has a second cost. The engine has already computed the keys and values, but transferring a chunk of KV data to a detector takes time. Compression could reduce how much data has to move, and streaming could let analysis begin as the data arrives. For an alert meant to inform a tool-call decision, the deadline is the decision itself. The detector has to receive the relevant data, analyze it and deliver the alert while the guardrail can still act. ## Key takeaways - Page size determines both how many examples a baseline contains and how narrowly an alert locates an event. The voice experiment used 16-token pages to get more observations from 72 ordinary clips. - On the text-agent benchmark, about seven or eight of the top ten alerts pointed to meaningful anomalies. The result describes what an investigator found within a limited review budget. - An alert intended to inform a tool-call decision faces two production constraints. The detector must interpret the serving model’s KV representation and process the relevant pages in time for the guardrail to act. ## Slides ### KV-first security. The signal your engine already writes. Between the prompt and response checks, the engine records its computation in the KV cache. The red square marks a page that stands out against a baseline learned from the agent’s own traffic, before the agent calls a tool. [0:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=0s) [Lines from agent failure types converge on a stream of KV pages between prompt and response classifiers. A red square marks a page that falls outside the baseline learned from normal traffic.](https://assets.ai.engineer/sites/aiecode2025/sha256/da72fbf30ae0f636d9bbc404038f5c50f3e50fb128fbc9d82a0dff017235abad.webp) THE IDEA KV-first security. The signal your engine already writes. WAYS AN AGENT GOES WRONG Prompt injection Jailbreaks Rogue tool calls Broken tool calls Sensitive-data exposure Hidden backdoor Drift …and the ones not named yet JUDGE OR CLASSIFIER reads the prompt IN THE COMPUTATION · KV PAGES one baseline, learned from your own traffic does not fit JUDGE OR CLASSIFIER reads the response Tool calls, actions THE GUARDRAILS’ BLIND SPOT ALREADY THERE Your engine writes it anyway. [illegible] the gems from the pile. COVERS THE BLIND SPOTS What slips past the guardrails still becomes computation. STAYS IN PLACE Read it where it lives. The prompt text goes nowhere new. [illegible]SEC AI · AI ENGINEER CODE SUMMIT 2026 ### Inference is stateful now. Read the state. As the inference engine writes each KV page, a live check can send an alert with the tool call for policy to act on. A cache check can score a stored page once because later turns reuse that page exactly as stored. [2:28](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=148s) [Three agent turns feed a KV-aware router and an inference engine. An engine tap reads pages as they are written. A cache tap reads stored pages for reuse across turns.](https://assets.ai.engineer/sites/aiecode2025/sha256/a5adafa324def18e185e82499770784ab99eff6e0a44f60c57928937a23fea6a.webp) Inference is stateful now. Read the state. WHERE TO READ IT llm-d and NVIDIA Dynamo route each turn back to the KV it built, and LMCache keeps that KV in tiers. AN AGENT SESSION Turn 1 · reads the README Turn 2 · runs the tests Turn 3 · calls a tool KV-aware router llm-d · NVIDIA Dynamo sends each turn back to its KV Inference engine vLLM · SGLang alert Tool calls, actions ENGINE TAP live, as the KV is written KV store GPU → CPU → SSD → remote, via LMCache TURN 1 TURN 2 TURN 3 CACHE TAP at rest, in the store ENGINE TAP · LIVE Reads each page as it is written. The alert rides along with the tool call, [illegible] your policy decides. CACHE TAP · AT REST Reads the session your stack already keeps. A cached page is reused exactly as stored, so score it once for every turn that reuses it. [illegible]SEC AI · AI ENGINEER CODE SUMMIT 2026 Tested on vLLM + LMCache · session routing: llm-d 1.0 and NVIDIA Dynamo router docs, Nov 2026 ### Normal stays together. What does not fit, separates. On live Qwen3-14B traffic, attacks tended to score as more unusual than normal traffic, though their scores overlapped. On ATBench, one baseline yielded AUROCs of 0.68–0.78 across eight kinds of agent failure. [5:08](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=308s) [A mirrored histogram plots attack counts above the axis and normal traffic counts below it. Attacks cluster farther right on the anomaly score axis. A table reports AUROC values from 0.68 to 0.78 across eight failure categories.](https://assets.ai.engineer/sites/aiecode2025/sha256/250458c0a76418bb63c4aa57996f3df6d16dc4f6de861a04ccbc742d61dd1e2c.webp) WHAT IT SHOWS Normal stays together. What does not fit, separates. ON AGENT TRAFFIC, SERVED LIVE what does not belong separates attacks, above the line normal traffic, below Y-axis ticks: 10, 20, 30, 40 X-axis ticks: −1, 0, 1, 2 score against your normal traffic → more unusual ACROSS EIGHT KINDS OF AGENT FAILURE · AUROC Corrupted tool feedback 0.78 Direct prompt injection 0.68 Indirect prompt injection 0.78 Inherent agent failures 0.74 Malicious tool execution 0.69 Jailbreak 0.77 Tool description injection 0.72 Misinformation 0.75 0.5 · chance 1.0 · perfect separation One baseline, every category. [illegible]SEC AI · AI ENGINEER CODE SUMMIT 2026 KV-page benchmark, Oct 2026 · Qwen3-14B on vLLM + LMCache · left: 62 normal / 216 attacks · right: ATBench, by risk source ### The hard parts. Each model writes KV pages in its own coordinates, so normal traffic on a new model can fall outside the old baseline. Moving those pages slows real-time analysis: 256 tokens on Qwen3-14B take about 1 KB as text but about 40 MB as KV. [6:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=362s) [The left diagram shows how a baseline learned on one model can miss clusters from other models. The right diagram compares a 256-token chunk of about 1 KB of text with about 40 MB of KV data for Qwen3-14B.](https://assets.ai.engineer/sites/aiecode2025/sha256/d6448b39211cb0bfa25295a397558485c3fe1fc6260ccc12793fafa599035b69.webp) WHAT IS STILL HARD The hard parts. ONE MODEL your baseline FINE-TUNES · AGENTS · ROUTERS the baseline you learned Models change, and multiply. Fine-tunes, new releases, agents on different models, routers that pick one per request. Each writes pages in its own coordinates, so a baseline, classifier or explainer built on one model does not carry over to the next. AS TEXT · ~1 KB THE SAME 256-TOKEN CHUNK AS KV · ~40 MB Real time means moving big objects. One 256-token chunk of Qwen3-14B is about 40 MB of KV. Moving it for real-time security analysis is still slow, even with KV compression and streaming such as CacheGen. Liu et al., “CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving,” ACM SIGCOMM 2024 That’s what we solved at Intrinsec AI. INTRINSEC AI We’re looking for teams building their own inference stacks and treating security as a foundational design decision, not something bolted on as an afterthought. intrinsec.ai ## Full transcript [0:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=0s) Hi, I'm Arius, founder of Pinterse AI. [0:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=4s) Hi, I'm Arius, founder of Pinterse AI. [0:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=4s) Hi, I'm Arius, founder of Pinterse AI. Um, this talk is for people who build [0:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=7s) Um, this talk is for people who build [0:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=7s) Um, this talk is for people who build agents and run their own inference stack [0:10](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=10s) agents and run their own inference stack [0:10](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=10s) agents and run their own inference stack or plan to. And if that's you, you [0:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=13s) or plan to. And if that's you, you [0:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=13s) or plan to. And if that's you, you associate typically the KB cache, the [0:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=15s) associate typically the KB cache, the [0:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=15s) associate typically the KB cache, the bird and butter of modern inference [0:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=17s) bird and butter of modern inference [0:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=17s) bird and butter of modern inference engines, usually as a throughput [0:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=18s) engines, usually as a throughput [0:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=18s) engines, usually as a throughput optimization, the keys and values [0:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=20s) optimization, the keys and values [0:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=20s) optimization, the keys and values computed per token kept so the model [0:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=22s) computed per token kept so the model [0:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=22s) computed per token kept so the model doesn't compute them again. Today, I'll [0:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=24s) doesn't compute them again. Today, I'll [0:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=24s) doesn't compute them again. Today, I'll like to show KV cache in a different [0:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=26s) like to show KV cache in a different [0:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=26s) like to show KV cache in a different light. The way agents agent AI agent [0:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=29s) light. The way agents agent AI agent [0:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=29s) light. The way agents agent AI agent application could go wrong are keep [0:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=31s) application could go wrong are keep [0:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=31s) application could go wrong are keep growing, right? Prom injection, [0:33](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=33s) growing, right? Prom injection, [0:33](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=33s) growing, right? Prom injection, jailbreaks, broken tool calls, sensitive [0:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=35s) jailbreaks, broken tool calls, sensitive [0:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=35s) jailbreaks, broken tool calls, sensitive tool calls, sensitive data exposures, [0:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=37s) tool calls, sensitive data exposures, [0:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=37s) tool calls, sensitive data exposures, hidden back doors and drifts and so [0:39](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=39s) hidden back doors and drifts and so [0:39](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=39s) hidden back doors and drifts and so forth, right? So the way right now we [0:41](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=41s) forth, right? So the way right now we [0:41](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=41s) forth, right? So the way right now we typically guard against against this [0:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=43s) typically guard against against this [0:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=43s) typically guard against against this numerous ways that an AI agent [0:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=45s) numerous ways that an AI agent [0:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=45s) numerous ways that an AI agent application could go wrong is by [0:46](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=46s) application could go wrong is by [0:46](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=46s) application could go wrong is by guarding the edge by perhaps passing a [0:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=49s) guarding the edge by perhaps passing a [0:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=49s) guarding the edge by perhaps passing a judge model that checks the prompts, the [0:52](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=52s) judge model that checks the prompts, the [0:52](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=52s) judge model that checks the prompts, the inputs and the response outputs against [0:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=55s) inputs and the response outputs against [0:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=55s) inputs and the response outputs against your policy. All right? or you build a [0:57](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=57s) your policy. All right? or you build a [0:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=58s) your policy. All right? or you build a classifier for each case. But here is a [1:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=61s) classifier for each case. But here is a [1:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=61s) classifier for each case. But here is a different light. The model turns every [1:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=64s) different light. The model turns every [1:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=64s) different light. The model turns every input into a computation before it acts [1:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=67s) input into a computation before it acts [1:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=67s) input into a computation before it acts before it decodes before it generates [1:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=69s) before it decodes before it generates [1:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=69s) before it decodes before it generates the response before that response [1:10](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=70s) the response before that response [1:10](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=70s) the response before that response becomes another tool call. The KV cache [1:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=73s) becomes another tool call. The KV cache [1:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=73s) becomes another tool call. The KV cache is your engine's record of that [1:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=75s) is your engine's record of that [1:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=75s) is your engine's record of that computation that modern inference engine [1:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=78s) computation that modern inference engine [1:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=78s) computation that modern inference engine for free are using it and reading it. [1:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=80s) for free are using it and reading it. [1:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=80s) for free are using it and reading it. And therefore in this talk I would like [1:23](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=83s) And therefore in this talk I would like [1:23](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=83s) And therefore in this talk I would like to suggest as as a complement to the way [1:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=86s) to suggest as as a complement to the way [1:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=86s) to suggest as as a complement to the way Are you already using your own [1:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=87s) Are you already using your own [1:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=87s) Are you already using your own guardrails to use this data and [1:30](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=90s) guardrails to use this data and [1:30](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=90s) guardrails to use this data and integrate it with your KV cache [1:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=91s) integrate it with your KV cache [1:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=91s) integrate it with your KV cache infrastructure? I refer to this as a KB [1:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=94s) infrastructure? I refer to this as a KB [1:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=94s) infrastructure? I refer to this as a KB first security signals and basically [1:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=97s) first security signals and basically [1:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=97s) first security signals and basically three reasons that matters, right? So [1:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=100s) three reasons that matters, right? So [1:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=100s) three reasons that matters, right? So the data is already there. You pick the [1:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=103s) the data is already there. You pick the [1:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=103s) the data is already there. You pick the gems from the pile your inference stack [1:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=105s) gems from the pile your inference stack [1:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=105s) gems from the pile your inference stack already was throwing away. It covers the [1:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=108s) already was throwing away. It covers the [1:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=108s) already was throwing away. It covers the guardrails blind spots because whatever [1:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=111s) guardrails blind spots because whatever [1:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=111s) guardrails blind spots because whatever slips past those guardrails, it still [1:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=113s) slips past those guardrails, it still [1:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=113s) slips past those guardrails, it still becomes computation and it stays in [1:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=115s) becomes computation and it stays in [1:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=115s) becomes computation and it stays in place. So it's perfect for data privacy [1:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=118s) place. So it's perfect for data privacy [1:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=118s) place. So it's perfect for data privacy application for cases where you do not [2:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=120s) application for cases where you do not [2:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=120s) application for cases where you do not have the luxury to read the inputs and [2:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=122s) have the luxury to read the inputs and [2:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=122s) have the luxury to read the inputs and outputs. And the recipe is pretty [2:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=124s) outputs. And the recipe is pretty [2:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=124s) outputs. And the recipe is pretty straightforward, right? It has three [2:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=126s) straightforward, right? It has three [2:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=126s) straightforward, right? It has three steps. You learn a baseline for your AI [2:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=129s) steps. You learn a baseline for your AI [2:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=129s) steps. You learn a baseline for your AI workload, for your AI inference traffic, [2:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=131s) workload, for your AI inference traffic, [2:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=131s) workload, for your AI inference traffic, how your KB page typically look like, [2:14](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=134s) how your KB page typically look like, [2:14](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=134s) how your KB page typically look like, and then find the KV pages that are rare [2:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=138s) and then find the KV pages that are rare [2:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=138s) and then find the KV pages that are rare against your baseline, typically your [2:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=140s) against your baseline, typically your [2:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=140s) against your baseline, typically your anomalies, and then you can even [2:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=142s) anomalies, and then you can even [2:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=142s) anomalies, and then you can even classify and explain those. So first [2:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=145s) classify and explain those. So first [2:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=145s) classify and explain those. So first let's start with where in this stack [2:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=147s) let's start with where in this stack [2:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=147s) let's start with where in this stack this integration could happen. [2:30](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=150s) this integration could happen. [2:30](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=150s) this integration could happen. The interesting thing about the KV cache [2:32](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=152s) The interesting thing about the KV cache [2:32](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=152s) The interesting thing about the KV cache is that in a way existence of those had [2:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=155s) is that in a way existence of those had [2:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=155s) is that in a way existence of those had made the inference in a way stateful [2:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=158s) made the inference in a way stateful [2:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=158s) made the inference in a way stateful right. So for example, things like KBK [2:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=160s) right. So for example, things like KBK [2:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=160s) right. So for example, things like KBK cache management layers such as LM cache [2:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=163s) cache management layers such as LM cache [2:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=163s) cache management layers such as LM cache stores them from GPU down to remote [2:46](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=166s) stores them from GPU down to remote [2:46](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=166s) stores them from GPU down to remote storage or LLMD and Nvidia dynamo route [2:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=169s) storage or LLMD and Nvidia dynamo route [2:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=169s) storage or LLMD and Nvidia dynamo route each new request towards the worker that [2:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=171s) each new request towards the worker that [2:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=171s) each new request towards the worker that already holds perhaps that cached KV [2:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=174s) already holds perhaps that cached KV [2:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=174s) already holds perhaps that cached KV page. So there are two integration [2:56](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=176s) page. So there are two integration [2:56](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=176s) page. So there are two integration points you you you can integrate those [2:59](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=179s) points you you you can integrate those [2:59](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=179s) points you you you can integrate those anomaly detection or those classifiers [3:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=182s) anomaly detection or those classifiers [3:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=182s) anomaly detection or those classifiers into either you can implement it [3:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=184s) into either you can implement it [3:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=184s) into either you can implement it directly inside of your inference [3:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=186s) directly inside of your inference [3:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=186s) directly inside of your inference engines vg so forth. The beauty of this [3:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=189s) engines vg so forth. The beauty of this [3:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=189s) engines vg so forth. The beauty of this approach is that it's real time and and [3:12](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=192s) approach is that it's real time and and [3:12](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=192s) approach is that it's real time and and the the the alert that it would perhaps [3:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=195s) the the the alert that it would perhaps [3:15](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=195s) the the the alert that it would perhaps generate could ride along your future [3:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=198s) generate could ride along your future [3:18](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=198s) generate could ride along your future tool calls or your future policies. So [3:21](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=201s) tool calls or your future policies. So [3:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=202s) tool calls or your future policies. So you can harden basically your policies [3:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=204s) you can harden basically your policies [3:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=204s) you can harden basically your policies or guardrails by just generating the [3:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=206s) or guardrails by just generating the [3:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=206s) or guardrails by just generating the computational anomalies that you observe [3:28](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=208s) computational anomalies that you observe [3:28](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=208s) computational anomalies that you observe within your inference engines [3:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=211s) within your inference engines [3:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=211s) within your inference engines and perhaps you can integrate it with as [3:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=214s) and perhaps you can integrate it with as [3:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=214s) and perhaps you can integrate it with as a as a way of thread discovery in a [3:36](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=216s) a as a way of thread discovery in a [3:36](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=216s) a as a way of thread discovery in a sense. So you can integrate with your LM [3:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=218s) sense. So you can integrate with your LM [3:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=218s) sense. So you can integrate with your LM cache. So it reads the stored KB pages [3:42](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=222s) cache. So it reads the stored KB pages [3:42](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=222s) cache. So it reads the stored KB pages and find things that don't belong with [3:44](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=224s) and find things that don't belong with [3:44](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=224s) and find things that don't belong with the rest. Right? So now I want to show [3:47](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=227s) the rest. Right? So now I want to show [3:47](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=227s) the rest. Right? So now I want to show you some numbers and and just a very [3:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=229s) you some numbers and and just a very [3:49](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=229s) you some numbers and and just a very simple hello world use case. if this is [3:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=231s) simple hello world use case. if this is [3:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=231s) simple hello world use case. if this is actually working and this is a valid [3:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=234s) actually working and this is a valid [3:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=234s) actually working and this is a valid approach and for that let me use a use [3:57](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=237s) approach and for that let me use a use [3:57](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=237s) approach and for that let me use a use case that has no text to filter at all a [4:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=240s) case that has no text to filter at all a [4:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=240s) case that has no text to filter at all a voice agent. So here's a case study of a [4:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=243s) voice agent. So here's a case study of a [4:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=243s) voice agent. So here's a case study of a voice agent. In this case it was a gem [4:05](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=245s) voice agent. In this case it was a gem [4:05](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=245s) voice agent. In this case it was a gem for E4B. Our baseline was 72 ordinary [4:08](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=248s) for E4B. Our baseline was 72 ordinary [4:08](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=248s) for E4B. Our baseline was 72 ordinary clips and spoken four voices. We're [4:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=251s) clips and spoken four voices. We're [4:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=251s) clips and spoken four voices. We're looking at 16 token pages. Obviously [4:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=253s) looking at 16 token pages. Obviously [4:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=253s) looking at 16 token pages. Obviously this granular granularity could change [4:16](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=256s) this granular granularity could change [4:16](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=256s) this granular granularity could change but here we just use shorter pages. So [4:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=260s) but here we just use shorter pages. So [4:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=260s) but here we just use shorter pages. So we could have more data in the baseline. [4:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=262s) we could have more data in the baseline. [4:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=262s) we could have more data in the baseline. And then on the test we have different [4:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=264s) And then on the test we have different [4:24](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=264s) And then on the test we have different voices and one clean voice and the other [4:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=267s) voices and one clean voice and the other [4:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=267s) voices and one clean voice and the other one has a hidden instruction within it. [4:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=269s) one has a hidden instruction within it. [4:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=269s) one has a hidden instruction within it. And as you can as is shown here not only [4:33](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=273s) And as you can as is shown here not only [4:33](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=273s) And as you can as is shown here not only you can detect these cases just using [4:36](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=276s) you can detect these cases just using [4:36](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=276s) you can detect these cases just using simple anomaly detection but also you [4:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=278s) simple anomaly detection but also you [4:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=278s) simple anomaly detection but also you would know which part of um which page [4:42](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=282s) would know which part of um which page [4:42](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=282s) would know which part of um which page which part of your request had this [4:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=285s) which part of your request had this [4:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=285s) which part of your request had this anomalous event. We even went further [4:47](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=287s) anomalous event. We even went further [4:47](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=287s) anomalous event. We even went further and we trained a model that uses the KD [4:50](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=290s) and we trained a model that uses the KD [4:50](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=290s) and we trained a model that uses the KD pages as features and we then we're [4:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=293s) pages as features and we then we're [4:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=293s) pages as features and we then we're trying to explain it or tag it with [4:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=295s) trying to explain it or tag it with [4:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=295s) trying to explain it or tag it with things that are useful. In this case, [4:57](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=297s) things that are useful. In this case, [4:57](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=297s) things that are useful. In this case, although it wasn't even trained on audio [4:59](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=299s) although it wasn't even trained on audio [4:59](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=299s) although it wasn't even trained on audio is able to generate correct information [5:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=301s) is able to generate correct information [5:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=301s) is able to generate correct information about personal information, home [5:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=303s) about personal information, home [5:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=303s) about personal information, home address, location data, travel [5:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=304s) address, location data, travel [5:04](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=304s) address, location data, travel information, and destination. [5:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=309s) So now back to text agents. In this [5:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=311s) So now back to text agents. In this [5:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=311s) So now back to text agents. In this case, it was a quan 3 14 billion served [5:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=313s) case, it was a quan 3 14 billion served [5:13](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=313s) case, it was a quan 3 14 billion served live on 11 cache. The one on the left [5:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=317s) live on 11 cache. The one on the left [5:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=317s) live on 11 cache. The one on the left shows you an idea of how attacks and [5:19](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=319s) shows you an idea of how attacks and [5:19](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=319s) shows you an idea of how attacks and normal traffic would typically separate [5:21](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=321s) normal traffic would typically separate [5:21](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=321s) normal traffic would typically separate from each other. And then the one on the [5:23](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=323s) from each other. And then the one on the [5:23](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=323s) from each other. And then the one on the right, which is perhaps more meaningful, [5:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=325s) right, which is perhaps more meaningful, [5:26](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=326s) right, which is perhaps more meaningful, it was using the AT bench and hogging [5:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=329s) it was using the AT bench and hogging [5:29](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=329s) it was using the AT bench and hogging face. It has its own baseline of benign [5:32](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=332s) face. It has its own baseline of benign [5:32](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=332s) face. It has its own baseline of benign agent traces and eight kind of agent [5:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=335s) agent traces and eight kind of agent [5:35](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=335s) agent traces and eight kind of agent failure from injected tools description [5:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=337s) failure from injected tools description [5:37](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=337s) failure from injected tools description to misinformation and so forth. We look [5:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=340s) to misinformation and so forth. We look [5:40](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=340s) to misinformation and so forth. We look at the top 10 alerts that anomalous [5:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=343s) at the top 10 alerts that anomalous [5:43](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=343s) at the top 10 alerts that anomalous anomaly detection would generate and [5:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=345s) anomaly detection would generate and [5:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=345s) anomaly detection would generate and about seven to eight times of out of 10 [5:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=348s) about seven to eight times of out of 10 [5:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=348s) about seven to eight times of out of 10 of these alerts are actually anomalous [5:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=351s) of these alerts are actually anomalous [5:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=351s) of these alerts are actually anomalous meaningful and useful anomalous [5:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=353s) meaningful and useful anomalous [5:53](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=353s) meaningful and useful anomalous activities. We even went further and we [5:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=355s) activities. We even went further and we [5:55](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=355s) activities. We even went further and we looked at whether there's any kind of [5:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=358s) looked at whether there's any kind of [5:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=358s) looked at whether there's any kind of anomalous events that stand out more [6:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=360s) anomalous events that stand out more [6:00](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=360s) anomalous events that stand out more than the rest. [6:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=363s) than the rest. [6:03](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=363s) than the rest. Um, two make things hard in production. [6:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=366s) Um, two make things hard in production. [6:06](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=366s) Um, two make things hard in production. First models change finetunes and new [6:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=371s) First models change finetunes and new [6:11](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=371s) First models change finetunes and new releases, but also agents that each run [6:14](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=374s) releases, but also agents that each run [6:14](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=374s) releases, but also agents that each run on a different model and routers that [6:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=377s) on a different model and routers that [6:17](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=377s) on a different model and routers that pick a specialist model per request. [6:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=380s) pick a specialist model per request. [6:20](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=380s) pick a specialist model per request. That means each model writes its pages [6:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=382s) That means each model writes its pages [6:22](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=382s) That means each model writes its pages its own coordinates. So either you learn [6:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=385s) its own coordinates. So either you learn [6:25](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=385s) its own coordinates. So either you learn every baseline, classifier and explainer [6:27](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=387s) every baseline, classifier and explainer [6:28](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=388s) every baseline, classifier and explainer which is computationally prohibitive or [6:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=391s) which is computationally prohibitive or [6:31](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=391s) which is computationally prohibitive or you have to find a way to map them all [6:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=394s) you have to find a way to map them all [6:34](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=394s) you have to find a way to map them all into one shared representation space. [6:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=398s) into one shared representation space. [6:38](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=398s) into one shared representation space. Second real time means moving big [6:41](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=401s) Second real time means moving big [6:41](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=401s) Second real time means moving big objects. For example, a 1256 token chunk [6:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=405s) objects. For example, a 1256 token chunk [6:45](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=405s) objects. For example, a 1256 token chunk of quen 314 billion is about 40 [6:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=408s) of quen 314 billion is about 40 [6:48](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=408s) of quen 314 billion is about 40 megabytes of KV compression and the [6:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=411s) megabytes of KV compression and the [6:51](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=411s) megabytes of KV compression and the streaming like hashgen could help but [6:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=414s) streaming like hashgen could help but [6:54](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=414s) streaming like hashgen could help but moving that for security analysis and [6:56](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=416s) moving that for security analysis and [6:56](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=416s) moving that for security analysis and acting in real times is still a slow and [6:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=418s) acting in real times is still a slow and [6:58](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=418s) acting in real times is still a slow and that's what we solve at in AI. We are [7:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=421s) that's what we solve at in AI. We are [7:01](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=421s) that's what we solve at in AI. We are looking for teams building their own [7:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=422s) looking for teams building their own [7:02](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=422s) looking for teams building their own inference stacks who treat security as a [7:05](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=425s) inference stacks who treat security as a [7:05](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=425s) inference stacks who treat security as a foundational design decision. I hope [7:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=427s) foundational design decision. I hope [7:07](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=427s) foundational design decision. I hope this was helpful for you. Thank you so [7:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=429s) this was helpful for you. Thank you so [7:09](https://www.youtube.com/watch?v=1NWnSwCB4NM&t=429s) this was helpful for you. Thank you so much for listening.