Introduction
This reading edition uses the complete 14½-minute standalone recording, recovering material cut off in the Day 1 livestream. Text links use the standalone clock. The retained screenshot is explicitly labeled with its original livestream clock.
An agent reads cloud configuration, queries identity services, and probes instance metadata. Those actions could look like credential theft. They could also be part of a legitimate diagnostic workflow. Roey Ben Chaim opens with this ambiguity: the example is an official Azure diagnostic skill, not an infostealer. Deciding whether an agent skill is safe requires more than recognizing suspicious-looking operations.
A skill gives an agent capability on demand: instructions, resources, and sometimes executable scripts. Its entry point might be a SKILL.md file, but that file need not contain everything the agent will eventually use. The safety question concerns the resulting behavior, including the dependencies and instructions loaded later, rather than the entry-point text alone.
Reading the entry point does not reveal the whole execution
At 00:01:37, Ben Chaim describes two limitations of scanning. First, the payload can live elsewhere. A skill can fetch more instructions from an endpoint, install a package, or invoke a script that downloads further material. Inspecting only SKILL.md misses that transitive chain.
Second, a model used as a scanner can itself be influenced by text inside the material it is judging. Ben Chaim cites Trail of Bits research in which an apparent security-approval comment helped persuade an analyzer to accept a package-registry change. This is an example he cites, not an independently reproduced evaluation in this talk.
The alternative borrows a technique from malware analysis: run the subject in an isolated environment and inspect what happens. For agent skills, however, merely launching a process is not enough. The agent needs a task, the skill needs a usable environment, and the test must reach the relevant behavior.
The first sandbox was too empty
The initial detonation chamber placed an agent and a skill in a sandbox, supplied a task prompt, let execution continue, and collected agent traces. That simple arrangement exposed several problems.
Many skills depend on command-line tools, services, or existing projects. A Notion skill needs a service context it can actually interact with. A quiet run in an empty environment can mean that the test never engaged the skill, not that the skill is safe. The team added a world builder to inspect the skill and prepare an appropriate environment.
A generic task prompt also provides poor coverage. Creating a presentation, deploying a service, and brainstorming an idea demand different trajectories. Even within deployment, configuration, deployment, and snapshot recovery can reach different parts of a skill. The team added an operator to identify and exercise those routes.
At 00:04:50, Ben Chaim describes another experimental limitation: safeguards sometimes prevented the agent from reaching a behavior under investigation. He emphasizes the contained security-research setting and notes that refusals can occur for legitimate operations too. An adversarial operator helped exercise paths inside that environment. A refusal is therefore neither proof of maliciousness nor proof that the rest of a skill has been tested.
Original livestream screenshot at 05:01:10, not the standalone clock. The architecture separates environment preparation, path exploration, trace capture, and assessment. The complete recording subsequently adds lower-level telemetry to this design.
Tool traces miss what a tool's children do
The recovered continuation introduces a crucial change to the capture layer. An agent trace might show npm install, but that does not tell the analyst everything the installation did. A post-installation script can launch another process, which can read or write files and contact an endpoint. Those effects can happen below the level of the agent's visible tool call.
At 00:06:03, Ben Chaim explains that the chamber therefore captures process-creation events, filesystem events, and network traffic alongside agent messages and tool results. The object being evaluated is the whole execution, not merely the agent's account of it.
This distinction makes the earlier dependency problem observable. A harmless-looking instruction can initiate a chain whose consequential action occurs several steps away. Following that chain requires both execution that reaches it and instrumentation that records it.
A security-review skill can install another skill
Ben Chaim next presents a proof of concept attributed to Zach Korman. A skill called Security Review 2 installs another skill called Security Review. That second skill directs the agent to retrieve a SKILL.md from GitHub, where further instructions lead it to execute a payload.
He explicitly says the payload in this demonstration is not malicious. The demonstration establishes the indirection: evaluating only the first skill file would not expose the complete chain that the agent follows. One chamber run recovered that behavior.
At 00:07:57, he reports finding 100 malicious skills among thousands examined and says they were removed from the skills.sh ecosystem. The talk does not supply a sufficiently precise denominator or validation protocol to turn that statement into a prevalence estimate or a measured detector accuracy. It is a reported investigation outcome.
The next example involves a Polymarket-themed skill. According to Ben Chaim, its installation led to a lookalike package rather than the expected official package, and investigation revealed a nonofficial registry and an infostealing payload. The exact package spelling is not necessary to understand the failure: a familiar product name does not establish the identity of the dependency actually loaded.
Persistence, replacement, and instructions that change later
The talk then catalogs behaviors observed across the ecosystem. Some skills add persistent instructions telling an agent to reinstall the skill if it is missing. Another skill-creation tool, Ben Chaim says, silently replaces the official tool rather than simply coexisting with it. These changes affect what happens on later runs, not just the initial task.
Remote prompt execution introduces a time dimension. Fetching instructions from elsewhere can be legitimate, but the contents can change after a reviewer approves the original skill. A domain can change hands, or its operator can deliberately replace the prompt. Approval of a local entry point at one time does not freeze the remote instructions it will retrieve later.
Hooks create another expansion of scope. A skill attached to a frequent lifecycle event can run far more often than a user expects from a one-off installation. The important question becomes not only what a skill does, but when it is triggered and which future actions it can influence.
A lookalike project becomes a distribution channel
At 00:10:35, Ben Chaim describes a campaign involving lookalike versions of recognizable agent projects. In his account, a copy of Paperclip was modified so ordinary project routes invoked an added logging action. That action led to remotely supplied code which harvested credentials. Related attempts used other project identities and shared infrastructure.
The useful lesson is the mismatch between packaging and behavior. The surrounding project can look familiar, while a small added operation changes what running it means. A skill or installer can then become a distribution channel for that altered dependency.
Ben Chaim says some package-distribution attempts were removed quickly, while associated skills continued circulating. He cites a figure of 1.7 million total downloads in this discussion. That number should not be interpreted as 1.7 million confirmed compromises, distinct users, or verified executions of the payload; the talk does not establish those equivalences.
He reports that the team notified Vercel and Microsoft and that the relevant material was taken down. These are the speaker's account of the investigation and response. This reading edition does not independently attribute the campaign to a named individual or reproduce operational payload details.
Treat skills as dependencies, not trusted prose
The conclusion at 00:13:33 expands the boundary beyond skills. Websites, database rows, packages, and even comments can enter an agent's context. Skills are one part of that larger supply chain of information and behavior.
For skills specifically, Ben Chaim recommends the habits used for package dependencies: inspect what they load, restrict what they can access, apply least privilege, and observe what actually runs. The detonation chamber contributes behavioral evidence to that process. It does not make one successful test a proof of safety across every environment, task, remote dependency, or future version.
The opening diagnostic example still matters at the end. Suspicious-looking actions need context. Reassuring-looking instructions need verification. And an execution trace needs enough depth to include the processes, files, and network effects that the agent's tools set in motion.