You Can't Tell What an Agent Skill Will Do by Reading It - Roey Ben Chaim, Zenity

An official Azure diagnostic checked the identity and configuration of a cloud environment. An information stealer might also look for sensitive information on that machine. If a security reviewer saw only the access event, how could they distinguish legitimate troubleshooting from credential theft? Roey Ben Chaim, a staff engineer at Zenity and a MATS research fellow, opens with this ambiguity. Even knowing which product supplied the skill leaves an investigator with the job of explaining what happened on a particular run.

A skill gives an agent a capability on demand. Someone else supplies the instructions, resources and sometimes executable code. That leaves anyone adopting a skill with two questions. What work did it cause the machine to do? Does that work make sense for the task the user gave it? Even a trace that answers the first question can leave the second open.

A skill can depend on packages, instructions or services outside its initial file. A reviewer examining the installed document sees what the author provided at that moment. The agent may encounter different material when it uses the skill. A thorough investigation needs to identify what influenced the run, including dependencies the user never selected directly.

Ben Chaim cites work by Trail of Bits in which an automated reviewer accepted a misleading explanation for a software change. The same material offered both the proposed change and the claim that somebody had authorized it. A reviewer needs an independent source for that assertion. Reading more words from the same source cannot establish who approved a change.

Zenity borrowed detonation from malware research. The team put an agent and a skill in a sandbox, supplied a task prompt, let the agent work for minutes or hours, then examined its trace. The first version ran into a mundane problem. Many skills expected a command-line tool, a service or an existing project. A Notion skill in an empty sandbox has no Notion service to talk to. A run that stops there tells you little about what the skill does once it connects.

The team added a world builder that reads the skill and prepares the environment it expects. That addressed one reason a run could stall. The next was the assignment itself. A presentation skill needs a deck to make, a deployment skill needs a service to ship, and a brainstorming skill needs an idea to explore.

Even asking the agent to ship a service leaves choices. Configuring the service, deploying it and recovering it from a snapshot exercise different parts of the skill. A successful deployment says little about the recovery path. The team added an operator that reads the skill, identifies routes worth exercising and supplies prompts for them. Coverage depends on which routes the operator recognizes and which ones the agent follows.

A model can also stop in the middle of a test. Some agents declined to continue when a step looked unsafe. That is an observation worth retaining alongside the task and environment. It narrows what the team can conclude from that run. Ben Chaim notes that legitimate tasks can trigger a safeguard, too. Treating every refusal as proof of malice would misclassify some skills; treating an unfinished test as evidence of safety would hide what the researchers had yet to observe.

Suppose the agent trace shows a single installation. Can an investigator use that entry to say which files were touched? Some of the work happens outside the model’s conversation. Processes started during installation may have effects that the agent never describes. The model can report that its request succeeded without knowing the consequences for the rest of the machine.

The chamber kept the agent trace and added three kinds of evidence. Process creation events showed what the installer launched. Filesystem events showed what those processes touched. Network packet capture recorded their traffic. Together the records let an investigator work back from a file access or an outbound connection to the installation that set it in motion.

In a harmless proof of concept by Zach Corman, the skill a user chose drew in another component. The first file gave no clear account of the behavior that followed. The chamber’s records let the researchers attribute the later action to an indirect dependency. A single successful test established a concrete possibility. It did not establish what every installation would do.

Across thousands of skills examined, Ben Chaim reported that the team identified 100 malicious skills and that those skills were taken down from skills.sh. That count describes findings in the collection the team examined. It leaves open how many malicious paths the chamber missed.

In another case, a skill appeared to use a Polymarket dependency. Its package identity differed slightly from the expected one, and the researchers traced it to an unofficial source. Inspection showed that the substituted software collected sensitive information. The meaningful comparison was between the dependency a user believed they had chosen and the one the installer actually supplied.

An uninstall is an unusually good test of what a reviewer knows about the environment. Some skills returned after users thought they had removed them. In a separate case, an unofficial skill creator quietly replaced the familiar implementation. A product interface could appear unchanged while a different component handled the user’s next request. An inventory from before installation, followed by a fresh look after removal, gives the investigator something concrete to compare.

The installed package can also stay unchanged while the material it relies on changes. A remote source can be updated or change hands. Some skills can run in response to later activity, including work unrelated to the original assignment. A good review therefore has a date and a scope: which version, which dependencies, under what conditions, and which behavior did someone actually observe?

Paperclip, an open-source project for orchestrating agents, gave the researchers a comparison case. They found an imitation of the project that looked familiar enough to borrow its reputation. Activity advertised as routine logging led somewhere outside the genuine project. Examining the destination gave the team a reason to investigate beyond the feature name.

In this case, the researchers established that sensitive authentication data left the machine and reached a service controlled by the attacker. They also observed that part of the activity was short-lived, which made the recording of the run especially important. Recall the genuine Azure diagnostic from the start of the talk. The destination and purpose of the transfer gave the team a much stronger basis for calling this credential theft.

The investigation linked the same infrastructure to software distributed through the Python ecosystem and then to a set of agent skills. Some packages disappeared quickly. The related skills had accumulated what Ben Chaim reported as 1.7 million total downloads. This is a distribution count. It cannot tell a responder how many installations completed, how many machines ran the behavior or how many people lost credentials.

Finding the same infrastructure across distribution channels changed the response. Removing one copy of a package could leave users exposed through another source. Ben Chaim reports that after the team notified the relevant platforms, the remaining campaign material was removed within 12 hours.

A skill is one way outside material enters an agent’s context. A website, a database row, a package or even a YouTube comment can also supply text the model reads while deciding what to do. Any of those sources can influence a task once the agent has read it.

Ben Chaim’s advice starts with established software dependency practices. Identify who supplies a skill and what it loads. Limit access to what its task requires. Keep enough runtime evidence to relate a sensitive action to the process that performed it and to the destination of any transfer. Someone who can answer those questions can explain why the Azure diagnostic and the Paperclip lookalike deserved different responses.