# We Vetted 2,000 AI Skills Before They Reached Developers

Lucas Palma

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/iKQ78wyJEXU-we-vetted-2-000-ai-skills-before) · [Video](https://www.youtube.com/watch?v=iKQ78wyJEXU)

## We Vetted 2,000 AI Skills Before They Reached Developers

Nubank’s skill review pipeline treats shared agent instructions as supply chain dependencies, combining local checks, mandatory CI, contextual review and execution controls.

### What enters the workflow before code is generated?

[0:23](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=23s)

A developer adopts an AI skill to make coding faster. It looks like configuration, but it can change what the agent does before anyone reviews the resulting code. Where should security review begin? Lucas Palma, introducing himself as Product Security Manager at Nubank, describes a system for reviewing skills before developers receive them. His team’s remit includes helping engineers and product managers make products safer; shared AI instructions extend that responsibility into the development workflow itself.

**Skills belong in the software supply chain because they influence behavior.** Reviewing generated code alone leaves the instructions that shaped it outside the review boundary. The familiar dependencies—packages, containers and models—remain, but skills, plugins, MCP servers and agent rules now sit alongside them. The security boundary therefore has to cover the whole workflow.

![The story slide presents four connected columns: the shift, the risk, the system, and the lesson to protect the whole workflow.](https://ai.engineer/api/talk-essay-media/3e855cc31740dfdac568ed556368df8cb25a7dc15a53b5c2e9c2fd2de96626f0)

AI skills look like configuration but behave like supply chain dependencies.

### Shared instructions transfer influence

[2:46](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=166s)

In the workflow Palma describes, a developer uses an AI tool to produce an output, usually code. A skill sits inside that tool, bundling instructions and context that guide a model or agent toward a capability. When someone shares a skill, they also share influence: its author can shape code generated for another developer, in another environment. That transfer is useful, but it is also the reason the skill needs review.

![What is an AI skill? slide shows Developer, AI Tool, and System Output, with an AI Skill beneath the tool and two panels explaining core capability and impact.](https://ai.engineer/api/talk-essay-media/c1ecd7393e66c5e48bfe3fd1c39fb17da7e36fcedc9543b2938cdfc2c1163b7c)

AI skills bundle instructions and context into reusable capabilities that shape developer output.

In regulated finance, the competing requirements are concrete. Developers want faster coding, richer context and less repetitive work. Security needs auditability, protected credentials and safe defaults. A review process has to preserve the productivity benefit while checking what authority and behavior those reusable instructions introduce.

### From a helpful instruction to a credential leak

[4:38](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=278s)

The credential example follows a short chain. A skill tells the agent to retrieve a token. The agent starts using the token as a hard-coded value, and that value can reach logs, creating a later data leak. The problem begins with an instruction about how to accomplish a task, before it becomes a problem in generated code or operational output.

Other risks depend on where the shared instructions run:

- **Shell commands:** An author’s command instructions are reused in another developer’s shell, where they can modify files or perform dangerous operations.
- **Excessive permissions:** A skill can receive more authority than its task needs, increasing the consequences of a mistake.
- **Accidental errors:** Harm does not require malicious intent. Even a typo can become dangerous in the wrong execution context.

Review therefore needs to examine both the instruction and the environment in which it could take effect.

### Make distribution the review boundary

[5:47](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=347s)

Nubank’s internal plugin marketplace provides a canonical place to discover and share skills. Authors contribute through pull requests, and the plugins distributed through the marketplace can include skills. That creates a practical intervention point: review a contribution before it becomes available to everyone else. Palma calls the CI review tool **Skill Vector**. It assesses the submitted skill, classifies risks and requests remediation before marketplace distribution.

The pipeline supports fast author feedback without depending on authors to enforce the policy themselves:

1. **Scan locally.** Developers can check a new or changed skill and iterate before submitting it.
2. **Rescan in CI.** Submission triggers another scan because the system cannot assume that the author ran the local check or used an up-to-date scanner.
3. **Check known patterns deterministically.** Regular expressions and other straightforward checks identify risks that do not require much interpretation.
4. **Review context with an LLM.** Contextual analysis follows where understanding the instructions or behavior requires more than pattern matching.

The two analysis methods serve different purposes. Palma points to variability in LLM judgments, including the effect of temperature, as a reason to retain deterministic checks rather than hand the entire decision to a model.

Findings return to the originating pull request, so the author can see what needs to change where they are already working. The pipeline also emits **SARIF**, allowing security tools to consume the results and bring them into the vulnerability management program. Developer feedback and security tracking become two destinations for the same review findings.

Severity and policy determine what happens next: a skill may be allowed, require remediation or be blocked before distribution. The complete path runs from local scanning to the pull request, deterministic analysis, LLM review, PR feedback and SARIF, then the policy decision. Local checks shorten the feedback loop; CI makes the review part of publication.

### Inspect behavior, authority and data handling

[9:18](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=558s)

Skill Vector’s review surface goes beyond finding suspicious command strings. Palma gives a non-exhaustive set of targets:

| Review surface | What the scanner examines |
| --- | --- |
| Instructions and behavior | Unsafe guidance and drift in agent behavior |
| Shell and filesystem | Destructive commands and inappropriate file modifications |
| Credentials and data | Credential requests and unintended exposure |
| Permissions and integrations | Overbroad authority and risky MCP usage |

The credential question, for example, is not simply whether a skill mentions a token. It is how the skill asks for credentials and what happens to the data afterward. Similarly, a file operation needs to be assessed as an action in context, not merely as text appearing in a skill.

### Count findings separately from affected skills

[9:58](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=598s)

Palma reports a presentation baseline of over 2,000 skills scanned, noting that the total had already grown. He reports more than 1,500 risk findings, not more than 1,500 affected skills: a single skill can contain several risks. He says 1,000 of those risks were probably remediated shortly afterward. The qualification matters: these are reported operational counts, not a measured detection accuracy or a confirmed remediation rate.

Some especially risky submissions were blocked before they reached the marketplace. The rollout also included a historical scan of skills created before Skill Vector existed. Those older findings entered the vulnerability management program for remediation, extending the effort beyond controlling new submissions to addressing the existing catalog.

### Make findings useful in their actual context

[11:17](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=677s)

The useful combination was deterministic scanning for known patterns, LLM review for behavior, checks around destructive commands and credentials, and delivery through SARIF and PR comments. The harder work was deciding which signals deserved attention. Comments initially received similar risk treatment even though their risk depended on context, and weak signals without enough context caused more trouble than they helped resolve.

Prompt instructions asking for confirmation also needed closer scrutiny. So did warnings that seemed harmless when a skill ran locally but could become consequential in production. A finding needed clear remediation guidance to be useful to the author. Coverage posed another operational problem: teams could create additional marketplaces. The security process therefore needed to discover those distribution channels proactively and bring Skill Vector into them too.

![What worked, and what was noisy slide contrasts scanners, credential checks, LLM review, SARIF visibility, and PR feedback with weak signals, unenforceable confirmation prompts, unclear remediation, and other marketplaces.](https://ai.engineer/api/talk-essay-media/3bbabd8ab3c83f6549e7a541537ba25c37220b26d926144471d72ca7e80dcb47)

Security review findings distinguish useful signals from warnings that need context.

### Put human approval at the execution boundary

[12:56](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=776s)

**An instruction to ask for confirmation does not establish human approval.** Palma describes the failure mode: an AI may ask itself for confirmation, or accept another AI’s confirmation, and treat the requirement as satisfied. The person reading the skill believes a human is in the loop, while the agent sees a completed confirmation step.

A minimal skill instruction illustrates the ambiguity:

```markdown
Before modifying files, ask for confirmation.
```

The sentence requests a behavior, but does not itself identify a human approver or enforce a pause in execution. The review needs to follow the action to the tool that will execute it: does that tool actually pass through a human approval gate? Palma points to approval gates and hooks as controls around execution. The relevant distinction is between asking the agent to obtain approval and requiring approval before an action can run.

Skills are only one component of a plugin marketplace, so the surrounding controls can be reused: review contributions before publication, let developers run checks locally, enforce those checks again in CI, and combine deterministic analysis with contextual LLM review. For dangerous actions, that review must connect to enforcement when the action occurs. Prompt wording alone cannot supply the execution boundary.

![Patterns I would reuse slide lists six checked practices, ending with moving dangerous actions from prompt nudges to tool-level enforcement.](https://ai.engineer/api/talk-essay-media/054f09418714985bffeb89ef3a2aeeebbafbc6b63595ba514316677036b99bc2)

Reusable practices include marketplace review, local and CI checks, and tool-level enforcement.

### Bring third-party components through the same gate

[14:23](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=863s)

The broader scope includes plugins, MCP servers, rules and hooks, with different risks for each. Palma names MCP Vector and rules checks as related controls already in use, while describing further work on gates that vary with policy and marketplace context. The direction also includes tool-level enforcement, audit logs and trusted gateways, extending control from what enters the catalog to what happens during execution.

A trusted marketplace supplies the canonical route for scanning and sharing, including components written outside the organization. Downloading a third-party skill or plugin should not bypass the review process: it should enter the marketplace for scanning before use and wider distribution. That makes the adoption path part of the security design. A component is reviewed because it will influence the workflow, regardless of who wrote it.

## Resources

- [Agent Skills specification](https://agentskills.io/specification) — Defines skill directories, instruction files, executable scripts, supporting resources, and optional tool declarations.
- [SARIF support in GitHub code scanning](https://docs.github.com/en/code-security/reference/code-scanning/sarif-files/sarif-support) — Explains how a security platform consumes SARIF results and which fields and format constraints integrations must satisfy.
- [Claude Code hooks reference](https://code.claude.com/docs/en/hooks) — Documents execution hooks and permission decisions, including examples that block destructive shell commands before execution.

## Slides

### Who am I

Lucas Palma, known as LP, is a Product Security Manager at Nubank. He supports teams in making products safer and brings experience in financial services, engineering, security and innovation.

[0:27](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=27s) [Lucas Palma introduces his role at Nubank and his background in financial services, engineering, security, and innovation.](https://assets.ai.engineer/sites/aiecode2025/sha256/5d6e35b21b3c7bf47cbc90b58f64f722de18e0cd0b6f269cdacf2d2e29b5dd85.webp)

Who am I

LP (Lucas Palma)

Product Security Manager at Nubank

Financial Services, Engineering, Security, Innovation

AI Engineer World’s Fair, July 2026

2

### The story

AI skills entered the developer workflow looking like configuration but behaving like supply chain dependencies. The team built a security review before skills reached developers, aiming to protect the whole workflow.

[1:02](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=62s) [Four stages summarize the shift to AI skills, supply-chain risk, security review, and protecting the whole workflow.](https://assets.ai.engineer/sites/aiecode2025/sha256/45c21108e639760e80848f426ba67bd0216f1fa4f6830cfcb592f5270467aa67.webp)

The story

THE SHIFT
AI skills became part of the developer workflow.

THE RISK
They looked like configuration, but behaved like supply chain dependencies.

THE SYSTEM
We built security review before skills reached developers.

THE LESSON
Protect the whole workflow, not only the generated code.

### The new supply chain does not always look like code

The AI-era supply chain includes traditional dependencies alongside skills, plugins, MCP servers, agent rules, and instructions that shape developer workflows.

[2:10](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=130s) [A comparison lists traditional software dependencies and AI-era additions such as skills, plugins, MCP servers, and agent rules.](https://assets.ai.engineer/sites/aiecode2025/sha256/92f1c0fd608b957a71fa07fb33a8511f4acea0304ceb8ff6bc1908bd82a2b2ce.webp)

The new supply chain does not always look like code

TRADITIONAL SUPPLY CHAIN
• Packages
• Containers
• CI/CD dependencies
• Infrastructure modules

AI-ERA SUPPLY CHAIN
• Traditional supply chain
• Skills
• Plugins
• MCP servers
• Agent rules
• Tool manifests
• Prompt instructions
• Local automation scripts

### What is an AI skill?

An AI skill supplies guidance to the tool between developer and output. Shared skills let one person’s instructions shape another developer’s generated code.

[2:40](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=160s) [A diagram places an AI skill within the developer-to-tool-to-output workflow; panels describe its capabilities and impact.](https://assets.ai.engineer/sites/aiecode2025/sha256/ac2ef6d420f1920ec31c8b4399e27e525db31ddd96fdb0ec4594d4000cdfd662.webp)

What is an AI skill?

Developer → AI Tool → System Output
AI Skill → AI Tool

CORE CAPABILITY (WHAT A SKILL IS)
• A packaged capability for a model or agent
• Bundles instructions, context, workflows, and supporting files
• May describe tools, commands, or how to interact with systems

IMPACT & SCALE (WHAT CHANGES / HOW FAR REACHES)
• Shapes how developers write, review, or modify code
• Reusable and shareable across many people and projects
• Turns guidance into concrete output

### In regulated environments, AI adoption needs speed and control

Developers want faster coding and better context, while regulated environments also demand auditability and credential safety. The slide calls for controls that run transparently within the existing workflow.

[3:45](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=225s) [A balance diagram contrasts developer velocity with customer and production safety, alongside lists of each side's needs.](https://assets.ai.engineer/sites/aiecode2025/sha256/f9b20d7a9b63c753b5d2212f1ca15dbd278dea7cc2e3161e15c3f327580bd407.webp)

In regulated environments, AI adoption needs speed and control

DEVELOPERS NEED
• Faster coding
• Better context
• Less repetitive work
• Reusable workflows

Developer Velocity
VS
Customer & Production Safety

VELOCITY
SAFETY

Regulated Balance: Controls must run transparently within the existing workflow.

TRUST & SAFETY NEEDS
• Customer protection
• Auditability
• Credential safety
• Production safety
• Clear ownership
• Safety by default

### Harmless-looking skills can create production-impacting behavior

A skill can retrieve a token into model context, where it may reach logs or vendor infrastructure. Shell commands and excessive permissions can also affect production; host-gated tools offer a safer design.

[4:27](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=267s) [Three columns describe credential exposure, destructive shell commands, and excessive permissions as potential skill risks.](https://assets.ai.engineer/sites/aiecode2025/sha256/a883af6c4e06de8f8214f46ebbac2131170e93a109f2f65f39c5db419fb2701d.webp)

Harmless-looking skills can create production-impacting behavior

SECRETS IN CONTEXT
Example 1 — Credential exposure
• A skill tells the agent to retrieve a local token
• The token output enters model context
• The credential may reach logs or vendor/proxy infrastructure
• The skill also gives the agent access to a service credential

MODIFICATION VIA SHELL
Example 2 — Destructive commands
• A skill includes shell commands that modify production data references
• Prompt-level confirmation says "ask before running"
• But the model can auto-confirm, skip the pause, or infer consent

EXCESSIVE PERMISSIONS
Example 3 — Over-broad access
• A skill can manipulate tables, grants, or registry entries
• A typo or stale source can create real production impact
• Safer design moves operations behind host-gated tools/MCPs

### Review before distribution, not after adoption

The internal marketplace makes skills easy to discover and scale. Skill Vetter adds a review gate after the pull request and before marketplace distribution, classifying risks before developers adopt a skill.

[5:45](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=345s) [Marketplace risks and a workflow placing Skill Vetter review before distribution.](https://assets.ai.engineer/sites/aiecode2025/sha256/3cf4681a41b0aa9bd4bab63ad2b4e07c635cdbad26688df153c968be37b17ee4.webp)

Review before distribution, not after adoption

THE MARKETPLACE PROBLEM

• Internal marketplace makes skills discoverable
• Discoverability creates rapid organizational scale
• Scale turns small mistakes into widely repeated behavior
• Therefore, review must happen before skills ever reach the marketplace

"The marketplace boundary became our security boundary."

VETTING PIPELINE WORKFLOW

01 Skill Creator
Author packages the capability

02 Pull Request
Code submission triggers automation

03 Skill Vetter GATE
Security review & risk classification

04 Marketplace
Approved & discoverable directory

05 Developers
Safe adoption in developer workflows

### What we scanned

The scan checks what a skill instructs an agent to do, the commands it runs, the data it might expose, and the permissions and approvals around tool use. These examples are not exhaustive.

[9:15](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=555s) [Scan categories cover instructions, commands, data and credentials, and tools.](https://assets.ai.engineer/sites/aiecode2025/sha256/bd0bf8f7bfa403bfb526d503f2037ec48da7003c6c5ffdb20e93eabb64e343ad.webp)

What we scanned

INSTRUCTIONS
• Unsafe instructions
• Review manipulation
• Agent behavior drift
• Prompt-level approvals pretending to be controls

COMMANDS
• Destructive shell commands
• Dynamic command execution
• Production-impacting CLI usage
• Host/file modifications

DATA & CREDENTIALS
• Credential requests
• Credential files
• Sensitive paths
• Sensitive data exposure
• External calls or exfiltration paths

TOOLS
• Risky MCP/tool use
• Over-broad permissions
• Shared service principals
• Lack of host-level approval

### What we found after reviewing 2,000+ skills

Across 2,000+ skills, the review found 1.6K potential risks; a single skill could have multiple risks. A retroactive pass added ~450 findings to a prioritized remediation queue.

[9:56](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=596s) [Results show over 2,000 skills analyzed, 1,600 potential risks, approximately 1,000 issues remediated and 90 priority review cases.](https://assets.ai.engineer/sites/aiecode2025/sha256/968c26c2198439a5fe03aadce7512e90560b7cd51370804ef8bc141e0003e05c.webp)

What we found after reviewing 2,000+ skills

SCOPE
2K+
Skills Analyzed

RISKS
1.6K
Potential Risks Found

RESOLVED
~1K
Issues Remediated

RED FLAGS
~90
Priority Review Cases

ADDITIONAL INSIGHTS & HISTORICAL CONTEXT
• ~650 previously unvetted skills
• ~220 skills escalated beyond initial deterministic checks

The retroactive pass emitted ~450 findings across previously unvetted skills, turning a blind backlog into a prioritized remediation queue.

### What worked, and what was noisy

Deterministic scanners and LLM review worked well. Noise came from treating commands equally and warnings that missed local versus production impact; findings also lacked clear remediation guidance.

[11:14](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=674s) [High-signal checks and feedback are contrasted with warnings and findings needing more context.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b66c6d89961b7d40f7d318ba11c5a87a729570f92b32ef8f5640013c5bf87b9.webp)

What worked, and what was noisy

HIGH SIGNAL (WHAT WORKED)
• Deterministic scanners for known risky patterns
• CLI/destructive command detection
• Credential and sensitive-path checks
• LLM review for holistic behavior
• SARIF/code scanning for visibility
• PR comments with actionable feedback

NOISE / NEEDS CONTEXT
• Treating all command usage as equally risky
• Weak signals without context
• Prompt-level “ask for confirmation” that looked safe but was not enforceable
• Warnings that did not distinguish local harmless behavior from production-impacting behavior
• Findings without clear remediation guidance
• Other marketplaces being created

### Example: Prompt-level approval is not a security boundary

An agent may confirm its own request and proceed. A host approval gate puts human confirmation between the tool request and execution.

[12:53](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=773s) [Prompt-level self-confirmation is compared with a host approval gate requiring human confirmation.](https://assets.ai.engineer/sites/aiecode2025/sha256/d80023f1b6abdb4bddef4e817af1d779ed442a714d7d49d5dd00437fe4d7dd3f.webp)

Example: Prompt-level approval is not a security boundary

CONCEPTUAL FLOW COMPARISON

BAD:
Agent asks itself → Agent proceeds auto-confirming

GOOD:
Agent requests tool call → Host approval gate → Human confirms → Tool executes

THE LESSON & RESOLUTION

PROBLEM
• Skill says: “Ask before running this command”
• But the agent may auto-confirm or treat prior context as consent
• The model is simultaneously the actor and the approver

BETTER PATTERN
• Move dangerous operations behind tool-level approval
• Use MCP or host-gated tools where the agent cannot bypass the prompt
• Leverage PreToolUse hooks or equivalent infrastructure controls
• Require explicit human approval outside the model's reasoning loop

### Patterns I would reuse

Review skills and plugins before distribution, run the same checks locally and in CI, and combine deterministic checks with LLM review. Enforce dangerous actions at the tool level.

[13:34](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=814s) [Six practices for reviewing and enforcing security checks on skills and plugins.](https://assets.ai.engineer/sites/aiecode2025/sha256/d12c3e2e2cb18f2b0999092d0f685b02a12777dc2efe32baa311e5c2317c4764.webp)

Patterns I would reuse

PRACTICAL TAKEAWAYS & WORKFLOW SECURITY BEST PRACTICES

Treat skills and plugins as supply chain dependencies.
Review before marketplace distribution.
Let developers run checks locally.
Enforce the same checks in CI.
Use deterministic checks for cheap signal, LLM review for context.
Move dangerous actions from prompt nudges to tool-level enforcement.

### Where this goes next

The roadmap extends vetting beyond skills, proposes CI and marketplace enforcement where confidence is high, and calls for tool-level controls and a trusted AI marketplace.

[14:21](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=861s) [Roadmap covering agent supply chain security, policy gates, tool-level enforcement, and a trusted AI marketplace.](https://assets.ai.engineer/sites/aiecode2025/sha256/cb05fba93ce2ffd3f8619f8adef42d6f7f59de066531e8fa31ef2fba8c11dce0.webp)

Where this goes next

STRATEGIC ROADMAP & FUTURE INITIATIVES

01 Agent supply chain security
Skills, plugins, MCP servers, rules, hooks, commands, agents, and manifests.

02 Policy-backed gates
Move from advisory findings to CI and marketplace enforcement where confidence is high.

03 Tool-level enforcement
Replace prompt-only controls with approvals, scoped permissions, trusted gateways, and audit logs.

04 Trusted AI marketplace
Make every shared AI component owned, versioned, reviewed, and safe to consume.

### Make the safe path the easiest path

Palma argues that AI adoption scales when safety is built into the developer workflow. He invites discussion of skill vetting, MCP governance, and AI-assisted coding security.

[15:43](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=943s) [Closing slide advocates building safety into the developer workflow and includes a contact panel and QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/afbcf7c26dfc09b2c7e58741dbf320afcc0d142a53a4c129562661dcc9849ab4.webp)

Make the safe path the easiest path

AI adoption scales when safety is built into the developer workflow, not added as a separate approval process.

QUESTIONS & DISCUSSION

CONTACT
Lucas Palma
Nubank
linkedin.com/in/lucaspalma

Happy to discuss skill vetting, MCP governance, AppSec automation, AI-assisted coding security and AI innovation in general.

"Protect the workflow, not only the generated code."

### AI Engineer World’s Fair

[16:16](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=976s) [AI Engineer World’s Fair logo in white on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/61c92ae62412f3721bb447169de80cdcd0e9c6bcea7c338c35979936d95f4d12.webp)

AI Engineer
World’s Fair

HTTPS://AI.ENGINEER

## Full transcript

[0:00](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=0s) [upbeat music] Hello, everyone. Good afternoon. Today, I'm going to talk about how we vetted two thousand AI skills before they reached the developers.

[0:23](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=23s) But before I-- before that, I'm Lucas Palma, but many people call me LP. I'm the Product Security Manager at Nubank. The product security structures, uh, structure that within security, looking upon how we make code safe and supporting engineers, product managers, and everybody into making our products safer.

[0:48](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=48s) I have, uh, over a decade of experience in financial services, engineering background, also a lot of years working here at security and a close relationship with the part that I love, which is innovation.

[1:04](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=64s) So before beginning, I believe I wanna bring to you, uh, why are we here. So one thing that's important for all of you to understand is that the--

[1:18](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=78s) now that we are using AI everywhere, even though-- e-even with, uh, coding, one thing that, uh, is important is that the AI skills are being part of the developer workflow.

[1:32](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=92s) And this, this might bring some risks because although they look like configuration, they behave like supply chain dependencies, like, uh, for example, libraries and others. So what we made here was to build a security review system in order to check if these skills were safe or not to be used before deploying them.

[1:58](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=118s) So the lesson that I want to bring you here by the end of this presentation is that we should be protecting the whole workflow, not only the code that's being generated.

[2:10](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=130s) All right. So what I mean about the supply chain part is that, uh, traditionally, the supply chain has, uh, packages, containers, models, and so on. But now in the AI era, it doesn't have only that.

[2:26](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=146s) It still have the traditional part, but it will-- it also includes skills, plugins, MCP servers, agent rules, and much more things to be acting as supply chain.

[2:42](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=162s) And where AI skill fits into this, uh,

[2:46](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=166s) I believe that before going to that, it's important for everybody to be on the same page on what is an AI skill. So an AI skill has, uh, there is-- the-- normally, the developer is using AI tools in order to generate an output, which will be code most of the case.

[3:06](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=186s) And within this AI tool, there are, are a bunch of things that can be embedded. One of them are the AI skill. So with this skill, we can have, uh, a capability to a model or to an agent, uh, bundling some instructions, some context in order to have better guidance over what it can be done.

[3:28](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=208s) But there is also an impact over that because somebody can create their own skill and share with others. So when we do that, this first person is guiding over the code that's being generated by the other person, and then c- that can be dangerous.

[3:47](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=227s) And since we are here talking in the AI in Finance track, it's also important for us to understand that we are in a regulated environment. So from one side, there are, are developers wanting better, faster coding, more context to have less repetitive work.

[4:09](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=249s) But on-- but from the other side, even more because of the regulate part, we need to be aware of the audit- auditability, of looking upon credentials, safety by default, and many other security aspects.

[4:26](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=266s) And keeping that balance is hard, right? So some people might say, like, "Are AI skills dangerous?" So

[4:38](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=278s) I brought here a few examples of what do I mean by AI skills being dangerous. So first, uh, one thing that can happen is that when people are describing what a skill can or cannot do, it can, it can ask for it to retrieve a token or something, and it will begin using that token, uh, hard-coded, which

[5:01](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=301s) will go to logs and so on, and it can generate a data leak in the future. Another thing that can happen is also the person to, uh, instruct the AI to use shell commands, and then this skill will be used by another person.

[5:18](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=318s) And when they use on their shell, a lot of dangerous things that, that can happen and a lot of files being modified and so on. And there is also permissions.

[5:28](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=328s) So depending on how the skill was configured, it might have excessive permissions, much more than what was needed. And even a typo can make some dangerous stuff depending on who is using that such skill.

[5:47](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=347s) So first thing first, what we did, uh, initially is that how do we share skills among ourselves, uh, how the engineers will be sharing these skills. So Uh, we went through the marketplace solution, so the skills are being canonically shared among, uh, marketplace with the plugins, including the skills among them.

[6:13](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=373s) So it's a internal marketplace where people can discover new skills, and that's our boundary where we are trying to make it safer. So what happens is that when someone creates an skill, it's, uh, will open the pull request, and normally it will go to the marketplace.

[6:32](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=392s) But we made a step before that, like a CI step, where we created a tool that's called Skill Vector. And this is-- this-- what this tool does is to check if this skill is safe or not to be used, uh, using a lot of assessments that I will bring it here.

[6:51](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=411s) And also classify those risks, uh, request remediation, and so on.

[6:59](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=419s) So what v- Skill Vector does in, in a single page is that when a skill is created or changed, not only during the creation phase, one thing that's important is that the engineers are able to use it locally and also be iterating until the skill is being considered safe before upload it.

[7:20](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=440s) And after them upload the skill, we also runs it again because we can't ensure that the engineer has run locally or has run the most updated version. So we also be scanning that after the upload.

[7:37](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=457s) And then we have some deterministic checks for the, uh, easiest parts to check some, uh, easy risks using regular, regular expressions and so on.

[7:48](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=468s) After that, when we check that we need better context, we then use LLM. Uh, it's important to have this hybrid approach with LLM checking the, the context, but also with the deterministic because you know how LLM is, depending on the temper-temperature that was set, sometimes it will check that it's a risk, sometimes it might not.

[8:12](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=492s) And then all of these findings are reported in the PR that was opened to upload the skill, so it will improve the usability since the engineer will have the-- in the same PR what has to be changed before uploading the skill.

[8:27](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=507s) And another good thing that we made that's important is to have a SARIF with all of this so it can be consumed by our security tools as well and generate a report on the risks and be part of our vulnerability management program.

[8:44](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=524s) So depending on the severity, depending on the policy, uh, the skill can require some remediation, can be blocked, all of this before the marketplace distribution.

[8:59](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=539s) So there is the local scan, the pull request, the deterministic scanner, then there is the LLM review, PR feedback, SARIF, and then the decision. Will we use it? Will we allow it?

[9:12](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=552s) Will we allow it but it requires remediation? And so on.

[9:18](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=558s) A few examples of what we have ex- uh, scanned here. It's, uh, a non-exhaustive list. So we are looking upon if there are some unsafe instructions, if there are some drift befi- within the behavior that the agent has, if there are some destructive shell commands that I commented earlier, if there are some file modifications that shouldn't be

[9:40](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=580s) there, uh, credential requests, how are they being done? Some data being exposed, uh, unintentionally. If there are permissions that are overbroad, risky, MCP usage, and much more. These are, these are the main ones.

[9:58](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=598s) And so getting back to the title, we have scanned on that over two thousand skills. Uh, now there is much more than that, but this is the baseline that I brought for you on this presentation.

[10:13](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=613s) Uh, inside this we have identified, uh, more than one thousand and half, uh, risks. So not that one thousand and half, uh, skills had risks because a, a single skill can has many risks.

[10:29](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=629s) But these were the total risks that we identified over, uh, this amount of skills. And one thousand of them were probably, uh, remediated right after.

[10:43](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=643s) And there are a few of them that were really risky that we were able to block before going to the marketplace. So we also had made a ba-- a historical scan looking upon the skills that were created before the Skill Vector we implemented.

[11:03](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=663s) Uh, over there we were able to identify new, uh, risks as well and put it down into the vulnerability management program so it can be re- could be remediated.

[11:17](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=677s) A few lessons that I want to bring here as well. So

[11:22](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=682s) what things that worked well is having both the deterministic scanners for known risk patterns, but also LLM review for, uh, behavior, checking upon the destructive commands, uh, looking upon the credential checks as well, uh, having the output in SARIF, and adding comments on

[11:47](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=707s) PRs. And things that needed improvement and we worked, uh, during the process were also, there were some risks like comments that we were treating equally, but depending on the comment it can be more or less risky.

[12:02](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=722s) Also some signals that were weak and didn't have much context that were, uh, more troublesome than helpful.

[12:11](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=731s) There is also the prompt level ask for confirmation. I, there is a next slide about that, that I will go deeper. That's an important one. Also,

[12:22](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=742s) uh, there were some warnings that seemed, uh, harmless, but only if it was running locally. If they were going to production, then they could be impactful, and they-- we had also to look up on that.

[12:37](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=757s) Uh, if the finding had, hadn't some clear guidance was troublesome as well. And last but not least, we know that other people could create other marketplace. So how can we proactively scan, check there is a new marketplace, and put Skill Vector into it as well?

[12:56](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=776s) So regarding the prompt level, that's something that's important for you to know. People sometimes will add the instruction like, "You need to ask for confirmation," but the AI may a-ask confirmation for itself.

[13:10](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=790s) So from your perspective, there is a human in the loop. But for the AI perspective, there is-- has been a confirmation, and that's okay. Another AI has confirmed it, and let's go.

[13:21](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=801s) So that's something that we were scanning as well, looking upon having proper human in the loop, looking the tool that's in be-executing, if it's going through the approval gates, and so on.

[13:33](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=813s) Having hooks. And within this, as I said, it's a plugin marketplace. Skill is one among many things that there is into that. So there are things that we can reuse from this lesson.

[13:50](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=830s) So for example, treating these, uh, supply chain is important. Reviewing what's being uploaded to the marketplace before it goes there. Letting developers to run these checks locally. Enforcing these checks that are being run locally also in the CI.

[14:08](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=848s) Having deterministic checks together with the LLM checks. Uh, looking upon dangerous actions and prompting, uh, uh, and having enforcement when they happen.

[14:23](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=863s) So next steps o-over here is that I'm talking a lot about skills here, but a lot of these, as I said, could be applied to plugins, to MCP servers, rules, hooks.

[14:35](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=875s) So all of this that I'm saying here, we also have the MCP Vector, the rules checks, and so on. That's also applicable here, but with different risks. Uh, having also different gates depending on policies that we're implementing, depending on the marketplace as well.

[14:54](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=894s) Have some enforcements on tool level here, enforcing that there are audit logs, trusted gateways, and so on. And also, last but not least,

[15:06](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=906s) having the trusted, uh, trusted AI marketplace is very important, so we can have a canonical way to scan and share knowing that them are being safe. And that's not about, only about the skills that are being created by people, but it also includes the third-party skills or plugins and so on.

[15:28](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=928s) So if someone downloads something and wants to use, it's important to upload it on the marketplace so all of this scanning can be done and check if it's safe or not to be used.

[15:39](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=939s) And also that way, allow other people to use in a safe way.

[15:45](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=945s) And that's it. Uh, I'm sharing here my contact. There is, uh, my LinkedIn profile if anybody wants to contact, talk more about that. The QR code will bring you to my profile.

[15:58](https://www.youtube.com/watch?v=iKQ78wyJEXU&t=958s) If you don't want to type, no problem at all. And I hope you, you've enjoyed the talk. [clapping] [outro jingle]
