Prompt Injection — The Video Signal

Sep 22, 2026 · 55:19

Ask the Experts: Evaluating Agent Skills | Nemotron Labs

This session details the evaluation of autonomous AI agent skills using NVIDIA's open-source framework, SkillEvaluator. The system provides a comprehensive, multi-tier pipeline to ensure that skills are safe, reliable, and genuinely useful before deployment. The evaluation process covers static security scanning (detecting risks like prompt injection and data exfiltration), deduplication checks, and live agent testing in a sandbox environment. This framework is critical for establishing quality gates in AI-driven workflows.

Key takeaways

Three-Tier Evaluation Framework 2:00
Skill evaluation utilizes a three-tier framework: 1) Static/Security checks (using SkillSpector), 2) Deduplication checks against an internal registry, and 3) Live Agent Testing, which assesses if the skill improves task performance in a sandbox environment.
Security Scanning for Agent Risks 4:00
SkillSpector is a standalone security scanner that checks for agent-native risks, including prompt injection, data exfiltration, trigger abuse, and tool poisoning. It combines static analysis with optional LLM semantic analysis.
Measuring Skill Utility (Skill Lift) 7:30
The evaluation measures 'Skill Lift' across five dimensions: efficiency, correctness, effectiveness, discoverability, and security. This determines if the skill genuinely helps the agent perform the task better compared to a baseline.
Live Agent Testing Methodology 9:00
Live agent evaluation runs the agent in a secure sandbox, performing an A/B test by comparing the agent's performance when the skill is available versus when it is restricted. This confirms if the skill is functionally necessary for the task.

Watch on YouTube Full article

Sep 14, 2026 · 21:00

We let an AI agent execute Bash and lived to talk about it — Sarah Sanders, PostHog

The talk details the architecture and security challenges of building agentic CLI tools, exemplified by PostHog's 'Wizard.' The speaker emphasizes that an agent with command execution capabilities is essentially a 'malware starter pack.' Security must be layered, deterministic, and must account for dangerous inputs originating not only from users but also from the system's own supply chain (the 'Context Mill'). The solution involves implementing a dedicated, deterministic scanner (Warlock) and ensuring that the LLM layer acts only as an 'adviser,' never an enforcer.

Key takeaways

The Danger of Agentic Tools 0:03
An agent that can run commands is structurally similar to malware, requiring extreme caution. The threat model must account for how 'attacks compose' from multiple, seemingly innocent vulnerabilities.
Context Supply Chain Risk 0:11
The most dangerous input can come from the system's own supply chain (the Context Mill), such as a seemingly harmless prompt injection payload signed by the development team.
Security Must Be Deterministic 0:15
For enforcement, security mechanisms must be deterministic and mechanical. The LLM layer should only be an 'adviser' to reduce noise, never the primary enforcer, to prevent failure modes.

Watch on YouTube Full article

Sep 10, 2026 · 55:36

AAIF Reading Group - Prompt Injection as Role Confusion: Rethinking Agent Security

This reading group explores Prompt Injection through the lens of 'Role Confusion,' arguing that the vulnerability stems from the model's inability to strictly isolate roles (User, System, Tool, Thought). Speakers detail advanced attack vectors, including forging chain-of-thought reasoning and exploiting model style, which significantly increases attack success rates (ASR). Mitigation strategies discussed include implementing instruction hierarchies, sandboxing environments, and ensuring robust access controls, particularly when dealing with smaller or less state-of-the-art (SOTA) models.

Key takeaways

Role Confusion as the Core Vulnerability 18:20
The core claim is that prompt injection is a problem of role confusion, where the model treats fabricated reasoning (e.g., forged chain-of-thought) as its own internal thought process, rather than recognizing it as external input. This allows attackers to mislead the model into following harmful instructions.
Advanced Attack Vectors 20:00
Attackers can increase ASR by injecting fabricated reasoning (forged thought) or by exploiting the model's style. One study showed that injecting forged chain-of-thought increased the ASR from 0% to 94% for OpenAI models.
System-Level Security Risks 30:00
Security risks extend beyond prompts to system components. Examples include exploiting API calls (e.g., Anthropic's Opus 4.8) by injecting encrypted tokens/signatures from a smaller model (Haiku), and supply chain attacks (e.g., the LightLM incident) where malicious packages are introduced via open-source frameworks.
Mitigation through Design 38:20
Proposed solutions include implementing 'instruction hierarchies' (prioritizing roles) and running models in isolated, sandboxed environments. The speakers emphasized that security must be baked into the design, not just added as a patch.

Watch on YouTube Full article

Aug 19, 2026 · 29:15

What should security leaders do with AI? They don’t know.

Cybersecurity leaders face 'AI decision fatigue' due to the rapid pace of AI adoption and emerging threats. To mitigate this paralysis, experts recommend starting AI deployment with low-risk, repetitive tasks (L1/L2 alerts) or by arming red teams. The discussion highlighted sophisticated new attack vectors like Ghostjacking—which poisons trusted data sources (logs, alerts)—and questioned the reliability of AI in patching vulnerabilities. Overall, successful AI integration requires a 'human-in-the-loop' approach and strict adherence to Zero Trust principles, particularly regarding agent permissions.

Key takeaways

Start AI deployment with repetitive tasks
Initial AI adoption should focus on automating predictable, high-volume activities (e.g., triaging L1/L2 alerts or vendor risk assessments) to reduce staff fatigue and build confidence in the technology.
Red Teaming is key for defensive AI use 17:25
Equipping red teams with advanced tools allows them to simulate sophisticated threat actor behavior, enabling defenders to proactively develop countermeasures against emerging attack methods.
Ghostjacking targets trusted data sources 17:25
A new attack method involves sneaking malicious prompts into highly trusted systems like logs, alerts, and error reports. This is a sophisticated evolution of prompt injection that bypasses initial security controls.
AI patching requires human oversight
Research indicates AI-generated patches (e.g., using GPT 5.5) have a low success rate, often failing to solve the underlying vulnerability or creating new issues. Human validation and partnership are crucial.

Watch on YouTube Full article

Aug 17, 2026 · 19:06

Security Firewall for Agents — Ryan Dahl, Deno

The video addresses the critical security challenge of granting autonomous AI agents write access to sensitive production systems (e.g., Postgres, Kubernetes). Since relying solely on model alignment is insufficient against prompt injection, the speaker introduces Claw Patrol: an MIT-licensed proxy that operates below the HTTP layer. This system inspects every byte leaving the agent, enforcing granular rules defined in HCL (Terraform configuration language) to prevent destructive actions like spawning `psql` subprocesses and dropping tables.

Key takeaways

The Danger of Autonomous Agents 5:28
Giving agents write access to systems like Postgres, Kubernetes, AWS, and ClickHouse is powerful for incident response but highly dangerous. Agents can be prompt-injected through support systems, allowing them to perform nefarious actions (e.g., running `psql` subprocesses to delete tables).
Security Must Be Externalized 4:07
The speaker asserts that security cannot be 'wishful thinking' based on model obedience. Since agents are untrusted software, the guard mechanism must sit outside of the agent itself.
Claw Patrol: Byte-Level Inspection 6:45
Claw Patrol is a proxy that sits in front of agents and parses every byte leaving them, operating below the HTTP layer. This allows it to detect dangerous paths that are not standard HTTP requests, such as a subprocess spawning `psql` to connect via an EKS endpoint.

Watch on YouTube Full article

Aug 10, 2026 · 13:22

5 Best Practices for Building AI Agent Skills

This guide outlines five best practices for building reliable, secure, and effective AI agent skills. Skills are defined as procedural knowledge packaged in a `skill.md` file that teaches an AI agent specific job functions. Best practices emphasize improving skill triggering via detailed descriptions, grounding content in real domain expertise, managing context window size by using progressive disclosure, enforcing deterministic logic through scripts for critical steps, and rigorously vetting all skills for security vulnerabilities.

Key takeaways

Best Practice 1: Optimize the Skill Description (Triggering) 2:19
The agent uses the skill's name and description to decide if it should run. The description must be highly informative, stating what the skill does and when it should be used. It is recommended to 'oversell' the description slightly rather than underselling it, as models tend to under-trigger.
Best Practice 2: Build from Real Expertise 5:58
Skills must contain domain expertise that the model cannot generate on its own. This content should be synthesized from existing artifacts (e.g., old reports, run books, PR feedback). The highest value section in the skill body is often 'gotchas'—environment-specific facts or corrections made during manual execution.
Best Practice 3: Spend Context Wisely 11:15
The goal is to keep the skill body lean. Since the entire skill body contributes to the context window, only include information the agent wouldn't know otherwise. For large bodies of text, use a dedicated `references` sub-folder and implement 'progressive disclosure,' allowing the agent to open files only when needed.
Best Practice 4: Use Deterministic Scripts for Fragile Steps
For steps that must be exactly correct (fragile steps), do not rely on the model's probabilistic improvisation. Instead, write deterministic code and place it in a dedicated `scripts` directory within the skills folder. This ensures consistent, reliable execution.
Best Practice 5: Vet Skills Before Running Them
Treat agent skills like any external dependency package. Because skills can run code and access local file systems or APIs, they must be audited for security flaws (e.g., prompt injection or malware) before deployment.

Watch on YouTube Full article

Aug 5, 2026 · 55:02

AI Security in Practice: Protecting Your AI-Powered Applications - Olivia Liddell

This talk provides a deep dive into securing AI-powered applications by addressing unique vulnerabilities that traditional security measures often miss. Using a fictional online retailer (ABC Company) as a case study, the presentation outlines three primary AI security risks—Prompt Injection, Data Poisoning, and Improper Output Handling. Mitigation requires implementing layered defenses across the entire application workflow: input validation, model protection, output sanitization, and continuous monitoring.

Key takeaways

AI Attacks are Subtle and Invisible 17:32
Unlike traditional attacks (e.g., DDoS), AI vulnerabilities can be slow, subtle, and partially invisible to standard firewalls or logging systems, requiring a shift in defensive thinking.
Understand the Three Core Risks 17:32
The three major risks are: Prompt Injection (tricking the model with disguised instructions), Data Poisoning (manipulating training data over time), and Improper Output Handling (trusting model output without validation).
Implement Layered Defenses 45:00
Security must be applied at four stages: Input Validation (sanitizing user input before it reaches the model), Model Protection (limiting scope and permissions), Output Sanitization (treating all AI-generated content as untrusted), and Monitoring/Operations (tracking model behavior changes).

Watch on YouTube Full article

Aug 5, 2026 · 23:21

Vibes Not Vulns: Securing the Era of AI-Written Software - Mackenzie Jackson

The integration of AI tools into software development introduces novel and complex security failure modes that traditional AppSec pipelines are unprepared for. The talk details how 'vibe coded' applications can ship insecure patterns, focusing heavily on prompt injection vulnerabilities within CI/CD workflows and the evolving risks in open-source supply chains (e.g., dependency hallucination). To mitigate these risks, guardrails must shift from simple code scanning to context-aware validation and strict access control.

Key takeaways

AI Code Vulnerabilities 3:50
AI systems are not perfect; they introduce vulnerabilities because they make assumptions about business logic. While models improve (especially with 'make sure it's secure' prompts), fundamental flaws like business logic errors remain, meaning AI code cannot be fully trusted yet.
Prompt Injection in CI/CD 11:45
A critical new vulnerability class is prompt injection, which allows an attacker to bypass system and application guardrails. This was demonstrated by exploiting the Gemini CLI tool within a CI/CD pipeline to achieve Remote Code Execution (RCE) and leak secrets from GitHub repositories.
Supply Chain Risks 20:30
Traditional vulnerability tracking using CVE numbers is fundamentally broken for modern malware attacks, which can spread rapidly. Furthermore, AI hallucination means package managers may suggest non-existent packages or outdated dependencies.

Watch on YouTube Full article

Jul 31, 2026 · 1:01:49

Prompt-Jacking: The Rise of a New Supply Chain Risk - Kasimir Schulz & Kenneth Yeung

The talk details how AI coding assistants and agentic systems introduce novel supply chain risks by allowing attackers to compromise developers through hidden text or malicious prompts. Vulnerabilities are often found not in traditional code flaws but in the design of agents themselves. Key attack vectors include indirect prompt injection via raw file data (e.g., READMEs), exploiting tool call responses, manipulating control sequences, and abusing structured formats like JSON within automated pipelines.

Key takeaways

Inspect Raw Data Sources 2:00
When reviewing codebases or documentation, always check the raw markdown/HTML data (e.g., a README file) rather than just the pre-rendered view, as malicious payloads can be hidden in comments or unrendered sections.
Guardrails are Not Enough 10:30
While guardrails (LLM as a judge, classification models) are useful, they can be bypassed by advanced techniques like recursive prompt injection or token manipulation. Security must address architectural weaknesses beyond just the prompt.
Separate Control and Data Planes 17:45
A critical defense is ensuring a strict separation between system instructions (the control plane) and user/tool input data (the data plane). LLMs often fail to distinguish between these two, making this separation crucial for security.
Audit All Skills and Tools 28:00
When using agentic skills or tools (e.g., in OpenWebUI), audit the skill's name, description, and parameters, as these are often inserted into the system prompt at a high privilege level and can be exploited for injection.

Watch on YouTube Full article

Jul 28, 2026 · 36:31

How to Break AI Systems (Before Someone Else Does) - Gary Lopez - NDC Toronto 2026

This talk provides an advanced overview of AI system vulnerabilities, arguing that Large Language Models (LLMs) are fundamentally different from traditional software because they predict the next token rather than executing code with clear separation. The presentation details various attack vectors—including direct and indirect prompt injections, encoding bypasses, and agentic risks—and outlines modern defense strategies like 'Spotlighting' to help developers build more resilient AI applications.

Key takeaways

LLMs Fail Due to Data/Code Ambiguity 18:03
Unlike traditional software with clear separation between executable code and data, LLMs treat all inputs (user prompts, system instructions, external data) as tokens for prediction. This fundamental flaw allows attackers to bypass safeguards.
AI Red Teaming is Evolving 5:20
Modern AI red teaming must consider not only traditional adversarial attacks but also Responsible AI (RAI) challenges, such as hallucinations, bias inference, and knowledge-to-action instructions.
Mitigation Requires Multi-Layered Defense 29:50
Defenses must be applied at multiple points: input filters (before the model receives data), output guards (after generation but before user display), and through techniques like 'Spotlighting' to enforce structural integrity.

Watch on YouTube Full article

Jul 23, 2026 · 32:16

BONUS EPISODE: 76 Malicious AI Skills Were Hiding in Plain Sight

The security landscape for AI coding agents is rapidly evolving, shifting focus from traditional code vulnerabilities to risks introduced by 'Skills' and context management. The presentation details findings from Snyk’s ToxicSkills research, which uncovered 76 malicious skills containing malware or prompt injection vectors in open agent repositories. To mitigate these threats, organizations must adopt secure-by-default practices, implement governance through private skill registries (like Tessl), and utilize advanced scanning tools that analyze natural language context alongside code.

Key takeaways

The Shift to Agent Security 6:38
AI agents are changing developer workflows from reviewing lines of code to managing a 'team of agents.' This necessitates securing the inputs (context) and artifacts produced by these agents, making agent security paramount.
Malicious Skills Threat 17:55
Research found that third-party skills can contain purely malicious code or prompt injection attacks. These threats are difficult for traditional scanners to detect because they operate by influencing the agent's context rather than just being visible malware.
Governance and Scanning Solutions 23:58
Implementing secure skill registries (e.g., Tessl) that integrate security scanning (Snyk) is crucial. This allows organizations to scan skills for potential prompt injection, malware, and versioning issues before deployment.
Secure-by-Default Stack 27:50
Best practices include scanning third-party skills *before* use and running security scanners on custom-authored skills to identify potential vulnerabilities, such as mishandling or passing secrets in plaintext.

Watch on YouTube Full article

Jul 22, 2026 · 31:30

GPT-Red: Can AI red teams stop prompt injections?

This technical discussion explores how AI is being used in advanced cybersecurity defense mechanisms, specifically focusing on automated red teaming and scam interception. Key tools discussed include OpenAI's internal GPT-Red model, which significantly improves model resilience against prompt injections (e.g., reducing attack effectiveness from 95% to 10%). Another tool, ScamBuster, uses AI to bait scammers into revealing their tactics and infrastructure for threat intelligence gathering. The conversation concludes by addressing the widening gap between technical skill and raw ability in cybersecurity, warning that while AI provides immense power, human professionals must maintain foundational skills to remain effective.

Key takeaways

GPT-Red's Effectiveness Against Prompt Injection 0:23
OpenAI utilizes GPT-Red, an internal automated red teaming model, which performs better than human red teamers. This process was key in making models like GPT 5.6 Sol more robust; for instance, 'fake chain of thought attacks' that were 95% effective on GPT 5.1 are only 10% effective on GPT 5.6.
ScamBuster for Threat Intelligence 5:21
ScamBuster is an open-source AI tool designed to interact with email scammers, subtly gathering information about their tactics and infrastructure (IOCs) that can be fed back into security teams and law enforcement.
The Skill vs. Ability Gap 10:35
Bruce Schneier's essay highlights that AI is decoupling skills from abilities in cybersecurity, meaning individuals can now perform sophisticated hacks without the years of training and ethical framework traditionally required.
Maintaining Foundational Skills 20:05
The consensus takeaway for professionals is that while AI acts as a force multiplier, individuals must continue to develop their personal skills (e.g., the ability to blue/red team) to handle scenarios where the AI fails or cannot complete the task.

Watch on YouTube Full article