Vibes Not Vulns: Securing the Era of AI-Written Software - Mackenzie Jackson
Summary
The integration of AI tools into software development introduces novel and complex security failure modes that traditional AppSec pipelines are unprepared for. The talk details how 'vibe coded' applications can ship insecure patterns, focusing heavily on prompt injection vulnerabilities within CI/CD workflows and the evolving risks in open-source supply chains (e.g., dependency hallucination). To mitigate these risks, guardrails must shift from simple code scanning to context-aware validation and strict access control.
Key takeaways
-
AI Code Vulnerabilities
3:50
AI systems are not perfect; they introduce vulnerabilities because they make assumptions about business logic. While models improve (especially with 'make sure it's secure' prompts), fundamental flaws like business logic errors remain, meaning AI code cannot be fully trusted yet.
-
Prompt Injection in CI/CD
11:45
A critical new vulnerability class is prompt injection, which allows an attacker to bypass system and application guardrails. This was demonstrated by exploiting the Gemini CLI tool within a CI/CD pipeline to achieve Remote Code Execution (RCE) and leak secrets from GitHub repositories.
-
Supply Chain Risks
20:30
Traditional vulnerability tracking using CVE numbers is fundamentally broken for modern malware attacks, which can spread rapidly. Furthermore, AI hallucination means package managers may suggest non-existent packages or outdated dependencies.
Technical details
-
Prompt Injection Mechanics
430s
Prompt injection occurs when untrusted user input bypasses both the application prompt (system guardrails) and the master prompt. Unlike SQL injection, which can be mitigated with prepared statements, AI context lacks this separation, making it difficult to solve completely.
-
Hallucination Squatting
AI models may suggest non-existent packages (hallucinations) when asked for dependencies. Attackers exploit this by creating these fake packages, leading to 'hallucination squatting' in the supply chain.
-
CI/CD Exploitation Example
750s
A vulnerability was found in a GitHub action using the Gemini CLI tool where an attacker could inject instructions via a GitHub issue body to force the model to leak secrets (e.g., GitHub tokens and API keys).
Mentioned resources
- Aikido Security
- Copilot, Claude, Cursor
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.