# Vibes Not Vulns: Securing the Era of AI-Written Software - Mackenzie Jackson

## Executive summary

The integration of AI tools into software development introduces novel and complex security failure modes that traditional AppSec pipelines are unprepared for. The talk details how 'vibe coded' applications can ship insecure patterns, focusing heavily on prompt injection vulnerabilities within CI/CD workflows and the evolving risks in open-source supply chains (e.g., dependency hallucination). To mitigate these risks, guardrails must shift from simple code scanning to context-aware validation and strict access control.

## Key takeaways

- AI Code Vulnerabilities: AI systems are not perfect; they introduce vulnerabilities because they make assumptions about business logic. While models improve (especially with 'make sure it's secure' prompts), fundamental flaws like business logic errors remain, meaning AI code cannot be fully trusted yet.
- Prompt Injection in CI/CD: A critical new vulnerability class is prompt injection, which allows an attacker to bypass system and application guardrails. This was demonstrated by exploiting the Gemini CLI tool within a CI/CD pipeline to achieve Remote Code Execution (RCE) and leak secrets from GitHub repositories.
- Supply Chain Risks: Traditional vulnerability tracking using CVE numbers is fundamentally broken for modern malware attacks, which can spread rapidly. Furthermore, AI hallucination means package managers may suggest non-existent packages or outdated dependencies.

## Technical details

- Prompt Injection Mechanics: Prompt injection occurs when untrusted user input bypasses both the application prompt (system guardrails) and the master prompt. Unlike SQL injection, which can be mitigated with prepared statements, AI context lacks this separation, making it difficult to solve completely.
- Hallucination Squatting: AI models may suggest non-existent packages (hallucinations) when asked for dependencies. Attackers exploit this by creating these fake packages, leading to 'hallucination squatting' in the supply chain.
- CI/CD Exploitation Example: A vulnerability was found in a GitHub action using the Gemini CLI tool where an attacker could inject instructions via a GitHub issue body to force the model to leak secrets (e.g., GitHub tokens and API keys).

## Practical implications

- Implement human-in-the-loop review processes for AI-generated code.
- Treat all untrusted input (e.g., GitHub issues) as potential vectors for prompt injection.
- Focus on limiting the ability of AI agents to pass sensitive information or execute commands, rather than trying to block specific prompts.
- Utilize advanced tools like Mthos that understand application context beyond single lines of code.

## Topics

AI Security, Prompt Injection, CI/CD Pipelines, Software Supply Chain, Vulnerability Management, Aikido Security, Copilot, Claude, Cursor

Source: https://www.youtube.com/watch?v=Vc1C-QOMGhk