Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd
Summary
David Brumley discusses designing reinforcement learning (RL) environments to train AI models in cybersecurity tasks. He argues that traditional benchmarks are flawed because they assume a single vulnerability or rely on unreliable grading oracles. To create robust training environments, he proposes 'audit tasks' using deterministic graders and open-world scoring based on precision and recall across multiple vulnerabilities. The talk highlights the difficulty of measuring true hacking capability—which goes beyond simply triggering a crash—by demonstrating advanced model performance (e.g., Mythos) against 41 real V8 vulnerabilities, including finding zero-day level exploits.
Key takeaways
-
Hacking as a Ladder of Tasks
18:10
Teaching AI to hack should follow a ladder structure: from triggering a crash to achieving arbitrary read/write in memory, and ultimately full arbitrary code execution (10:30). This structured approach allows for measurable progress.
-
The Flaw of Existing Benchmarks
22:02
Current benchmarks often assume only one vulnerability or use LLMs as judges, which is flawed. The model will tend to 'reward hack' by repeatedly finding the easiest known bug (7:46).
-
Deterministic Grading and Open-World Scoring
25:20
To accurately measure capability, environments must use deterministic graders that check for specific bugs. The proposed 'audit task' allows scoring precision and recall across multiple known and unknown vulnerabilities (14:49).
-
High-Value Target Example: V8
Testing on the JavaScript engine V8 in Chrome showed that while models achieved high rates of simple crashes, only advanced models could achieve out-of-sandbox exploits (full control flow hijack), demonstrating a clear capability gap (21:10).
Technical details
-
RL Environment Design
1322s
Environments must be reproducible and sandboxed, utilizing key components like a vulnerable application, an orchestrator, and a deterministic grading oracle. The task prompt should require the LLM to actually exploit the vulnerability, not just find it (13:22).
-
Vulnerability Scoring Metrics
1520s
The 'audit task' calculates precision (found/submitted) and recall (known found/total known) across multiple bugs. This method prevents the model from focusing only on the easiest vulnerability, providing a cleaner learning signal (14:49).
-
V8 Exploitation Chain
A high-value exploit requires chaining vulnerabilities: first achieving in-sandbox primitives (e.g., arbitrary read/write), and then escaping the sandbox to achieve out-of-sandbox control flow hijack (21:10).
Mentioned resources
- exploitbench.ai
- picoCTF
- Pwn2Own
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.