Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd
David Brumley discusses designing reinforcement learning (RL) environments to train AI models in cybersecurity tasks. He argues that traditional benchmarks are flawed because they assume a single vulnerability or rely on unreliable grading oracles. To create robust training environments, he proposes 'audit tasks' using deterministic graders and open-world scoring based on precision and recall across multiple vulnerabilities. The talk highlights the difficulty of measuring true hacking capability—which goes beyond simply triggering a crash—by demonstrating advanced model performance (e.g., Mythos) against 41 real V8 vulnerabilities, including finding zero-day level exploits.
Key takeaways
-
Hacking as a Ladder of Tasks
18:10
Teaching AI to hack should follow a ladder structure: from triggering a crash to achieving arbitrary read/write in memory, and ultimately full arbitrary code execution (10:30). This structured approach allows for measurable progress.
-
The Flaw of Existing Benchmarks
22:02
Current benchmarks often assume only one vulnerability or use LLMs as judges, which is flawed. The model will tend to 'reward hack' by repeatedly finding the easiest known bug (7:46).
-
Deterministic Grading and Open-World Scoring
25:20
To accurately measure capability, environments must use deterministic graders that check for specific bugs. The proposed 'audit task' allows scoring precision and recall across multiple known and unknown vulnerabilities (14:49).
-
High-Value Target Example: V8
Testing on the JavaScript engine V8 in Chrome showed that while models achieved high rates of simple crashes, only advanced models could achieve out-of-sandbox exploits (full control flow hijack), demonstrating a clear capability gap (21:10).