# Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

## Executive summary

David Brumley discusses designing reinforcement learning (RL) environments to train AI models in cybersecurity tasks. He argues that traditional benchmarks are flawed because they assume a single vulnerability or rely on unreliable grading oracles. To create robust training environments, he proposes 'audit tasks' using deterministic graders and open-world scoring based on precision and recall across multiple vulnerabilities. The talk highlights the difficulty of measuring true hacking capability—which goes beyond simply triggering a crash—by demonstrating advanced model performance (e.g., Mythos) against 41 real V8 vulnerabilities, including finding zero-day level exploits.

## Key takeaways

- Hacking as a Ladder of Tasks: Teaching AI to hack should follow a ladder structure: from triggering a crash to achieving arbitrary read/write in memory, and ultimately full arbitrary code execution (10:30). This structured approach allows for measurable progress.
- The Flaw of Existing Benchmarks: Current benchmarks often assume only one vulnerability or use LLMs as judges, which is flawed. The model will tend to 'reward hack' by repeatedly finding the easiest known bug (7:46).
- Deterministic Grading and Open-World Scoring: To accurately measure capability, environments must use deterministic graders that check for specific bugs. The proposed 'audit task' allows scoring precision and recall across multiple known and unknown vulnerabilities (14:49).
- High-Value Target Example: V8: Testing on the JavaScript engine V8 in Chrome showed that while models achieved high rates of simple crashes, only advanced models could achieve out-of-sandbox exploits (full control flow hijack), demonstrating a clear capability gap (21:10).

## Technical details

- RL Environment Design: Environments must be reproducible and sandboxed, utilizing key components like a vulnerable application, an orchestrator, and a deterministic grading oracle. The task prompt should require the LLM to actually exploit the vulnerability, not just find it (13:22).
- Vulnerability Scoring Metrics: The 'audit task' calculates precision (found/submitted) and recall (known found/total known) across multiple bugs. This method prevents the model from focusing only on the easiest vulnerability, providing a cleaner learning signal (14:49).
- V8 Exploitation Chain: A high-value exploit requires chaining vulnerabilities: first achieving in-sandbox primitives (e.g., arbitrary read/write), and then escaping the sandbox to achieve out-of-sandbox control flow hijack (21:10).

## Practical implications

- Security testing must move beyond simple crash detection (e.g., CyberGym) to measure complex capabilities like sandbox escape and arbitrary code execution.
- Building environments grounded in real-world bugs and using deterministic graders is crucial for accurate security benchmarking ('Don't benchmaxx security').
- The use of open-source targets with multiple known vulnerabilities allows for a more realistic assessment of model reasoning and capability (the 'audit task').

## Topics

Reinforcement Learning, Cybersecurity, Vulnerability Discovery, LLMs, Software Exploitation, exploitbench.ai, picoCTF, Pwn2Own

Source: https://www.youtube.com/watch?v=ZFxh7sqbUZo