Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face
This talk addresses the rapidly shifting economics of cyber security due to increasingly powerful AI models. While frontier models can perform extensive reconnaissance and find vulnerabilities, they often fail at the critical 'logical leap' required for exploitation—the ability to reason across complex system states. Arithmetic proposes a new defense paradigm: creating specialized benchmarks focused on access control by having human researchers discover real-world zero days in open-source software. The goal is to train models to replicate the deep reasoning and multi-step logic of skilled attackers, thereby giving defenders a lasting edge.
Key takeaways
-
AI Models Lack World Modeling for Exploitation
17:01
Current large language models (LLMs) struggle with building dynamic world models. The benchmark developed by Arithmetic shows that even advanced models like GPT 5.5 and Opus fail to make the necessary logical leap required to exploit a system, despite successfully reaching the vulnerability check.
-
Open Source Models are Key to Cyber Defense
5:21
The solution to modern cyber challenges requires open-source models. The speaker argues that relying solely on a few large companies is insufficient, and collaboration using open source LLMs will be crucial for building the next generation of defensive systems.
-
Focusing on Access Control Vulnerabilities
11:21
Arithmetic's benchmark focuses specifically on access control, as this is the primary entry point for most attacks. The vulnerabilities tested are 'logic based,' meaning they exploit discrepancies between how different parts of a system check permissions (e.g., checking by name vs. checking by ID).