The Video Signal

Topic

Vulnerability Discovery

All digests tagged Vulnerability Discovery

Who’s afraid of an open-weight model? GLM, context bombing and post-Black Hat attacks thumbnail

· 26:43

Who’s afraid of an open-weight model? GLM, context bombing and post-Black Hat attacks

The discussion explores the rapid advancement and associated risks of open-weight AI models like GLM-5.3, which show strong capabilities in vulnerability discovery and validation. Defensively, researchers developed 'context bombing,' a technique using malicious prompts to shut down attacking AI agents. The conversation emphasizes that while offensive security (AI model development) is accelerating faster than defensive measures (automated patching/blue team), classic principles like defense-in-depth and assuming breach remain critical. Finally, the segment warns against sophisticated social engineering attacks targeting cybersecurity professionals post-conference.

Key takeaways

  1. AI Vulnerability Discovery is Accelerating 2:00

    Open-weight models like GLM-5.3 demonstrate advanced cyber capabilities through post-training, achieving a score of 84.5% on CyberGym for vulnerability discovery and validation, reaching parity with competitors like GPT Sol and Mythos.

  2. Context Bombing as Defensive Measure 12:10

    Tracebit researchers developed 'context bombing,' which uses malicious prompts placed alongside assets to confuse attacking AI agents. Testing showed that instances of models proceeding with an attack dropped from 91% to 15%.

  3. Blue Team Must Match Offensive Pace 4:00

    Experts stressed the need for significant investment in automated patching and blue team capabilities (e.g., automated SOC) to keep pace with AI-driven offensive security, noting that manual processes are insufficient.

Watch on YouTube Full article

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd thumbnail

· 27:17

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

David Brumley discusses designing reinforcement learning (RL) environments to train AI models in cybersecurity tasks. He argues that traditional benchmarks are flawed because they assume a single vulnerability or rely on unreliable grading oracles. To create robust training environments, he proposes 'audit tasks' using deterministic graders and open-world scoring based on precision and recall across multiple vulnerabilities. The talk highlights the difficulty of measuring true hacking capability—which goes beyond simply triggering a crash—by demonstrating advanced model performance (e.g., Mythos) against 41 real V8 vulnerabilities, including finding zero-day level exploits.

Key takeaways

  1. Hacking as a Ladder of Tasks 18:10

    Teaching AI to hack should follow a ladder structure: from triggering a crash to achieving arbitrary read/write in memory, and ultimately full arbitrary code execution (10:30). This structured approach allows for measurable progress.

  2. The Flaw of Existing Benchmarks 22:02

    Current benchmarks often assume only one vulnerability or use LLMs as judges, which is flawed. The model will tend to 'reward hack' by repeatedly finding the easiest known bug (7:46).

  3. Deterministic Grading and Open-World Scoring 25:20

    To accurately measure capability, environments must use deterministic graders that check for specific bugs. The proposed 'audit task' allows scoring precision and recall across multiple known and unknown vulnerabilities (14:49).

  4. High-Value Target Example: V8

    Testing on the JavaScript engine V8 in Chrome showed that while models achieved high rates of simple crashes, only advanced models could achieve out-of-sandbox exploits (full control flow hijack), demonstrating a clear capability gap (21:10).

Watch on YouTube Full article