Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium
This presentation details a sophisticated timing side-channel attack demonstrating how an attacker on a shared GPU can infer sensitive details about a victim's Large Language Model (LLM) operation. By monitoring minute timing variations (as little as 3 microseconds) across shared hardware components, the attacker can determine the model's type, depth, decoding speed, and current workload category (e.g., code, JSON, prose). The attack leverages the fact that memory and compute isolation (like NVIDIA MIG) do not eliminate timing signals from hardware operations that affect the entire GPU. The attack can be executed even across different GPUs connected via NVLink, requiring only physical proximity on the same server.
Key takeaways
-
Timing Reveals Model Fingerprints
An attacker can infer the LLM's model type, structure, and depth with high accuracy (e.g., 97% for model type, 99.3% for decoding speed) solely by observing timing, without accessing the victim's memory or exploiting software vulnerabilities. (0:00)
-
Attack Scope Limitations
2:34
The attack observes timing signals, not content. The attacker cannot recover the actual prompts, answers, or contents of the LLM. (1:54)
-
Cross-GPU Attack via NVLink
9:15
The attack can be extended beyond a single GPU (MIG) to a separate GPU connected via NVLink. While the signal is weaker (losing layer bits), it remains informative, demonstrating that timing leakage is a systemic issue across shared server infrastructure. (9:15)
-
Active Monitoring is Crucial
11:23
Operators must actively monitor the GPU itself for suspicious workloads. The presenter's product, Stealthy, monitors kernel operations and suspicious workloads in real-time, detecting attacks even when the workload itself is the 'payload.' (11:23)