Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium
Summary
This presentation details a sophisticated timing side-channel attack demonstrating how an attacker on a shared GPU can infer sensitive details about a victim's Large Language Model (LLM) operation. By monitoring minute timing variations (as little as 3 microseconds) across shared hardware components, the attacker can determine the model's type, depth, decoding speed, and current workload category (e.g., code, JSON, prose). The attack leverages the fact that memory and compute isolation (like NVIDIA MIG) do not eliminate timing signals from hardware operations that affect the entire GPU. The attack can be executed even across different GPUs connected via NVLink, requiring only physical proximity on the same server.
Key takeaways
-
Timing Reveals Model Fingerprints
An attacker can infer the LLM's model type, structure, and depth with high accuracy (e.g., 97% for model type, 99.3% for decoding speed) solely by observing timing, without accessing the victim's memory or exploiting software vulnerabilities. (0:00)
-
Attack Scope Limitations
2:34
The attack observes timing signals, not content. The attacker cannot recover the actual prompts, answers, or contents of the LLM. (1:54)
-
Cross-GPU Attack via NVLink
9:15
The attack can be extended beyond a single GPU (MIG) to a separate GPU connected via NVLink. While the signal is weaker (losing layer bits), it remains informative, demonstrating that timing leakage is a systemic issue across shared server infrastructure. (9:15)
-
Active Monitoring is Crucial
11:23
Operators must actively monitor the GPU itself for suspicious workloads. The presenter's product, Stealthy, monitors kernel operations and suspicious workloads in real-time, detecting attacks even when the workload itself is the 'payload.' (11:23)
Technical details
-
GPU Isolation and Leakage
140s
While NVIDIA MIG provides hardware isolation (dedicated compute units, VRAM, and L2 cache slices), the timing signal remains unisolated. The memory bus and the overall GPU clock cycle are shared, allowing timing attacks. (2:20)
-
Side-Channel Mechanism
196s
The attack involves the attacker repeatedly reading their own memory and timing these operations (e.g., every 5 microseconds). The attacker does not need to touch the victim's memory, only to measure the slight slowdown caused by the victim's activity on the shared memory bus. (3:16)
-
Workload Inference (Speculative Decoding)
465s
The attack can distinguish between different workload categories (code, JSON, prose, retrieval) by observing the timing changes associated with speculative decoding, which alters the step timing based on the text's predictability. (7:45)
-
Signal Processing for Model Depth
368s
Model depth is inferred by treating the timing trace as a signal and applying a 'best fit' algorithm (not a trained classifier). The method measures the expected signature strength of a given depth (16 to 48 layers) against the observed trace, yielding a highly accurate estimate. (6:08)
Mentioned resources
- Branislav Brzak
- Stealthium
- Behind Bars
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.