# Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium

## Executive summary

This presentation details a sophisticated timing side-channel attack demonstrating how an attacker on a shared GPU can infer sensitive details about a victim's Large Language Model (LLM) operation. By monitoring minute timing variations (as little as 3 microseconds) across shared hardware components, the attacker can determine the model's type, depth, decoding speed, and current workload category (e.g., code, JSON, prose). The attack leverages the fact that memory and compute isolation (like NVIDIA MIG) do not eliminate timing signals from hardware operations that affect the entire GPU. The attack can be executed even across different GPUs connected via NVLink, requiring only physical proximity on the same server.

## Key takeaways

- Timing Reveals Model Fingerprints: An attacker can infer the LLM's model type, structure, and depth with high accuracy (e.g., 97% for model type, 99.3% for decoding speed) solely by observing timing, without accessing the victim's memory or exploiting software vulnerabilities. (0:00)
- Attack Scope Limitations: The attack observes timing signals, not content. The attacker cannot recover the actual prompts, answers, or contents of the LLM. (1:54)
- Cross-GPU Attack via NVLink: The attack can be extended beyond a single GPU (MIG) to a separate GPU connected via NVLink. While the signal is weaker (losing layer bits), it remains informative, demonstrating that timing leakage is a systemic issue across shared server infrastructure. (9:15)
- Active Monitoring is Crucial: Operators must actively monitor the GPU itself for suspicious workloads. The presenter's product, Stealthy, monitors kernel operations and suspicious workloads in real-time, detecting attacks even when the workload itself is the 'payload.' (11:23)

## Technical details

- GPU Isolation and Leakage: While NVIDIA MIG provides hardware isolation (dedicated compute units, VRAM, and L2 cache slices), the timing signal remains unisolated. The memory bus and the overall GPU clock cycle are shared, allowing timing attacks. (2:20)
- Side-Channel Mechanism: The attack involves the attacker repeatedly reading their own memory and timing these operations (e.g., every 5 microseconds). The attacker does not need to touch the victim's memory, only to measure the slight slowdown caused by the victim's activity on the shared memory bus. (3:16)
- Workload Inference (Speculative Decoding): The attack can distinguish between different workload categories (code, JSON, prose, retrieval) by observing the timing changes associated with speculative decoding, which alters the step timing based on the text's predictability. (7:45)
- Signal Processing for Model Depth: Model depth is inferred by treating the timing trace as a signal and applying a 'best fit' algorithm (not a trained classifier). The method measures the expected signature strength of a given depth (16 to 48 layers) against the observed trace, yielding a highly accurate estimate. (6:08)

## Practical implications

- Operators must consider tenant placement, ensuring sensitive models are not placed adjacent to untrusted tenants on the same GPU.
- Be aware of the isolation trade-offs: MIG prevents peer-to-peer communication (tensor parallelism) and limits leakage to timing only.
- Implement continuous, active monitoring of the GPU hardware itself (e.g., using tools like Salty I'm) to detect malicious workloads running on the device, rather than relying solely on software-level isolation.
- The risk persists even when connecting multiple GPUs via NVLink, as timing leakage remains the primary vulnerability.

## Topics

Side-Channel Attacks, GPU Security, LLM Inference, NVIDIA MIG, NVLink, Timing Attacks, Branislav Brzak, Stealthium, Behind Bars

Source: https://www.youtube.com/watch?v=SacJfmp33vM