Channel
Digests from GPU MODE — GPU MODE videos on GPU programming, performance, and machine learning systems.
Learn how progressive IR lowering transforms PyTorch layers into efficient GPU kernels. We explore the compiler stack: PyTorch $\rightarrow$ Tensor AR $\rightarrow$ Loop IR $\rightarrow$ Tile IR $\rightarrow$ CUDA, detailing fusion, normalization, and tiling for maximum performance.
Read article
Learn how ML compilers progressively lower PyTorch graphs to efficient CUDA kernels. We explore the IR stack: from PyTorch to Loop IR, Tiling, and Kernel IR, detailing fusion and scheduling.
Read article
The talk presents formal verification for AI-generated GPU kernels, detailing how tools like Tatwa use Z3 solvers to prove semantic equivalence between reference and optimized kernels, moving beyond the limitations of traditional testing.
Read article
Formal verification is crucial for guaranteeing the correctness of AI-generated GPU kernels. The speaker details how tools like Tatwa prove semantic equivalence by decomposing tensor operations into scalar math primitives for solvers like Z3.
Read article
Formal verification is necessary for GPU kernels because testing fails due to non-determinism and massive input spaces. The solution involves using SMT solvers to prove algorithmic correctness at the PTX level.
Read article
The talk details how formal verification proves GPU kernel correctness instead of testing it. By operating at the PTX level and using SMT solvers, it addresses issues like reward hacking and non-determinism inherent in AI-generated code.
Read article
This talk details advanced low-latency collective communication algorithms for GPU clusters, crucial for accelerating LLM inference. By replacing expensive global memory barriers with techniques like Sentinel synchronization and atomic operations, significant speedups are achieved.
Read article
Achieving near speed-of-light latency in GPU collectives is vital for modern AI workloads. The talk introduces low-latency techniques like Sentinel synchronization and the LL128 atomic algorithm to eliminate expensive global memory barriers.
Read article
Mega kernels fuse entire forward passes into one persistent kernel, eliminating traditional GPU overheads like global synchronization and SRAM clearing. This enables fine-grained scheduling by allowing cores to dynamically pick up slack from slower units.
Read article