Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives
Achieving near speed-of-light latency in GPU collectives is vital for modern AI workloads. The talk introduces low-latency techniques like Sentinel synchronization and the LL128 atomic algorithm to eliminate expensive global memory barriers.
Read article