# Lecture 115: Proving Kernels Correct Instead of Testing Them

## Executive summary

This lecture details the necessity and methodology of formally verifying GPU kernels, moving beyond traditional testing due to the massive input space and non-deterministic nature of GPU execution. The core approach involves reasoning about kernel correctness at the PTX level using Satisfiability Modulo Theories (SMT) solvers. The process requires modeling mathematical concepts (like floating-point operations and unbounded integers) as abstract 'reals' to prove equivalence between a candidate kernel and a reference implementation, thereby guaranteeing algorithmic correctness.

## Key takeaways

- Limitations of Testing GPU Kernels: Due to the vast input space and the non-deterministic scheduling inherent in GPU hardware (e.g., Nvidia), testing alone cannot guarantee correctness. Errors can occur across various boundaries (e.g., denormal boundaries, cancellation terms), and the execution order cannot be guaranteed at runtime. (0:01:30)
- Formal Verification at the PTX Level: To prove correctness, the goal is to show that for every value X, the candidate kernel produces the same output as the reference kernel. This verification is performed at the PTX level because it provides relatively well-defined semantics, allowing the system to cover inputs from various sources (Triton, CUDA, inline PTX). (0:02:10)
- SMT Solvers and Mathematical Abstraction: Formal verification uses SMT solvers to convert code into mathematical boolean abstractions. By modeling values as 'reals' (abstract values with infinite precision) and unbounded mathematical integers, the system can prove algorithmic equivalence without requiring bit-exact checks, which would lead to an unmanageable 'explosion' of clauses. (0:03:20)
- Advanced Verification Boundaries: Future work focuses on formally verifying complex boundaries, including sandboxes (to prevent escape paths from LLM-generated code) and numerical stability (e.g., quantization stability), which are addressed separately from algorithmic correctness. (0:04:30)

## Technical details

- PTX Memory Semantics (Weak vs. Strong Loads): At the PTX level, memory loads can be classified as weak or strong. A weak load (e.g., `LD global`) tells the compiler that consistency checks are not required, allowing the compiler to hoist the load out of a loop. This behavior must be carefully analyzed, as it can lead to incorrect assumptions about data state if race conditions exist. (0:02:40)
- SMT Solver Operation: SMT solvers function by declaring mathematical theories (e.g., arithmetic, boolean logic). These theories are converted into boolean abstractions solved by a SAT solver, which is then checked by theory checkers. The process iterates until a counterexample is found or a formal proof of equivalence is established. (0:03:40)
- Modeling Floating Point Operations: To avoid the computational explosion of bit-exact verification, the system models floating-point values as abstract 'reals' (infinite precision). For complex functions like `exp` (used in Softmax), the properties (e.g., product rule: $e^a imes e^b = e^{a+b}$) are defined mathematically rather than being implemented bit-by-bit. (0:04:00)
- Sandbox Verification: Formal verification of sandboxes involves formally verifying the system call interfaces (e.g., using GVisor's syscall interface) to guarantee that no escape paths exist, ensuring the code cannot access resources outside the sandbox boundary. (0:04:50)

## Practical implications

- Enables the creation of highly reliable, formally verified ML/AI kernels that are resistant to non-deterministic hardware behavior.
- Provides a method to verify the safety and integrity of sandboxed environments, crucial for running untrusted code (e.g., LLM agents).
- Allows compiler developers to reason about algorithmic correctness (e.g., mathematical equivalence) without needing to test every possible hardware state or input value.

## Topics

Formal Verification, GPU Computing, Compiler Design, SMT Solvers, PTX, Numerical Stability, Sandboxing, LLM Agents, Blog Post

Source: https://www.youtube.com/watch?v=7XsSd9mqay4