# Stanford CS229 Machine Learning | Spring 2026 | Lecture 11: Diffusion Models

## Executive summary

The lecture introduces Diffusion Models as a predominant class of generative models used for tasks like image, video, and action generation. The core mechanism involves two processes: the forward process (gradually adding Gaussian noise to an original clean image $X_0$ over time steps $T$) and the reverse process (learning to iteratively denoise the noisy data back to a clean state). Training these models is achieved by applying the Evidence Lower Bound (ELBO) framework, which minimizes the Kullback-Leibler (KL) divergence between the true conditional distribution and the parameterized model's prediction.

## Key takeaways

- Diffusion Models Overview: These models are generative frameworks that can be applied to images, videos, and robotics actions. They are noted as a superior approach compared to GANs and Variational Autoencoders (VAEs) for image generation.
- Forward vs. Reverse Process: The forward process is a fixed, known noising process that transforms $X_0$ to noise $X_T$. The reverse process is the learned component, parameterized by $ heta$, which predicts $p_ heta(X_{t-1}|X_t)$—the step-by-step denoising path.
- Training Objective (ELBO): The goal is to maximize the marginal likelihood $ ext{log } p_ heta(X_0)$ by applying the Evidence Lower Bound (ELBO). This involves minimizing the KL divergence between the true posterior $q(X_{t-1}|X_t, X_0)$ and the parameterized model's distribution.

## Technical details

- Forward Diffusion Process (Noising): The process is defined by adding Gaussian noise iteratively. The transition from $X_{t-1}$ to $X_t$ can be modeled as: $X_t = rac{1}{ ext{sqrt}(ar{eta}_t)} X_0 + ext{sqrt}(1 - ar{eta}_t) oldsymbol{ ilde{ u}}$, where $oldsymbol{ ilde{ u}}$ is standard normal Gaussian noise. The overall process maintains the scale and co-variance of $X_0$ (assuming normalization).
- Reverse Diffusion Process (Denoising): The reverse process is modeled as a Markov chain, predicting the distribution $p_ heta(X_{t-1}|X_t)$. This model is parameterized by $ heta$ and assumes a Gaussian form with mean $oldsymbol{ ilde{oldsymbol{ u}}}(X_t, t)$ and variance $ ext{sigma}_t^2$.
- Loss Function Derivation: The training objective requires decomposing the KL divergence using a chain rule for KL. The final loss function minimizes the difference between two Gaussian distributions: $q(X_{t-1}|X_t, X_0)$ and $p_ heta(X_{t-1}|X_t)$. When assuming equal co-variance, the loss simplifies to minimizing a quadratic form based on the mean difference.

## Practical implications

- The step-by-step nature of the reverse process allows for gradual, controlled reconstruction (denoising), which is beneficial compared to one-shot generation methods.
- Diffusion models can generate actions in Vision Language Action Models (VLAMs), suggesting applications in robotics and control systems.
- One benefit noted is that diffusion models can potentially generate results in parallel, leading to faster inference time than autoregressive models.

## Topics

Generative Modeling, Diffusion Models, Machine Learning Theory, Gaussian Processes, Variational Inference (ELBO), Stanford CS229 Machine Learning, Stanford AI Programs Information

Source: https://www.youtube.com/watch?v=dqUMCzWjZSI