Topic

Preference Tuning (RLHF)

All digests tagged Preference Tuning (RLHF)

Suggested route

Topic journey

Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.

Start here

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 4 - Reinforcement Learning with LLMs Oct 19, 2026 · 1:43:27 The lecture details advanced RL methods for LLMs, covering PPO, GRPO, and On-Policy Distillation (OPD). These techniques maximize expected rewards by optimizing policies against human preferences or verifiable outcomes, while managing gradient variance and policy deviation.

Stanford Online

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 4 - Reinforcement Learning with LLMs

The lecture details advanced RL methods for LLMs, covering PPO, GRPO, and On-Policy Distillation (OPD). These techniques maximize expected rewards by optimizing policies against human preferences or verifiable outcomes, while managing gradient variance and policy deviation.

Read article