AStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 8: Nonlinearity
This lecture provides an advanced overview of Linear-Quadratic Regulator (LQR) theory, extending its application from simple state regulation to complex nonlinear trajectory tracking and optimization. Key concepts include reformulating nonlinear tracking problems using deviation variables ($\delta x$, $\delta u$) to create an auxiliary LQR problem. The discussion culminates in two sophisticated iterative methods: Iterative LQR (iLQR), which linearizes dynamics and quadratizes costs, and Differential Dynamic Programming (DDP), which directly approximates the Bellman equation, offering a second-order approach for optimal control.
Key takeaways
-
LQR as a General Tool
18:03
While LQR is fundamentally designed to drive a state to the origin (regulation), it can be generalized to perform trajectory tracking by defining an auxiliary problem based on deviation variables. The optimal control law structure remains consistent: $u = u_{nominal} + ext{feedback term}$.
-
Nonlinear Tracking via Linearization
24:10
For nonlinear dynamics ($x_{k+1} = f(x_k, u_k)$), the tracking problem can be linearized by performing a Taylor expansion around the nominal trajectory ($\bar{x}, \bar{u}$), allowing the use of LQR techniques on the deviation variables.
-
iLQR vs. DDP
35:05
Both iLQR and DDP are methods for solving nonlinear optimal control problems iteratively. iLQR linearizes dynamics and quadratizes costs, while DDP directly approximates the Bellman equation by quadratizing the Q-function, making it a second-order algorithm in terms of dynamic derivatives.