AStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 8: Nonlinearity
Summary
This lecture provides an advanced overview of Linear-Quadratic Regulator (LQR) theory, extending its application from simple state regulation to complex nonlinear trajectory tracking and optimization. Key concepts include reformulating nonlinear tracking problems using deviation variables ($\delta x$, $\delta u$) to create an auxiliary LQR problem. The discussion culminates in two sophisticated iterative methods: Iterative LQR (iLQR), which linearizes dynamics and quadratizes costs, and Differential Dynamic Programming (DDP), which directly approximates the Bellman equation, offering a second-order approach for optimal control.
Key takeaways
-
LQR as a General Tool
18:03
While LQR is fundamentally designed to drive a state to the origin (regulation), it can be generalized to perform trajectory tracking by defining an auxiliary problem based on deviation variables. The optimal control law structure remains consistent: $u = u_{nominal} + ext{feedback term}$.
-
Nonlinear Tracking via Linearization
24:10
For nonlinear dynamics ($x_{k+1} = f(x_k, u_k)$), the tracking problem can be linearized by performing a Taylor expansion around the nominal trajectory ($\bar{x}, \bar{u}$), allowing the use of LQR techniques on the deviation variables.
-
iLQR vs. DDP
35:05
Both iLQR and DDP are methods for solving nonlinear optimal control problems iteratively. iLQR linearizes dynamics and quadratizes costs, while DDP directly approximates the Bellman equation by quadratizing the Q-function, making it a second-order algorithm in terms of dynamic derivatives.
Technical details
-
LQR Generalization
730s
The standard LQR formulation assumes linear dynamics ($x_k = A x_k + B u_k$) and quadratic cost. This can be generalized to include cross terms ($x H u$), linear state/control terms, and affine dynamics ($x_{k+1} = A_k x_k + B_k u_k + d_k$). The optimal control policy remains a linear feedback term plus an optional feedforward term.
-
Trajectory Tracking Formulation
1083s
To track a desired trajectory ($\bar{x}_k, \bar{u}_k$), the problem is reformulated using deviation variables: $\delta x_k = x_k - \bar{x}_k$ and $\delta u_k = u_k - \bar{u}_k$. The goal becomes minimizing the cost on these deviations, effectively turning a tracking problem into an LQR-style regulation problem in the deviation space.
-
Iterative LQR (iLQR)
1700s
iLQR solves nonlinear optimal control problems by iteratively linearizing the dynamics and quadratizing the cost around a nominal trajectory. The process involves a backward pass (solving Riccati equations) to find optimal deviation variables ($\delta u$), followed by a forward pass using the full, non-linear dynamics to propagate the state sequence.
-
Differential Dynamic Programming (DDP)
2105s
DDP is an advanced technique that approximates the Bellman equation directly by quadratizing the Q-function, rather than linearizing the dynamics. This makes DDP a second-order algorithm in terms of dynamic derivatives compared to iLQR's first-order approximation.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.