# AStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 8: Nonlinearity

## Executive summary

This lecture provides an advanced overview of Linear-Quadratic Regulator (LQR) theory, extending its application from simple state regulation to complex nonlinear trajectory tracking and optimization. Key concepts include reformulating nonlinear tracking problems using deviation variables ($\delta x$, $\delta u$) to create an auxiliary LQR problem. The discussion culminates in two sophisticated iterative methods: Iterative LQR (iLQR), which linearizes dynamics and quadratizes costs, and Differential Dynamic Programming (DDP), which directly approximates the Bellman equation, offering a second-order approach for optimal control.

## Key takeaways

- LQR as a General Tool: While LQR is fundamentally designed to drive a state to the origin (regulation), it can be generalized to perform trajectory tracking by defining an auxiliary problem based on deviation variables. The optimal control law structure remains consistent: $u = u_{nominal} + ext{feedback term}$.
- Nonlinear Tracking via Linearization: For nonlinear dynamics ($x_{k+1} = f(x_k, u_k)$), the tracking problem can be linearized by performing a Taylor expansion around the nominal trajectory ($\bar{x}, \bar{u}$), allowing the use of LQR techniques on the deviation variables.
- iLQR vs. DDP: Both iLQR and DDP are methods for solving nonlinear optimal control problems iteratively. iLQR linearizes dynamics and quadratizes costs, while DDP directly approximates the Bellman equation by quadratizing the Q-function, making it a second-order algorithm in terms of dynamic derivatives.

## Technical details

- LQR Generalization: The standard LQR formulation assumes linear dynamics ($x_k = A x_k + B u_k$) and quadratic cost. This can be generalized to include cross terms ($x H u$), linear state/control terms, and affine dynamics ($x_{k+1} = A_k x_k + B_k u_k + d_k$). The optimal control policy remains a linear feedback term plus an optional feedforward term.
- Trajectory Tracking Formulation: To track a desired trajectory ($\bar{x}_k, \bar{u}_k$), the problem is reformulated using deviation variables: $\delta x_k = x_k - \bar{x}_k$ and $\delta u_k = u_k - \bar{u}_k$. The goal becomes minimizing the cost on these deviations, effectively turning a tracking problem into an LQR-style regulation problem in the deviation space.
- Iterative LQR (iLQR): iLQR solves nonlinear optimal control problems by iteratively linearizing the dynamics and quadratizing the cost around a nominal trajectory. The process involves a backward pass (solving Riccati equations) to find optimal deviation variables ($\delta u$), followed by a forward pass using the full, non-linear dynamics to propagate the state sequence.
- Differential Dynamic Programming (DDP): DDP is an advanced technique that approximates the Bellman equation directly by quadratizing the Q-function, rather than linearizing the dynamics. This makes DDP a second-order algorithm in terms of dynamic derivatives compared to iLQR's first-order approximation.

## Practical implications

- The structure of the resulting tracking controller (nominal control + linear feedback term) is widely used in robotics and autonomous systems, providing a balance between open-loop planning efficiency and closed-loop robustness.
- These techniques are critical for developing robust controllers in dynamic environments where model mismatches or disturbances occur.

## Topics

Optimal Control, Nonlinear Dynamics, Control Theory, Trajectory Optimization, Machine Learning (AI), Principles of Robot Autonomy, AA203 Optimal and Learning-Based Control Course Info, Lecture Slides (AA203)

Source: https://www.youtube.com/watch?v=1YdgSwEtf_s