Stanford CS229 Machine Learning | Spring 2026 | Lecture 2: Supervised Learning Setup
This lecture establishes the foundational mathematical framework for supervised machine learning, focusing on linear regression as a canonical example. Key concepts include defining a hypothesis function ($h: X o Y$), minimizing error using $ ext{L}_2$ loss (Mean Squared Error), and implementing optimization via Gradient Descent. The discussion emphasizes that modern ML success relies heavily on scaling models and computational efficiency, particularly through techniques like mini-batching and understanding the relationship between batch size and generalization.
Key takeaways
-
Supervised Learning Setup
17:12
Supervised learning requires a training set of paired examples $(X_i, Y_i)$, where $X$ is the input feature space (e.g., images, house data) and $Y$ is the corresponding label or value. The goal is to learn a hypothesis function $h$ that generalizes well to unseen data.
-
Regression vs. Classification
22:00
If the output $Y$ is a continuous real number (e.g., house price), it is a regression problem. If $Y$ is discrete or categorical (e.g., cat/dog label, positive/negative review), it is a classification problem.
-
Optimization via $ ext{L}_2$ Loss
30:40
The model parameters ($ heta$) are chosen by minimizing the empirical risk, typically using the squared error (the $ ext{L}_2$ norm). The objective function $J( heta)$ is minimized to find the best fit for all training points.
-
Stochastic Gradient Descent (SGD)
59:50
To handle massive datasets, optimization uses SGD, which updates parameters by calculating gradients over small subsets of data called mini-batches. This approach is computationally efficient and critical for scaling large models.