Scaling Laws

Aug 4, 2026 · 30:57

Hugging Face Journal Club: Scaling Laws for Pre-training & RL

The discussion analyzes a paper proposing a joint scaling law for pre-training and Reinforcement Learning (RL), which models how compute allocation across these stages impacts downstream task performance. Key findings suggest that increasing compute allocated to pre-training significantly boosts the model's ability on specific tasks (Pass 1) but has diminishing returns on generalized capability (Pass K). The analysis highlights the importance of optimizing the trade-off between SFT and RL compute budgets.

Key takeaways

1. Pre-training vs. RL Compute Allocation 15:20

   The primary takeaway is that increasing compute allocated to pre-training leads to higher performance on downstream tasks (Pass 1). Conversely, while RL improves Pass 1, the model's generalized capability (Pass K) remains relatively stable regardless of the pre-training scale.

2. Scaling Laws and Model Size 24:45

   When fixing the total compute budget, training smaller models for longer is generally more effective than attempting to train larger models, contradicting simple Chinchilla scaling assumptions in certain contexts.

• Hugging Face
• Scaling Laws
• Large Language Models (LLMs)
• Reinforcement Learning (RL)
• Model Optimization
• Compute Budgeting
• Understanding Reasoning from Pretraining to Post-Training

Watch on YouTube Full article

Aug 3, 2026 · 1:09:42

Stanford CS329A Self-Improving AI Agents | Part 1 | Course Overview

This course overview details the evolution of Large Language Models (LLMs) from simple predictive models to complex, self-improving agents. Key advancements include understanding scaling laws based on parameters, compute, and data size; developing advanced reasoning capabilities through Chain-of-Thought (CoT); and moving beyond single-turn chat interactions into sophisticated agentic workflows. The focus shifts to building systems that can plan, interact with external tools, self-correct using verifiers, and achieve end-to-end goals in domains like software engineering and data analysis.

Key takeaways

1. LLM Scaling Laws 2:20

   Model performance improves significantly by scaling up parameters, compute (compute on the x-axis), or dataset size. This exponential growth has been foundational for models like GPT-3 and PaLM.

2. Agentic Workflows vs. Chatbots 17:05

   Modern LLM applications are moving from single-turn chat formats to complex agentic workflows (e.g., Deep Research, Cloud Code). These systems can take a goal, plan steps, interact with environments/tools, and correct their actions until the goal is achieved.

3. The Role of Verifiers 22:00

   In agentic systems, verifiers (like unit tests for code) are crucial feedback mechanisms. They allow models to verify generated outputs and guide self-correction, which is essential in verifiable domains like math and coding.

4. Self-Improvement Loop 27:30

   The most advanced frontier involves combining test time scaling (e.g., repeated sampling) with fine-tuning to generate synthetic, high-quality data. This creates a self-improving loop where the model trains on its own generated solutions.

• Stanford Online
• Large Language Models (LLMs)
• Scaling Laws
• Chain-of-Thought Reasoning
• Reinforcement Learning from Human Feedback (RLHF)
• Agentic Systems
• Workflow Orchestration
• Verifiers and Self-Correction
• cs329a.stanford.edu
• XCS329 graduate course
• Large Language Monkeys project

Watch on YouTube Full article

Aug 3, 2026 · 1:03:21

Stanford CS329A Self-Improving AI Agents | Part 2 | Test-Time Compute Scaling

The lecture details advanced methods for improving Large Language Model (LLM) performance through 'inference scaling' or 'test-time compute scaling,' rather than relying solely on expensive pre-training. Key techniques include repeated sampling (Large Language Monkeys), which shows that coverage follows a predictable power law with the number of samples. The discussion highlights the critical need for robust verification mechanisms to bridge the generation-verification gap, and concludes by introducing the Arkon framework, an architecture search method that optimizes complex inference pipelines using components like Fusion, Critic, and Ranker.

Key takeaways

1. Inference Scaling Paradigm Shift 1:45

   LLM capability can be significantly enhanced at inference time by increasing compute (e.g., repeated sampling) without modifying model parameters or requiring fine-tuning, offering a new paradigm compared to traditional pre-training and fine-tuning.

2. Repeated Sampling Effectiveness 2:40

   By repeatedly querying the same problem (e.g., using Llama 3-8b or DeepSeek), selecting the correct response among candidates, models can achieve performance comparable to larger proprietary models like GPT-4o.

3. The Role of Verification 7:50

   For repeated sampling to be effective, automated verification is crucial. The 'generation-verification gap' describes the large difference between the best possible outcome (Oracle selection) and what can be achieved using simple methods like majority voting.

4. Advanced Scaling Architectures 20:30

   The Arkon framework treats inference scaling as an architecture design problem, optimizing the combination of techniques (e.g., Fusion, Critic, Ranker) to maximize accuracy given a limited compute budget.

• Stanford Online
• Large Language Models (LLMs)
• AI Agents
• Scaling Laws
• Inference Optimization
• Machine Learning Architecture Search
• Large Language Monkeys paper
• SWE-bench
• KernelBench

Watch on YouTube Full article