Large Language Models (LLMs)

Aug 7, 2026 · 43:21

Local Models: Trust, Control, Optimization — Carter Abdallah, NVIDIA

The panel emphasized that for AI systems to achieve true sovereignty and trust, the ecosystem must be open—encompassing not just models but the entire training stack. Open weights allow users to own their data traces and customize models (e.g., Neotron, Trinity) via post-training environments, enabling specialized performance far exceeding generalized frontier closed APIs. The future points toward local/on-device compute becoming viable for most daily tasks, shifting AI development from relying solely on massive cloud endpoints.

Key takeaways

1. Open Models Ensure Trust and Sovereignty 17:32

   Trust in open models is derived from verifiability: users can inspect the files, matrices, and running code (e.g., implementations from Prime Intellect, VLM, SGLang) rather than relying on unverifiable closed APIs. The ability to run a model locally ensures predictable output regardless of geopolitical or corporate access changes.

2. Specialization Outperforms Generalization 22:00

   Open models allow for deep customization and post-training on specific use cases (e.g., finance automation). This specialization can yield better performance than generalized frontier models while being significantly cheaper to operate, enabling a data flywheel by allowing users to own their output traces.

3. Local Compute is the Next Inflection Point 40:01

   The industry is moving toward local AI capability. The panel predicts that within the next year, open models will achieve capabilities comparable to frontier closed models (e.g., better than Fable), making it possible for most daily tasks to run on personal devices.

• AI Engineer
• Large Language Models (LLMs)
• Open Source AI
• Edge Computing
• Model Optimization
• Data Sovereignty
• Neotron
• Trinity
• Prime Intellect
• Arcee AI (RCAI)
• MDW License

Watch on YouTube Full article

Aug 7, 2026 · 46:01

Compression at the Edge — Chris Alexiuk, NVIDIA

This panel discusses model compression techniques—primarily quantization—that enable running massive Large Language Models (LLMs) on resource-constrained edge devices. Key advancements include formats like NVFP4 and the ability to shrink models dramatically (e.g., GLM 5.2 from 1.5 TB to 250 GB). The discussion emphasizes that successful compression requires understanding model architecture, using advanced methods like Quantization Aware Distillation (QAD), and prioritizing evaluation metrics such as KL divergence over simple accuracy scores.

Key takeaways

1. Model Compression is Critical for Edge AI

   Compression techniques are essential to democratize LLMs, making them viable for local deployment on consumer hardware (e.g., laptops/phones). The goal is enabling powerful models to run without constant reliance on cloud APIs.

2. Advanced Quantization Formats and Techniques 0:04

   NVIDIA's NVFP4 is a specialized 4-bit float format where every group of 16 values shares one FP8 scale. For large models (>20B parameters), Post-Training Quantization (PTQ) works well, while smaller models (<20B) require Quantization Aware Distillation (QAD).

3. Evaluation Focus Shifts to Logits and Architecture 0:08

   Verifying model integrity after compression is complex. The Super Weights paper suggests that quantizing even one number can degrade performance by 20%. Therefore, the preferred signal for evaluation is KL divergence between BF16 and quantized output logits, rather than traditional accuracy scores.

• AI Engineer
• Large Language Models (LLMs)
• Quantization
• Model Compression
• Edge Computing
• AI Architecture
• Nvidia model optimizer space
• Super Weights paper

Watch on YouTube Full article

Aug 6, 2026 · 26:37

How AI agents reproduced ICML 2026 papers

The ICML 2026 Agents Reproduction Challenge was a large-scale community effort involving over 1,200 participants and AI agents attempting to reproduce claims from accepted machine learning papers. The initiative demonstrated the potential for automated reproducibility testing in academic research, finding that while a majority of papers were reproducible (some fully, some via smaller scale experiments), significant flaws were also identified. Key technical takeaways include the use of specialized tools like `tracko` and Hugging Face infrastructure to create fully auditable, machine-readable log books for every reproduction attempt.

Key takeaways

1. Scale of Reproduction Effort 4:18

   The challenge involved 1,200+ participants attempting to reproduce claims from a subset of ICML 2026 papers. A total of 2,200 unique papers were attempted, resulting in approximately 35,000 different claims being judged (Timestamp: ~4:18).

2. Reproducibility Success Rate 12:34

   A majority of the papers looked at were reproducible. Specifically, over 2,000 papers had at least one major claim independently verified (Timestamp: ~6:34).

3. Identification of Flaws and Contested Claims 13:10

   The community found that about 23% of papers could not be fully reproduced as claimed, leading to at least 496 contested or falsified claims. Furthermore, 49 papers were almost fully falsified (Timestamp: ~8:15).

4. Best Practices in Agent Use 15:42

   The 'Best Human in the Loop' award highlighted that effective reproduction requires human intervention to guide agents, especially when evaluating qualitative results (e.g., building a UI to compare quantized images) (Timestamp: ~10:35).

• Hugging Face
• AI Agents
• Machine Learning Reproducibility
• Experiment Tracking
• Build Engineering
• Large Language Models (LLMs)
• ICML 2026 Agents Reproduction Challenge Website
• tracko

Watch on YouTube Full article

Aug 4, 2026 · 30:57

Hugging Face Journal Club: Scaling Laws for Pre-training & RL

The discussion analyzes a paper proposing a joint scaling law for pre-training and Reinforcement Learning (RL), which models how compute allocation across these stages impacts downstream task performance. Key findings suggest that increasing compute allocated to pre-training significantly boosts the model's ability on specific tasks (Pass 1) but has diminishing returns on generalized capability (Pass K). The analysis highlights the importance of optimizing the trade-off between SFT and RL compute budgets.

Key takeaways

1. Pre-training vs. RL Compute Allocation 15:20

   The primary takeaway is that increasing compute allocated to pre-training leads to higher performance on downstream tasks (Pass 1). Conversely, while RL improves Pass 1, the model's generalized capability (Pass K) remains relatively stable regardless of the pre-training scale.

2. Scaling Laws and Model Size 24:45

   When fixing the total compute budget, training smaller models for longer is generally more effective than attempting to train larger models, contradicting simple Chinchilla scaling assumptions in certain contexts.

• Hugging Face
• Scaling Laws
• Large Language Models (LLMs)
• Reinforcement Learning (RL)
• Model Optimization
• Compute Budgeting
• Understanding Reasoning from Pretraining to Post-Training

Watch on YouTube Full article

Aug 4, 2026 · 56:48

Between the Layers– Interpreting Large Language Models - Michelle Frost - NDC Toronto 2026

This talk provides a deep dive into AI Interpretability, arguing that for Large Language Models (LLMs) to achieve reliability and trust, we must move beyond external explanations (explainability) and understand the internal mechanisms (interpretability). The speaker outlines advanced research—particularly Anthropic's work on mechanistic interpretability—that focuses on decomposing model activations into sparse, single-meaning features. This shift allows engineers to treat LLMs not as black boxes, but as auditable systems that can be tested, monitored, and debugged at the circuit level.

Key takeaways

1. Interpretability vs. Explainability 1:48

   Interpretability is the 'inside view' (understanding which internal parts wrote the answer), while explainability is the 'outside view' (a human-readable reason for the output). Mechanistic evidence comes from ablating suspected units and observing predictable behavior changes, whereas a salency map or local surrogate provides an account.

2. The Need for Operational Trust 3:50

   To build trust in LLMs, we must make concepts like truthfulness and calibration operational. This requires understanding the model's internal mechanisms to ensure consistency (robustness) and prevent unsafe answers or agreement bias (syncopancy).

3. The Interpretability Litmus Test 5:20

   A system should achieve 'understanding' if it can be able to: Name the units, Test their causal importance, Predict counterfactual failures (e.g., misinformation), Instrument them in production, and allow another engineer to Reproduce findings.

4. Modern Mechanistic Approaches 9:20

   Advanced research uses tools like Sparse Autoencoders (SAEs) and overcomplete dictionaries to decompose model activations into 'features'—internal signals that track a single concept (monosemanticity). This allows for the mapping of complex behaviors to specific, nameable circuits.

5. The Evolution of AI Safety 11:20

   Interpretability is presented as 'ethical infrastructure.' By understanding internal mechanisms (e.g., default circuits that trigger refusals), we can move from simply measuring outputs to actively debugging and governing the system's behavior.

• NDC Conferences
• AI Interpretability
• Large Language Models (LLMs)
• Mechanistic AI
• Causal Inference
• Model Governance and Ethics
• Anthropic Research Papers
• Neuronipedia

Watch on YouTube Full article

Aug 4, 2026 · 1:01:23

Datadog Deleted All Its AI Context. It Worked.

Datadog detailed its journey scaling AI coding agents across 4,000 engineers, highlighting that performance improvements were achieved by deleting years of accumulated context files (context rot). The discussion emphasizes the critical role of building dedicated evaluation (evals) platforms to make data-driven decisions about model selection and agent capabilities. Key findings include using evals to replay historical PRs for code review guardrails and adapting hiring practices away from traditional LeetCode interviews toward real-world, large codebase tasks.

Key takeaways

1. Context Rot: Deleting Context Improved Performance 2:49

   The team found that deleting old, accumulated AI context files (written prior to models like Sonnet 3.5) led to better evaluation scores, demonstrating 'context rot'—where historical information becomes irrelevant or harmful to the agent's performance.

2. Evals for Code Review and Regression Testing 5:49

   The first concrete application of evals was building a platform that replays historical PRs known to have caused incidents, allowing agents to act as a last guardrail before production deployment.

3. Shift from Productivity to Ambition 59:02

   The core lesson learned is that the goal of AI adoption should not solely be increasing productivity, but rather 'increasing ambition'—enabling teams to attempt and validate more complex ideas.

4. AI-Driven Interviewing 53:25

   The process of hiring is evolving away from low-signal LeetCode interviews toward real-world scenarios that require AI to navigate and understand large, complex codebases.

• AI Native Dev
• AI Agentic Workflows
• Evaluation Engineering (Evals)
• Code Review Automation
• Large Language Models (LLMs)
• Software Development Lifecycle (SDLC)
• Datadog
• Cursor
• Claude Code
• GLM 5.2
• Tessl

Watch on YouTube Full article

Aug 3, 2026 · 1:09:42

Stanford CS329A Self-Improving AI Agents | Part 1 | Course Overview

This course overview details the evolution of Large Language Models (LLMs) from simple predictive models to complex, self-improving agents. Key advancements include understanding scaling laws based on parameters, compute, and data size; developing advanced reasoning capabilities through Chain-of-Thought (CoT); and moving beyond single-turn chat interactions into sophisticated agentic workflows. The focus shifts to building systems that can plan, interact with external tools, self-correct using verifiers, and achieve end-to-end goals in domains like software engineering and data analysis.

Key takeaways

1. LLM Scaling Laws 2:20

   Model performance improves significantly by scaling up parameters, compute (compute on the x-axis), or dataset size. This exponential growth has been foundational for models like GPT-3 and PaLM.

2. Agentic Workflows vs. Chatbots 17:05

   Modern LLM applications are moving from single-turn chat formats to complex agentic workflows (e.g., Deep Research, Cloud Code). These systems can take a goal, plan steps, interact with environments/tools, and correct their actions until the goal is achieved.

3. The Role of Verifiers 22:00

   In agentic systems, verifiers (like unit tests for code) are crucial feedback mechanisms. They allow models to verify generated outputs and guide self-correction, which is essential in verifiable domains like math and coding.

4. Self-Improvement Loop 27:30

   The most advanced frontier involves combining test time scaling (e.g., repeated sampling) with fine-tuning to generate synthetic, high-quality data. This creates a self-improving loop where the model trains on its own generated solutions.

• Stanford Online
• Large Language Models (LLMs)
• Scaling Laws
• Chain-of-Thought Reasoning
• Reinforcement Learning from Human Feedback (RLHF)
• Agentic Systems
• Workflow Orchestration
• Verifiers and Self-Correction
• cs329a.stanford.edu
• XCS329 graduate course
• Large Language Monkeys project

Watch on YouTube Full article

Aug 3, 2026 · 1:03:21

Stanford CS329A Self-Improving AI Agents | Part 2 | Test-Time Compute Scaling

The lecture details advanced methods for improving Large Language Model (LLM) performance through 'inference scaling' or 'test-time compute scaling,' rather than relying solely on expensive pre-training. Key techniques include repeated sampling (Large Language Monkeys), which shows that coverage follows a predictable power law with the number of samples. The discussion highlights the critical need for robust verification mechanisms to bridge the generation-verification gap, and concludes by introducing the Arkon framework, an architecture search method that optimizes complex inference pipelines using components like Fusion, Critic, and Ranker.

Key takeaways

1. Inference Scaling Paradigm Shift 1:45

   LLM capability can be significantly enhanced at inference time by increasing compute (e.g., repeated sampling) without modifying model parameters or requiring fine-tuning, offering a new paradigm compared to traditional pre-training and fine-tuning.

2. Repeated Sampling Effectiveness 2:40

   By repeatedly querying the same problem (e.g., using Llama 3-8b or DeepSeek), selecting the correct response among candidates, models can achieve performance comparable to larger proprietary models like GPT-4o.

3. The Role of Verification 7:50

   For repeated sampling to be effective, automated verification is crucial. The 'generation-verification gap' describes the large difference between the best possible outcome (Oracle selection) and what can be achieved using simple methods like majority voting.

4. Advanced Scaling Architectures 20:30

   The Arkon framework treats inference scaling as an architecture design problem, optimizing the combination of techniques (e.g., Fusion, Critic, Ranker) to maximize accuracy given a limited compute budget.

• Stanford Online
• Large Language Models (LLMs)
• AI Agents
• Scaling Laws
• Inference Optimization
• Machine Learning Architecture Search
• Large Language Monkeys paper
• SWE-bench
• KernelBench

Watch on YouTube Full article

Aug 3, 2026 · 1:12:59

Stanford CS329A Self-Improving AI Agents | Part 3 | Robust Verification

This lecture traces the evolution of verification methods for Large Language Model (LLM) outputs, aiming to close the generation-verification gap. The discussion covers four major research advancements: training verifiers using outcome-based reward models (GSM8K), implementing process-based supervision via PRMs (PRM800K), automating annotation with Math-Shepherd, and finally, combining multiple weak verifiers into a robust system using Weaver. Key findings highlight that process supervision is generally superior to outcome supervision, and ensembling multiple specialized verifiers significantly boosts model accuracy on complex reasoning tasks.

Key takeaways

1. Process Supervision vs. Outcome Supervision 26:00

   While outcome-based reward models (ORM) only check the final answer's correctness, process-based reward models (PRM) assign rewards per step of reasoning. PRMs are superior because they manage false positives better and encourage interpretable, human-endorsed steps [2:36:00].

2. Ensembling Weak Verifiers (Weaver) 23:02

   The Weaver approach combines multiple weak verifiers (e.g., LLM judges, reward models) using techniques like Naive Bayes or logistic regression to create a single, highly capable verifier. This method significantly improves performance by leveraging the collective signal of diverse sources [3:42:00].

3. Data Efficiency and Scaling 26:00

   PRMs are shown to be more data-efficient than ORMs. Furthermore, the lecture demonstrates that scaling verification by increasing the number of verifiers (rather than just sampling more completions) can improve results while maintaining computational efficiency [3:42:00].

4. The Role of Self-Improvement 17:36

   Advanced techniques involve using the model itself to generate data (e.g., Math-Shepherd) and then training a PRM on this synthetic, semi-automated data, allowing the system to self-improve its reasoning capabilities [2:56:00].

• Stanford Online
• Large Language Models (LLMs)
• Verification
• Reward Modeling
• Reinforcement Learning (RL)
• Ensemble Methods
• Test-Time Scaling
• Reasoning Benchmarks
• GSM8K Dataset
• PRM800K Dataset
• Stanford CS329A Self-Improving AI Agents

Watch on YouTube Full article

Aug 3, 2026 · 1:12:39

Stanford CS329A Self-Improving AI Agents | Part 6 | Train Time Scaling/Scaling RL

This lecture explores advanced techniques for scaling Large Language Model (LLM) reasoning capabilities through 'train-time scaling' and Reinforcement Learning (RL). The discussion covers three key papers: STaR (Self-Taught Reasoner), DeepSeekMath, and DAPO. Key findings demonstrate that smaller models can achieve high accuracy on complex benchmarks like AIME by leveraging structured training data generation (STaR) or advanced RL algorithms (DAPO/GRPO). The core insight is that closing the feedback loop—using model outputs to improve the model itself—is crucial for boosting reasoning, especially in domains with verifiability.

Key takeaways

1. Train-Time Scaling vs. Test-Time Scaling 2:00

   While test-time scaling (inference-based techniques like majority voting) improves accuracy by sampling outputs, train-time scaling uses the model's own filtered outputs to fine-tune and improve the model weights directly, creating a powerful closed feedback loop.

2. STaR Boosts Reasoning via Rationalization 6:30

   The STaR method bootstraps reasoning by generating solutions on a small set of examples. It filters for correct answers and then generates rationales (hints) for incorrect attempts, allowing the model to learn from failed paths iteratively.

3. GRPO Addresses RL Memory Constraints 10:05

   DeepSeekMath introduced Group Relative Policy Optimization (GRPO), an efficient alternative to PPO that reduces memory overhead by using a group baseline instead of maintaining multiple policy copies, enabling scaling RL to larger models.

4. DAPO Stabilizes Complex Reasoning 17:30

   DAPO addresses training instability in long chain-of-thought reasoning by implementing asymmetric clipping (allowing bigger increases) and dynamic sampling (filtering out zero or one reward groups to maintain a useful gradient signal).

• Stanford Online
• Reinforcement Learning (RL)
• Large Language Models (LLMs)
• Reasoning Capabilities
• Self-Improving Agents
• Train-Time Scaling
• Model Optimization
• Stanford CS329A Self-Improving AI Agents
• Agentic AI professional education program

Watch on YouTube Full article

Aug 2, 2026 · 17:25

When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI

The talk argues that the concept of 'benchmaxxing'—where models are trained excessively on benchmarks in ways that deviate from real-world utility—is rampant across AI. The speaker identifies several critical anti-patterns in benchmark creation, including contamination (memorizing test data), reward hacking, and misalignment between prompts and verifiers. Heiner advocates for a shift toward high-fidelity human evaluation, domain expertise, rigorous Quality Control (QC), and ensuring that benchmarks are aspirational artifacts reflecting true user needs rather than arbitrary scores.

Key takeaways

1. Benchmaxxing is an industry problem

   The existence of 'benchmaxxing' indicates a gap between benchmark scores and real-world performance. This phenomenon is driven by incentives, poor methodologies, and the need for easily digestible metrics in a hyped market.

2. Contamination is often default 7:21

   Contamination occurs when models memorize public questions and answers from the internet (e.g., Opus memorizing SWE-bench contents), making scores reflect recall rather than generalized ability.

3. Verifiers must be fully aligned with prompts 10:23

   Misalignment between the prompt and the verifier (e.g., asking for no commas but accepting Hindi) or using hard-coded string matches introduces noise, leading to misleading scores.

4. High-quality benchmarks require domain expertise

   Creating effective benchmarks requires not just technical experts (e.g., doctors for a medical benchmark) but also those with business and regulatory sense to understand the deployment environment.

5. Human evaluation is necessary, despite cost

   The core value remains human preference. While expensive, maximizing quality requires paying for good workers rather than minimizing costs to achieve scalable metrics.

• AI Engineer
• AI Benchmarking
• Large Language Models (LLMs)
• Evaluation Metrics
• Data Contamination
• Build Process Quality Control
• SWE-bench
• Opus 4.8
• Haiku and Fable
• Apex
• Hemingway bench

Watch on YouTube Full article

Jul 31, 2026 · 18:05

What's Next After RLHF? — Diogo Almeida, TypeSafe AI

Diogo Almeida argues that current Large Language Models (LLMs), particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are fundamentally optimized for 'assistance'—meaning they prioritize pleasing the human user. This optimization leads to overpromising and a lack of reliability in autonomous tasks. The next frontier, he asserts, is not simply better code generation or enhanced chat capabilities, but achieving true automation by optimizing models for verifiable rewards and calibrated decision-making, moving beyond the need for constant human oversight.

Key takeaways

1. The Limitation of RLHF 12:11

   RLHF trains LLMs to optimize for human preference (engagement), which makes them excellent assistants but poor autonomous agents. The goal is to please the user, not necessarily to execute a task correctly in a background server environment [7:31].

2. Assistance vs. Automation 5:14

   The core divide in modern AI is between 'assistance' (where the human remains in the loop) and 'automation' (where the system operates autonomously with real stakes). Current models are optimized for the former, making them unreliable for critical business decisions [3:14].

3. The Path to True Automation

   Future AI must shift its optimization target from human preference to verifiable rewards and calibrated decision-making. This requires redesigning the entire AI stack for reliability, moving beyond current LLM post-training methods like RLHF or even RLVR [15:43].

4. The Importance of Software Expressibility 17:15

   True automation requires smarter software that is more expressive than current SaaS models. The focus must shift from merely automating the writing of code to solving extremely rote, simple tasks that can be done repeatedly and reliably by a computer [10:35].

• AI Engineer
• Large Language Models (LLMs)
• Reinforcement Learning
• AI Architecture
• Automation Theory
• Software Engineering
• TypeSafe AI
• GPT-4 / ChatGPT

Watch on YouTube Full article

Jul 31, 2026 · 19:05

Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI

In an era of increasing compute scarcity—evidenced by rising H100 prices and skyrocketing token usage—data quality has emerged as the critical 'compute multiplier' for model training. The presentation outlines a systematic approach to data enhancement through four stages: Clean, Curate, Create, and Compose. By maximizing the signal per token (marginal information gain), organizations can achieve performance levels comparable to models trained with vastly more compute budgets. Practical applications include improving Vision Language Models (VLMs) and enhancing multilingual capabilities using proprietary or public datasets.

Key takeaways

1. Compute Scarcity Drives Data Focus

   The availability of compute is becoming increasingly constrained, leading to market actions like Google capping Meta's Gemini usage and OpenAI selling token futures. This necessitates a shift in focus from raw compute power to data quality.

2. Data Quality as Compute Multiplier 3:39

   Improving data quality allows for dramatically better performance (blue curve) compared to training with the same limited compute budget (gray curve), effectively simulating much larger compute investments.

3. The Four C's of Data Enhancement 5:48

   Data improvement is achieved through a pipeline: Clean (heuristic filters, decontamination), Curate (quality classifiers, redundancy reduction), Create (synthetic data generation/rephrasing), and Compose (sequencing across multiple training stages).

4. Cross-Lingual Benefits from Curation 15:24

   Curating English data can positively benefit non-English performance, demonstrating cross-lingual transfer. Similarly, curating non-English data benefits English performance.

• AI Engineer
• AI Model Training
• Data Engineering
• Large Language Models (LLMs)
• Compute Optimization
• DatologyAI
• H100
• Gemini
• Meta's Gemini usage cap
• Quen 3.5b
• Thomson Reuters
• RCI Trendy Large

Watch on YouTube Full article

Jul 31, 2026 · 1:18:56

Stanford CS229 Machine Learning | Spring 2026 | Lecture 20: GMM (EM), PCA

This lecture provides an advanced deep dive into training Large Language Models (LLMs) using Reinforcement Learning (RL). It reviews Policy Gradient methods, detailing the mathematical derivations and limitations. The core focus shifts to Proximal Policy Optimization (PPO), a critical algorithm for stabilizing RL updates by utilizing importance sampling ratios and clipping mechanisms. Finally, the lecture applies these concepts to LLM generation, explaining how Chain-of-Thought (CoT) prompting can be formalized as an MDP problem solved via PPO/SISO.

Key takeaways

1. Policy Gradient Theory 20:40

   The policy gradient estimator is necessary because the dependency on parameters ($ heta$) is complex. The fundamental property that $ abla_{ heta} ext{E}_{ ext{P}_{ heta}}[ abla_{ heta} ext{log } ext{P}_{ heta}(a|s)]$ equals zero shows that without a reward function, there are no preferences to optimize for.

2. Proximal Policy Optimization (PPO) 26:40

   PPO is designed to stabilize RL training by using importance sampling and clipping the objective function. This prevents the new policy ($ heta$) from deviating too far from the old policy ($ heta_{old}$), which helps maintain stable learning.

3. LLM Generation as an MDP 1:01:40

   The LLM generation process is modeled as a Markov Decision Process (MDP). The state ($s_t$) includes the history, and the action ($a_t$) is the next generated token. The reward function is typically applied only at the end of the trajectory based on whether the final answer matches the ground truth.

4. Chain-of-Thought (CoT) Training 1:05:00

   To train models for complex reasoning, RL can be used to reward the entire trajectory based on the final answer's correctness. This approach bypasses the need for explicit labeling of the internal 'thinking tokens,' focusing only on verifiable outcomes.

• Stanford Online
• Reinforcement Learning
• Large Language Models (LLMs)
• Policy Gradient
• Proximal Policy Optimization (PPO)
• Chain-of-Thought (CoT)
• Markov Decision Process (MDP)
• Stanford CS229 Machine Learning
• Stanford AI Programs Info

Watch on YouTube Full article

Jul 31, 2026 · 1:17:32

Stanford CS229 Machine Learning | Spring 2026 | Lecture 14: Transformers, In-Context Learning

This lecture provides a deep technical dive into Large Language Models (LLMs), focusing on the Transformer architecture. Key concepts covered include subword tokenization (using techniques like Byte Pair Encoding - BPE) to convert text into numerical IDs, and modeling language distribution using auto-regressive conditional probabilities. The core mechanism is the self-attention layer, which uses Query (Q), Key (K), and Value (V) vectors to determine dependencies between tokens. A critical technical discussion centers on computational efficiency: standard attention has a quadratic $O(T^2)$ dependency on sequence length ($T$), leading to memory and computation bottlenecks for long contexts. Solutions like masking and specialized variants (e.g., Flash Attention) are discussed as methods to maintain performance while managing resource constraints.

Key takeaways

1. Auto-Regressive Modeling 1:35

   LLMs model language distribution by decomposing the joint probability into a product of conditional probabilities: $P(X_t | X_{1...t-1})$. This requires generating tokens sequentially, making the process auto-regressive.

2. Subword Tokenization (BPE) 3:30

   To handle rare or long words efficiently, models use subword tokenization (e.g., BPE), which breaks down text into smaller units (tokens). This allows leveraging common prefixes and suffixes to understand novel words.

3. Self-Attention Mechanism 11:10

   The attention mechanism calculates the relevance between all tokens in a sequence using Query ($Q$), Key ($K$), and Value ($V$) vectors via an inner product: $ ext{Softmax}(Q K^T / ext{scale}) V$. The output is a weighted linear combination of $V$ vectors.

4. Causal Masking for Auto-regression 21:20

   To ensure the model only predicts based on previous tokens (required for generation), a masking technique is applied to the attention matrix, setting all future dependencies to $- ext{infinity}$ before applying Softmax.

5. Computational Bottleneck ($O(T^2)$) 30:00

   The standard self-attention mechanism has a computational and memory complexity that scales quadratically with sequence length $T$ (i.e., $O(T^2)$). This is the primary limitation for processing very long contexts.

• Stanford Online
• Large Language Models (LLMs)
• Transformer Architecture
• Self-Attention Mechanism
• Auto-Regressive Generation
• Byte Pair Encoding (BPE)
• Computational Complexity
• Stanford CS229 Machine Learning
• OpenAI Tokenizer Playground

Watch on YouTube Full article

Jul 31, 2026 · 18:07

Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning

The talk discusses scaling AI agents to solve long-horizon problems, arguing that success requires a shift in mindset from merely increasing context windows to improving environment quality and algorithmic efficiency. Key technical advancements include using value models (critics) to reduce gradient variance and employing techniques like compaction and bootstrapping to manage sparse rewards over extended trajectories. The speakers emphasize that real-world complexity and robust simulation environments are more critical than simply having larger base models.

Key takeaways

1. Long Horizon is a Mindset, Not Just an Engineering Problem 15:12

   Solving major human challenges (e.g., curing cancer) requires patience and thinking in long timeframes, necessitating a fundamental shift in how AI systems are designed for sustained coherence.

2. RLHF is Crucial for Productizing LLMs 4:00

   The breakthrough that made LLMs usable was not the base model size, but the application of Reinforcement Learning from Human Feedback (RLHF), which provided necessary alignment and structure.

3. Value Models Mitigate Long-Horizon Challenges 16:56

   To handle long trajectories, value models (critics) are essential for reducing gradient variance and facilitating credit assignment, which is necessary when rewards are sparse.

4. Simulation Quality Trumps Context Window Size

   The failure of frontier models in real-world tasks (like trading football matches) demonstrated that scaling requires better, more complex environments and simulations, not just larger context windows.

• AI Engineer
• Reinforcement Learning
• Large Language Models (LLMs)
• AI Scaling
• Compute Optimization
• Simulation Environments
• openreward.ai
• General Reasoning (GR)

Watch on YouTube Full article

Jul 31, 2026 · 1:15:57

Stanford CS229 Machine Learning | Spring 2026 | Lecture 12: Representation Learning

The lecture provides a deep dive into advanced generative and adaptation techniques, covering Diffusion Models, Foundation Model paradigms, and Representation Learning. Key technical focus areas include deriving the loss function for diffusion models by comparing true and predicted mean reconstructions ($\mu_{ ilde{t}, t}$ vs $\mu_{ heta, t}$). For large language models (LLMs), the discussion highlights the shift toward using massive unlabelled data for pre-training and adapting via zero-shot or few-shot methods. Finally, it details advanced parameter efficiency techniques like Linear Probing and Low-Rank Adaptation (LoRA), which are crucial for managing memory and compute when deploying large models across multiple users.

Key takeaways

1. Diffusion Model Training Objective 33:55

   The training objective is derived from the lower bound on $\log p_ heta(x_0)$, simplifying to minimizing a sum of terms ($\mathcal{L}_{t-1}$) that compares the mean predicted reconstruction ($\mu_{ heta, t}$) with the true process mean ($\mu_{ ilde{t}, t}$). This loss function is used to train the model to predict noise/reconstruct missing information at each time step.

2. Foundation Model Paradigm Shift 1:02:00

   Modern LLMs utilize a paradigm shift involving pre-training on massive unlabelled data and adapting via zero-shot or few-shot learning, reducing the need for collecting extensive downstream labeled datasets. The goal is to build a general foundation that can be adapted to unlimited tasks.

3. Low-Rank Adaptation (LoRA) Efficiency 1:15:00

   To adapt massive pre-trained models without changing all parameters ($\theta$), LoRA restricts the weight update matrix $\Delta W$ to a low rank ($A B$). This significantly reduces the number of trainable parameters and is critical for memory efficiency, especially when serving multiple users.

• Stanford Online
• Generative Models
• Diffusion Models
• Foundation Models
• Large Language Models (LLMs)
• Representation Learning
• Low-Rank Adaptation (LoRA)
• Stanford CS229 Machine Learning
• Stanford Online AI Programs

Watch on YouTube Full article

Jul 31, 2026 · 17:42

Verifiable Environments for AI in Biology — Kenny Workman, LatchBio

Kenny Workman discusses the massive data challenge in modern biology (e.g., single cell experiments yielding 2–6 terabytes per run) and proposes treating experimental biology as a 'verifiable substrate' for AI training. LatchBio is positioning itself as an agent engineering lab, building benchmarks like 'spatial bench' to train agents on scientific reasoning rather than just memorized knowledge. The core argument is that verifiable data analysis pipelines (modeled after code execution) are necessary because frontier models currently lack the capability to reliably extract scientific insight from complex, real-world biological datasets.

Key takeaways

1. Data Volume and Challenge

   Modern techniques like single cell biology (2–6 TB per run) and spatial biology generate data volumes that exceed consumer storage capacity, necessitating automated analysis tools. The output is often a 'giant matrix of numbers' requiring complex processing.

2. Verifiable Substrate for Science 4:04

   The goal is to treat biological data analysis as an executable substrate, similar to how code provides verifiability in software. This allows the creation of natural benchmarks and measurable progress metrics.

3. Benchmarking Scientific Reasoning 5:23

   Benchmarks must move beyond simple Q&A settings to measure complex, multi-step scientific reasoning. LatchBio developed 'spatial bench' (146 problems) which requires deterministic graders and verifiable outcomes based on analysis DAGs.

4. Agent Limitations and Progress 9:00

   Frontier models cannot be trusted with real-world scientific work because they are missing the capability between knowing biology and writing code. The process requires iterative human verification and developing tasks that force rigorous reasoning.

• AI Engineer
• AI Agent Engineering
• Spatial Biology
• Scientific Benchmarking
• Data Analysis Pipelines
• Large Language Models (LLMs)
• LatchBio
• spatial bench

Watch on YouTube Full article

Jul 31, 2026 · 59:49

From LLM Theory to Practical Agentic Implementations - Seth Juarez

The talk demystifies Large Language Models (LLMs), arguing that their power does not come from inherent intelligence but from engineered 'harnesses' and fundamental primitives. The speaker outlines a progression of techniques—from simple completions to complex agentic loops—that allow developers to build robust, reliable AI systems by controlling the model's input context, output structure, and execution flow. Key concepts include using structured JSON for control flow, implementing tool calling via runtimes (not the LLM itself), establishing guardrails, and orchestrating multi-step workflows through shared 'threads.'

Key takeaways

1. LLMs are Next Token Predictors 1:42

   At its core, an LLM is a giant mathematical function that predicts the next token based on input tokens. The perceived intelligence comes from the surrounding runtime logic (the harness), not the model itself.

2. The Agentic Loop is the Core Primitive 21:20

   True agency is achieved by implementing an 'agentic loop'—a while loop in the runtime that continuously checks for and executes tool calls, feeding the results back into the context until no more tools are needed. This is the fundamental unit of agency.

3. Guardrails and Bindings Enhance Reliability 35:05

   To prevent misuse or incorrect execution (e.g., unauthorized memory access), developers must implement guardrail functions in their runtime logic. Furthermore, 'bindings' allow injecting fixed parameters (like `user ID`) into function calls, removing the burden of context-dependent reasoning from the LLM.

4. Composition Workflow for Multi-Step Agents 50:50

   Complex tasks are solved by chaining multiple specialized agents or prompts (e.g., Research $ ightarrow$ Draft $ ightarrow$ Review). All these components share a single, persistent 'thread' context to maintain state and pass control flow sequentially.

• NDC Conferences
• Large Language Models (LLMs)
• Agentic Systems
• Prompt Engineering
• Tool Calling
• State Management
• Software Architecture
• Prompty
• Seth Juarez Draft Flow Repo

Watch on YouTube Full article

Jul 31, 2026 · 37:37

AI Security Costs Rise: Cost of a Data Breach Report & Claude Opus 5

The discussion analyzes the rapidly escalating security risks posed by AI, noting that while attackers find it cheaper and easier to launch attacks using frontier models without proper guardrails, defenders face increasing costs in prevention. Key technical discussions covered include identifying top vulnerabilities (Model Inversion and Prompt Injection), critiquing new LLM releases like Claude Opus 5 for performance inconsistencies, and exploring the concept of AI as an 'extended mind' through daily rituals. The session also provided a high-level explanation of LLMs, emphasizing that future software development will increasingly rely on higher levels of abstraction rather than low-level code.

Key takeaways

1. AI is driving the cybersecurity cost increase 2:15

   The IBM Cost of a Data Breach Report 2026 highlights that AI dominates all sections, increasing investment intention from 64% to 85%. Top vulnerabilities include Model Inversion ($6-$7 million) and Prompt Injection ($4.89 million).

2. Guardrails and Policy are critical for AI safety 4:10

   To mitigate risks, the focus must shift to treating LLM agents as first-class citizens, requiring robust guardrails, identity management, proper access control, and encryption at rest.

3. LLMs are evolving toward higher abstraction 6:15

   The history of computing is defined by increasing levels of abstraction (e.g., from assembly to declarative languages like Terraform). Future AI development will follow this trend, allowing users to describe desired outcomes rather than specific steps.

4. The business case for 'extended mind' AI 6:25

   Midjourney acquiring the astrology app CoStar suggests a strategic move to integrate AI into daily, ritualistic life patterns, making it an 'extended mind' rather than just a separate tool.

• IBM Technology
• Cybersecurity
• Large Language Models (LLMs)
• AI Ethics and Safety
• Software Architecture
• Data Privacy
• IBM Cost of a Data Breach Report 2026
• What Does AI Look Like?

Watch on YouTube Full article