Large Language Models (LLMs)

Aug 3, 2026 · 1:03:21

Stanford CS329A Self-Improving AI Agents | Part 2 | Test-Time Compute Scaling

The lecture details advanced methods for improving Large Language Model (LLM) performance through 'inference scaling' or 'test-time compute scaling,' rather than relying solely on expensive pre-training. Key techniques include repeated sampling (Large Language Monkeys), which shows that coverage follows a predictable power law with the number of samples. The discussion highlights the critical need for robust verification mechanisms to bridge the generation-verification gap, and concludes by introducing the Arkon framework, an architecture search method that optimizes complex inference pipelines using components like Fusion, Critic, and Ranker.

Key takeaways

1. Inference Scaling Paradigm Shift 1:45

   LLM capability can be significantly enhanced at inference time by increasing compute (e.g., repeated sampling) without modifying model parameters or requiring fine-tuning, offering a new paradigm compared to traditional pre-training and fine-tuning.

2. Repeated Sampling Effectiveness 2:40

   By repeatedly querying the same problem (e.g., using Llama 3-8b or DeepSeek), selecting the correct response among candidates, models can achieve performance comparable to larger proprietary models like GPT-4o.

3. The Role of Verification 7:50

   For repeated sampling to be effective, automated verification is crucial. The 'generation-verification gap' describes the large difference between the best possible outcome (Oracle selection) and what can be achieved using simple methods like majority voting.

4. Advanced Scaling Architectures 20:30

   The Arkon framework treats inference scaling as an architecture design problem, optimizing the combination of techniques (e.g., Fusion, Critic, Ranker) to maximize accuracy given a limited compute budget.

• Stanford Online
• Large Language Models (LLMs)
• AI Agents
• Scaling Laws
• Inference Optimization
• Machine Learning Architecture Search
• Large Language Monkeys paper
• SWE-bench
• KernelBench

Watch on YouTube Full article

Aug 3, 2026 · 1:12:59

Stanford CS329A Self-Improving AI Agents | Part 3 | Robust Verification

This lecture traces the evolution of verification methods for Large Language Model (LLM) outputs, aiming to close the generation-verification gap. The discussion covers four major research advancements: training verifiers using outcome-based reward models (GSM8K), implementing process-based supervision via PRMs (PRM800K), automating annotation with Math-Shepherd, and finally, combining multiple weak verifiers into a robust system using Weaver. Key findings highlight that process supervision is generally superior to outcome supervision, and ensembling multiple specialized verifiers significantly boosts model accuracy on complex reasoning tasks.

Key takeaways

1. Process Supervision vs. Outcome Supervision 26:00

   While outcome-based reward models (ORM) only check the final answer's correctness, process-based reward models (PRM) assign rewards per step of reasoning. PRMs are superior because they manage false positives better and encourage interpretable, human-endorsed steps [2:36:00].

2. Ensembling Weak Verifiers (Weaver) 23:02

   The Weaver approach combines multiple weak verifiers (e.g., LLM judges, reward models) using techniques like Naive Bayes or logistic regression to create a single, highly capable verifier. This method significantly improves performance by leveraging the collective signal of diverse sources [3:42:00].

3. Data Efficiency and Scaling 26:00

   PRMs are shown to be more data-efficient than ORMs. Furthermore, the lecture demonstrates that scaling verification by increasing the number of verifiers (rather than just sampling more completions) can improve results while maintaining computational efficiency [3:42:00].

4. The Role of Self-Improvement 17:36

   Advanced techniques involve using the model itself to generate data (e.g., Math-Shepherd) and then training a PRM on this synthetic, semi-automated data, allowing the system to self-improve its reasoning capabilities [2:56:00].

• Stanford Online
• Large Language Models (LLMs)
• Verification
• Reward Modeling
• Reinforcement Learning (RL)
• Ensemble Methods
• Test-Time Scaling
• Reasoning Benchmarks
• GSM8K Dataset
• PRM800K Dataset
• Stanford CS329A Self-Improving AI Agents

Watch on YouTube Full article

Aug 3, 2026 · 1:12:39

Stanford CS329A Self-Improving AI Agents | Part 6 | Train Time Scaling/Scaling RL

This lecture explores advanced techniques for scaling Large Language Model (LLM) reasoning capabilities through 'train-time scaling' and Reinforcement Learning (RL). The discussion covers three key papers: STaR (Self-Taught Reasoner), DeepSeekMath, and DAPO. Key findings demonstrate that smaller models can achieve high accuracy on complex benchmarks like AIME by leveraging structured training data generation (STaR) or advanced RL algorithms (DAPO/GRPO). The core insight is that closing the feedback loop—using model outputs to improve the model itself—is crucial for boosting reasoning, especially in domains with verifiability.

Key takeaways

1. Train-Time Scaling vs. Test-Time Scaling 2:00

   While test-time scaling (inference-based techniques like majority voting) improves accuracy by sampling outputs, train-time scaling uses the model's own filtered outputs to fine-tune and improve the model weights directly, creating a powerful closed feedback loop.

2. STaR Boosts Reasoning via Rationalization 6:30

   The STaR method bootstraps reasoning by generating solutions on a small set of examples. It filters for correct answers and then generates rationales (hints) for incorrect attempts, allowing the model to learn from failed paths iteratively.

3. GRPO Addresses RL Memory Constraints 10:05

   DeepSeekMath introduced Group Relative Policy Optimization (GRPO), an efficient alternative to PPO that reduces memory overhead by using a group baseline instead of maintaining multiple policy copies, enabling scaling RL to larger models.

4. DAPO Stabilizes Complex Reasoning 17:30

   DAPO addresses training instability in long chain-of-thought reasoning by implementing asymmetric clipping (allowing bigger increases) and dynamic sampling (filtering out zero or one reward groups to maintain a useful gradient signal).

• Stanford Online
• Reinforcement Learning (RL)
• Large Language Models (LLMs)
• Reasoning Capabilities
• Self-Improving Agents
• Train-Time Scaling
• Model Optimization
• Stanford CS329A Self-Improving AI Agents
• Agentic AI professional education program

Watch on YouTube Full article

Aug 2, 2026 · 17:25

When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI

The talk argues that the concept of 'benchmaxxing'—where models are trained excessively on benchmarks in ways that deviate from real-world utility—is rampant across AI. The speaker identifies several critical anti-patterns in benchmark creation, including contamination (memorizing test data), reward hacking, and misalignment between prompts and verifiers. Heiner advocates for a shift toward high-fidelity human evaluation, domain expertise, rigorous Quality Control (QC), and ensuring that benchmarks are aspirational artifacts reflecting true user needs rather than arbitrary scores.

Key takeaways

1. Benchmaxxing is an industry problem

   The existence of 'benchmaxxing' indicates a gap between benchmark scores and real-world performance. This phenomenon is driven by incentives, poor methodologies, and the need for easily digestible metrics in a hyped market.

2. Contamination is often default 7:21

   Contamination occurs when models memorize public questions and answers from the internet (e.g., Opus memorizing SWE-bench contents), making scores reflect recall rather than generalized ability.

3. Verifiers must be fully aligned with prompts 10:23

   Misalignment between the prompt and the verifier (e.g., asking for no commas but accepting Hindi) or using hard-coded string matches introduces noise, leading to misleading scores.

4. High-quality benchmarks require domain expertise

   Creating effective benchmarks requires not just technical experts (e.g., doctors for a medical benchmark) but also those with business and regulatory sense to understand the deployment environment.

5. Human evaluation is necessary, despite cost

   The core value remains human preference. While expensive, maximizing quality requires paying for good workers rather than minimizing costs to achieve scalable metrics.

• AI Engineer
• AI Benchmarking
• Large Language Models (LLMs)
• Evaluation Metrics
• Data Contamination
• Build Process Quality Control
• SWE-bench
• Opus 4.8
• Haiku and Fable
• Apex
• Hemingway bench

Watch on YouTube Full article

Jul 31, 2026 · 18:05

What's Next After RLHF? — Diogo Almeida, TypeSafe AI

Diogo Almeida argues that current Large Language Models (LLMs), particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are fundamentally optimized for 'assistance'—meaning they prioritize pleasing the human user. This optimization leads to overpromising and a lack of reliability in autonomous tasks. The next frontier, he asserts, is not simply better code generation or enhanced chat capabilities, but achieving true automation by optimizing models for verifiable rewards and calibrated decision-making, moving beyond the need for constant human oversight.

Key takeaways

1. The Limitation of RLHF 12:11

   RLHF trains LLMs to optimize for human preference (engagement), which makes them excellent assistants but poor autonomous agents. The goal is to please the user, not necessarily to execute a task correctly in a background server environment [7:31].

2. Assistance vs. Automation 5:14

   The core divide in modern AI is between 'assistance' (where the human remains in the loop) and 'automation' (where the system operates autonomously with real stakes). Current models are optimized for the former, making them unreliable for critical business decisions [3:14].

3. The Path to True Automation

   Future AI must shift its optimization target from human preference to verifiable rewards and calibrated decision-making. This requires redesigning the entire AI stack for reliability, moving beyond current LLM post-training methods like RLHF or even RLVR [15:43].

4. The Importance of Software Expressibility 17:15

   True automation requires smarter software that is more expressive than current SaaS models. The focus must shift from merely automating the writing of code to solving extremely rote, simple tasks that can be done repeatedly and reliably by a computer [10:35].

• AI Engineer
• Large Language Models (LLMs)
• Reinforcement Learning
• AI Architecture
• Automation Theory
• Software Engineering
• TypeSafe AI
• GPT-4 / ChatGPT

Watch on YouTube Full article

Jul 31, 2026 · 19:05

Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI

In an era of increasing compute scarcity—evidenced by rising H100 prices and skyrocketing token usage—data quality has emerged as the critical 'compute multiplier' for model training. The presentation outlines a systematic approach to data enhancement through four stages: Clean, Curate, Create, and Compose. By maximizing the signal per token (marginal information gain), organizations can achieve performance levels comparable to models trained with vastly more compute budgets. Practical applications include improving Vision Language Models (VLMs) and enhancing multilingual capabilities using proprietary or public datasets.

Key takeaways

1. Compute Scarcity Drives Data Focus

   The availability of compute is becoming increasingly constrained, leading to market actions like Google capping Meta's Gemini usage and OpenAI selling token futures. This necessitates a shift in focus from raw compute power to data quality.

2. Data Quality as Compute Multiplier 3:39

   Improving data quality allows for dramatically better performance (blue curve) compared to training with the same limited compute budget (gray curve), effectively simulating much larger compute investments.

3. The Four C's of Data Enhancement 5:48

   Data improvement is achieved through a pipeline: Clean (heuristic filters, decontamination), Curate (quality classifiers, redundancy reduction), Create (synthetic data generation/rephrasing), and Compose (sequencing across multiple training stages).

4. Cross-Lingual Benefits from Curation 15:24

   Curating English data can positively benefit non-English performance, demonstrating cross-lingual transfer. Similarly, curating non-English data benefits English performance.

• AI Engineer
• AI Model Training
• Data Engineering
• Large Language Models (LLMs)
• Compute Optimization
• DatologyAI
• H100
• Gemini
• Meta's Gemini usage cap
• Quen 3.5b
• Thomson Reuters
• RCI Trendy Large

Watch on YouTube Full article

Jul 31, 2026 · 1:18:56

Stanford CS229 Machine Learning | Spring 2026 | Lecture 20: GMM (EM), PCA

This lecture provides an advanced deep dive into training Large Language Models (LLMs) using Reinforcement Learning (RL). It reviews Policy Gradient methods, detailing the mathematical derivations and limitations. The core focus shifts to Proximal Policy Optimization (PPO), a critical algorithm for stabilizing RL updates by utilizing importance sampling ratios and clipping mechanisms. Finally, the lecture applies these concepts to LLM generation, explaining how Chain-of-Thought (CoT) prompting can be formalized as an MDP problem solved via PPO/SISO.

Key takeaways

1. Policy Gradient Theory 20:40

   The policy gradient estimator is necessary because the dependency on parameters ($ heta$) is complex. The fundamental property that $ abla_{ heta} ext{E}_{ ext{P}_{ heta}}[ abla_{ heta} ext{log } ext{P}_{ heta}(a|s)]$ equals zero shows that without a reward function, there are no preferences to optimize for.

2. Proximal Policy Optimization (PPO) 26:40

   PPO is designed to stabilize RL training by using importance sampling and clipping the objective function. This prevents the new policy ($ heta$) from deviating too far from the old policy ($ heta_{old}$), which helps maintain stable learning.

3. LLM Generation as an MDP 1:01:40

   The LLM generation process is modeled as a Markov Decision Process (MDP). The state ($s_t$) includes the history, and the action ($a_t$) is the next generated token. The reward function is typically applied only at the end of the trajectory based on whether the final answer matches the ground truth.

4. Chain-of-Thought (CoT) Training 1:05:00

   To train models for complex reasoning, RL can be used to reward the entire trajectory based on the final answer's correctness. This approach bypasses the need for explicit labeling of the internal 'thinking tokens,' focusing only on verifiable outcomes.

• Stanford Online
• Reinforcement Learning
• Large Language Models (LLMs)
• Policy Gradient
• Proximal Policy Optimization (PPO)
• Chain-of-Thought (CoT)
• Markov Decision Process (MDP)
• Stanford CS229 Machine Learning
• Stanford AI Programs Info

Watch on YouTube Full article

Jul 31, 2026 · 1:17:32

Stanford CS229 Machine Learning | Spring 2026 | Lecture 14: Transformers, In-Context Learning

This lecture provides a deep technical dive into Large Language Models (LLMs), focusing on the Transformer architecture. Key concepts covered include subword tokenization (using techniques like Byte Pair Encoding - BPE) to convert text into numerical IDs, and modeling language distribution using auto-regressive conditional probabilities. The core mechanism is the self-attention layer, which uses Query (Q), Key (K), and Value (V) vectors to determine dependencies between tokens. A critical technical discussion centers on computational efficiency: standard attention has a quadratic $O(T^2)$ dependency on sequence length ($T$), leading to memory and computation bottlenecks for long contexts. Solutions like masking and specialized variants (e.g., Flash Attention) are discussed as methods to maintain performance while managing resource constraints.

Key takeaways

1. Auto-Regressive Modeling 1:35

   LLMs model language distribution by decomposing the joint probability into a product of conditional probabilities: $P(X_t | X_{1...t-1})$. This requires generating tokens sequentially, making the process auto-regressive.

2. Subword Tokenization (BPE) 3:30

   To handle rare or long words efficiently, models use subword tokenization (e.g., BPE), which breaks down text into smaller units (tokens). This allows leveraging common prefixes and suffixes to understand novel words.

3. Self-Attention Mechanism 11:10

   The attention mechanism calculates the relevance between all tokens in a sequence using Query ($Q$), Key ($K$), and Value ($V$) vectors via an inner product: $ ext{Softmax}(Q K^T / ext{scale}) V$. The output is a weighted linear combination of $V$ vectors.

4. Causal Masking for Auto-regression 21:20

   To ensure the model only predicts based on previous tokens (required for generation), a masking technique is applied to the attention matrix, setting all future dependencies to $- ext{infinity}$ before applying Softmax.

5. Computational Bottleneck ($O(T^2)$) 30:00

   The standard self-attention mechanism has a computational and memory complexity that scales quadratically with sequence length $T$ (i.e., $O(T^2)$). This is the primary limitation for processing very long contexts.

• Stanford Online
• Large Language Models (LLMs)
• Transformer Architecture
• Self-Attention Mechanism
• Auto-Regressive Generation
• Byte Pair Encoding (BPE)
• Computational Complexity
• Stanford CS229 Machine Learning
• OpenAI Tokenizer Playground

Watch on YouTube Full article

Jul 31, 2026 · 18:07

Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning

The talk discusses scaling AI agents to solve long-horizon problems, arguing that success requires a shift in mindset from merely increasing context windows to improving environment quality and algorithmic efficiency. Key technical advancements include using value models (critics) to reduce gradient variance and employing techniques like compaction and bootstrapping to manage sparse rewards over extended trajectories. The speakers emphasize that real-world complexity and robust simulation environments are more critical than simply having larger base models.

Key takeaways

1. Long Horizon is a Mindset, Not Just an Engineering Problem 15:12

   Solving major human challenges (e.g., curing cancer) requires patience and thinking in long timeframes, necessitating a fundamental shift in how AI systems are designed for sustained coherence.

2. RLHF is Crucial for Productizing LLMs 4:00

   The breakthrough that made LLMs usable was not the base model size, but the application of Reinforcement Learning from Human Feedback (RLHF), which provided necessary alignment and structure.

3. Value Models Mitigate Long-Horizon Challenges 16:56

   To handle long trajectories, value models (critics) are essential for reducing gradient variance and facilitating credit assignment, which is necessary when rewards are sparse.

4. Simulation Quality Trumps Context Window Size

   The failure of frontier models in real-world tasks (like trading football matches) demonstrated that scaling requires better, more complex environments and simulations, not just larger context windows.

• AI Engineer
• Reinforcement Learning
• Large Language Models (LLMs)
• AI Scaling
• Compute Optimization
• Simulation Environments
• openreward.ai
• General Reasoning (GR)

Watch on YouTube Full article

Jul 31, 2026 · 1:15:57

Stanford CS229 Machine Learning | Spring 2026 | Lecture 12: Representation Learning

The lecture provides a deep dive into advanced generative and adaptation techniques, covering Diffusion Models, Foundation Model paradigms, and Representation Learning. Key technical focus areas include deriving the loss function for diffusion models by comparing true and predicted mean reconstructions ($\mu_{ ilde{t}, t}$ vs $\mu_{ heta, t}$). For large language models (LLMs), the discussion highlights the shift toward using massive unlabelled data for pre-training and adapting via zero-shot or few-shot methods. Finally, it details advanced parameter efficiency techniques like Linear Probing and Low-Rank Adaptation (LoRA), which are crucial for managing memory and compute when deploying large models across multiple users.

Key takeaways

1. Diffusion Model Training Objective 33:55

   The training objective is derived from the lower bound on $\log p_ heta(x_0)$, simplifying to minimizing a sum of terms ($\mathcal{L}_{t-1}$) that compares the mean predicted reconstruction ($\mu_{ heta, t}$) with the true process mean ($\mu_{ ilde{t}, t}$). This loss function is used to train the model to predict noise/reconstruct missing information at each time step.

2. Foundation Model Paradigm Shift 1:02:00

   Modern LLMs utilize a paradigm shift involving pre-training on massive unlabelled data and adapting via zero-shot or few-shot learning, reducing the need for collecting extensive downstream labeled datasets. The goal is to build a general foundation that can be adapted to unlimited tasks.

3. Low-Rank Adaptation (LoRA) Efficiency 1:15:00

   To adapt massive pre-trained models without changing all parameters ($\theta$), LoRA restricts the weight update matrix $\Delta W$ to a low rank ($A B$). This significantly reduces the number of trainable parameters and is critical for memory efficiency, especially when serving multiple users.

• Stanford Online
• Generative Models
• Diffusion Models
• Foundation Models
• Large Language Models (LLMs)
• Representation Learning
• Low-Rank Adaptation (LoRA)
• Stanford CS229 Machine Learning
• Stanford Online AI Programs

Watch on YouTube Full article

Jul 31, 2026 · 17:42

Verifiable Environments for AI in Biology — Kenny Workman, LatchBio

Kenny Workman discusses the massive data challenge in modern biology (e.g., single cell experiments yielding 2–6 terabytes per run) and proposes treating experimental biology as a 'verifiable substrate' for AI training. LatchBio is positioning itself as an agent engineering lab, building benchmarks like 'spatial bench' to train agents on scientific reasoning rather than just memorized knowledge. The core argument is that verifiable data analysis pipelines (modeled after code execution) are necessary because frontier models currently lack the capability to reliably extract scientific insight from complex, real-world biological datasets.

Key takeaways

1. Data Volume and Challenge

   Modern techniques like single cell biology (2–6 TB per run) and spatial biology generate data volumes that exceed consumer storage capacity, necessitating automated analysis tools. The output is often a 'giant matrix of numbers' requiring complex processing.

2. Verifiable Substrate for Science 4:04

   The goal is to treat biological data analysis as an executable substrate, similar to how code provides verifiability in software. This allows the creation of natural benchmarks and measurable progress metrics.

3. Benchmarking Scientific Reasoning 5:23

   Benchmarks must move beyond simple Q&A settings to measure complex, multi-step scientific reasoning. LatchBio developed 'spatial bench' (146 problems) which requires deterministic graders and verifiable outcomes based on analysis DAGs.

4. Agent Limitations and Progress 9:00

   Frontier models cannot be trusted with real-world scientific work because they are missing the capability between knowing biology and writing code. The process requires iterative human verification and developing tasks that force rigorous reasoning.

• AI Engineer
• AI Agent Engineering
• Spatial Biology
• Scientific Benchmarking
• Data Analysis Pipelines
• Large Language Models (LLMs)
• LatchBio
• spatial bench

Watch on YouTube Full article

Jul 31, 2026 · 59:49

From LLM Theory to Practical Agentic Implementations - Seth Juarez

The talk demystifies Large Language Models (LLMs), arguing that their power does not come from inherent intelligence but from engineered 'harnesses' and fundamental primitives. The speaker outlines a progression of techniques—from simple completions to complex agentic loops—that allow developers to build robust, reliable AI systems by controlling the model's input context, output structure, and execution flow. Key concepts include using structured JSON for control flow, implementing tool calling via runtimes (not the LLM itself), establishing guardrails, and orchestrating multi-step workflows through shared 'threads.'

Key takeaways

1. LLMs are Next Token Predictors 1:42

   At its core, an LLM is a giant mathematical function that predicts the next token based on input tokens. The perceived intelligence comes from the surrounding runtime logic (the harness), not the model itself.

2. The Agentic Loop is the Core Primitive 21:20

   True agency is achieved by implementing an 'agentic loop'—a while loop in the runtime that continuously checks for and executes tool calls, feeding the results back into the context until no more tools are needed. This is the fundamental unit of agency.

3. Guardrails and Bindings Enhance Reliability 35:05

   To prevent misuse or incorrect execution (e.g., unauthorized memory access), developers must implement guardrail functions in their runtime logic. Furthermore, 'bindings' allow injecting fixed parameters (like `user ID`) into function calls, removing the burden of context-dependent reasoning from the LLM.

4. Composition Workflow for Multi-Step Agents 50:50

   Complex tasks are solved by chaining multiple specialized agents or prompts (e.g., Research $ ightarrow$ Draft $ ightarrow$ Review). All these components share a single, persistent 'thread' context to maintain state and pass control flow sequentially.

• NDC Conferences
• Large Language Models (LLMs)
• Agentic Systems
• Prompt Engineering
• Tool Calling
• State Management
• Software Architecture
• Prompty
• Seth Juarez Draft Flow Repo

Watch on YouTube Full article

Jul 31, 2026 · 37:37

AI Security Costs Rise: Cost of a Data Breach Report & Claude Opus 5

The discussion analyzes the rapidly escalating security risks posed by AI, noting that while attackers find it cheaper and easier to launch attacks using frontier models without proper guardrails, defenders face increasing costs in prevention. Key technical discussions covered include identifying top vulnerabilities (Model Inversion and Prompt Injection), critiquing new LLM releases like Claude Opus 5 for performance inconsistencies, and exploring the concept of AI as an 'extended mind' through daily rituals. The session also provided a high-level explanation of LLMs, emphasizing that future software development will increasingly rely on higher levels of abstraction rather than low-level code.

Key takeaways

1. AI is driving the cybersecurity cost increase 2:15

   The IBM Cost of a Data Breach Report 2026 highlights that AI dominates all sections, increasing investment intention from 64% to 85%. Top vulnerabilities include Model Inversion ($6-$7 million) and Prompt Injection ($4.89 million).

2. Guardrails and Policy are critical for AI safety 4:10

   To mitigate risks, the focus must shift to treating LLM agents as first-class citizens, requiring robust guardrails, identity management, proper access control, and encryption at rest.

3. LLMs are evolving toward higher abstraction 6:15

   The history of computing is defined by increasing levels of abstraction (e.g., from assembly to declarative languages like Terraform). Future AI development will follow this trend, allowing users to describe desired outcomes rather than specific steps.

4. The business case for 'extended mind' AI 6:25

   Midjourney acquiring the astrology app CoStar suggests a strategic move to integrate AI into daily, ritualistic life patterns, making it an 'extended mind' rather than just a separate tool.

• IBM Technology
• Cybersecurity
• Large Language Models (LLMs)
• AI Ethics and Safety
• Software Architecture
• Data Privacy
• IBM Cost of a Data Breach Report 2026
• What Does AI Look Like?

Watch on YouTube Full article

Jul 31, 2026 · 20:14

Agents at Scale: Inside MiniMax's Model and the Infrastructure Behind It — Olive Song

The discussion details the engineering stack and open-source philosophy behind MiniMax's model, M3. Olive Song emphasizes that the open weights approach allows the community to build upon and optimize the model, fostering widespread intelligence access. The technical focus covers advanced training techniques—including multimodality (text, image, video) and Reinforcement Learning (RL) for long-horizon tasks like replicating academic papers (12-hour runs)—and the complex infrastructure required for deployment. Key engineering challenges discussed include writing specialized GPU kernels, optimizing the inference stack from 'day zero,' managing KV cache growth in agentic workflows, and adapting to shifting workloads from chat-based to multi-turn, tool-calling agents.

Key takeaways

1. Open Weights Philosophy 2:07

   MiniMax advocates for open source because it aligns with their mission of making intelligence widely accessible. By releasing weights, they enable developers (like Together AI) to optimize the model's inference speed and capabilities through community contributions.

2. Multimodality Training 8:02

   MiniMax M3 is multimodal, understanding text, code, images, and videos. Crucially, it was trained multimodally from scratch to prevent 'training collapse,' ensuring that the modalities naturally interact (e.g., visual tokens attending to text tokens).

3. Agentic Workloads and Inference Shifts 13:40

   The workload is shifting from simple chat turns to complex agentic workflows involving hundreds of multi-turn tool calls. This requires significant optimization in the inference stack, particularly concerning KV cache management and routing.

4. Long-Horizon RL Tasks

   Training for complex tasks (e.g., replicating an ICLR paper over 12 hours) requires careful formulation of the problem, defining environments, and optimizing reward functions within the Reinforcement Learning framework.

• AI Engineer
• Large Language Models (LLMs)
• Reinforcement Learning (RL)
• GPU Optimization
• Inference Engineering
• Open Source AI
• Multimodality
• MiniMax M3
• OS World
• SVG bench / Kernel bench

Watch on YouTube Full article

Jul 30, 2026 · 5:24

Android Studio Quail 2, the Build with Gemini XPRIZE, and more! - Google Developer News July 2026

This Google Developer News update details major advancements in Android development tools, including the stable release of Android Studio Quail 2. Key features include parallel agentic task management and enhanced memory leak detection via Leak Canary. Furthermore, the session covers updates to Android Bench for LLM evaluation using the Harbor framework, an inside look at YouTube's AI prototyping stack (Emergent), and details the Build with Gemini XPRIZE competition offering a $2 million prize pool.

Key takeaways

1. Android Studio Quail 2 Stable Release

   The stable release introduces parallel chats for managing multiple agentic tasks simultaneously, upgrading Leak Canary integration into the profiler for faster heap analysis (up to five times faster), and integrating App Quality Insights with production reports (Firebase/Crashlytics) to automatically suggest or fix crash-causing code.

2. Android Bench Methodology Upgrade 0:01

   Updates are rolling out to Android Bench, standardizing on the Harbor framework for model evaluation. Cloud fable 5 currently leads the leaderboard with a score of 84.5, and users can now submit unique development tasks to shape the benchmark.

3. YouTube AI Prototyping Stack (Emergent) 0:02

   The series reveals how YouTube builds flexible, safe prototyping environments that minimize technical debt and system layers for rapid idea testing across Google's product lines.

4. Build with Gemini XPRIZE Competition 0:02

   A $2 million competition challenges developers to build a real business using AI by solving a real-world problem, utilizing the full Google stack from Gemini app validation to Google Cloud scaling. The deadline is August 17.

• Google Developers
• Android Development
• Large Language Models (LLMs)
• Build Tools
• AI Prototyping
• Software Testing
• Android Studio Quail 2
• Android Bench
• Emergent (YouTube AI Prototyping)
• Build with Gemini XPRIZE

Watch on YouTube Full article

Jul 29, 2026 · 36:59

Stanford CS229 Machine Learning | Spring 2026 | Lecture 1: Introduction

This lecture provides a high-level introduction to Machine Learning fundamentals (Supervised, Unsupervised, and Reinforcement Learning) within the context of modern AI. The course emphasizes understanding the mathematical foundations and core techniques behind algorithms rather than focusing on programming implementation. Key topics include model training using large datasets, advanced concepts like embeddings, and the architectural differences between traditional ML tasks and general-purpose Large Language Models (LLMs). A critical focus for system builders is placed on the necessity of optimizing ML systems for hardware compatibility and speed.

Key takeaways

1. ML Paradigm Shift 6:11

   The field has moved from specific, task-oriented models to general-purpose agents (LLMs). While traditional methods still apply, the focus is on tuning fundamental model capabilities rather than building complex data pipelines for every single use case.

2. The Importance of ML Systems 33:00

   A planned lecture will cover 'ML system,' which addresses the critical need to make software and hardware compatible. Optimizing algorithms for speed (e.g., making them run 2x faster) is crucial due to the high cost associated with AI computation.

3. Reinforcement Learning (RL) in LLMs 24:50

   RL can be used to train models dealing with stochastic sampling, such as text generation. Techniques like Policy Gradient and using human feedback (e.g., RLHF/RAG) are necessary because the generation process is not differentiable.

• Stanford Online
• Machine Learning Fundamentals
• Supervised Learning (SFT)
• Unsupervised Learning (Clustering, LSA)
• Reinforcement Learning (RL)
• Large Language Models (LLMs)
• Diffusion Models
• ML System Optimization
• CS229 Machine Learning
• Stanford AI Programs Info

Watch on YouTube Full article

Jul 29, 2026 · 22:30

How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh

The talk details how Kepler built a verifiable AI system for financial services, addressing the core limitation that Large Language Models (LLMs) are inherently non-deterministic probability machines. To achieve reliable work product—such as DCF models or investment memos—the system must augment the LLM with a deterministic substrate. This architecture enforces three tenets: Atomic Provenance (tracking every number's source), Scope Determinism (separating reasoning from computation), and Reconciliation/Derivation Chains (ensuring numerical accuracy through verifiable steps). The goal is to shift AI's edge from content generation to verifiable, traceable output.

Key takeaways

1. LLMs are Probability Machines, Not Deterministic Calculators 4:17

   AI models excel at next token prediction (writing) but fail when deterministic accuracy is required, such as arithmetic or pulling specific figures from filings. Using LLMs for verification alone is insufficient because they are non-deterministic.

2. The Need for Verifiability in Finance 7:07

   In finance, information must be traceable to its source (provenance). The challenge is moving beyond simple citation (an after-the-fact audit) to true deterministic verification of a number's correctness.

3. The Three Tenets of Verifiable AI 15:26

   Kepler’s platform ensures numerical accuracy through three mechanisms: Atomic Provenance (writing references instead of numbers), Scope Determinism (separating the model's reasoning from deterministic computation), and Reconciliation/Derivation Chains (tracking every step to produce a final number).

4. AI Must Be Modeled Like a Portfolio Manager (PM) 20:01

   The system must not let the LLM perform computation. Instead, it uses deterministic tools to calculate figures and pull data from structured sources like XBRL or filings, ensuring the model only dictates *what* needs to be computed, not *how*. This is crucial for producing reliable work product.

• AI Engineer
• Verifiable AI
• Financial Technology (FinTech)
• Large Language Models (LLMs)
• Deterministic Computing
• Data Provenance
• System Architecture
• Kepler
• Anthropic

Watch on YouTube Full article

Jul 28, 2026 · 1:17:20

Training Agents 3: Reinforcement Learning

This session introduces Group Relative Policy Optimization (GRPO), a reinforcement learning (RL) method that advances agent training beyond mere imitation (SFT/Distillation). GRPO trains models by sampling multiple completions per prompt and using the group's relative scores—calculated via a reward function—as the primary training signal. This approach is highly effective for complex tasks, allowing agents to learn from their own varied trajectories in an iterative loop.

Key takeaways

1. GRPO Mechanics 20:30

   GRPO calculates advantages relative to the group average (Reward - Group Average / Group Standard Deviation). This method eliminates the need for a separate value model, reward model, or critic, simplifying the RL loop. The process involves generating multiple rollouts, scoring them with a verifiable Python function (the reward function), and updating the policy based on these relative advantages.

2. Training Pipeline Progression 5:05

   The training pipeline typically progresses from Supervised Fine-Tuning (SFT) for dense, off-policy signals, to Distillation for richer online rollouts, and finally to RL/GRPO for sparse, on-policy learning. SFT is often used first to bootstrap the model's understanding of the task structure.

3. Reward Function Design 26:40

   Defining a verifiable reward function is critical; it acts as a 'contract' defining success. Rewards can be composed of multiple components, such as a format check (e.g., ensuring JSON structure) and an accuracy check (e.g., passing unit tests). For agentic tasks, using test suites or compiling code provides robust signals.

4. Interpreting Training Curves 35:00

   Monitoring training curves via tools like Track.io is essential for debugging. Key metrics include the reward (should rise), entropy (should remain stable/flat, not dive or spike), and completion length. Failure modes—such as 'reward hacking' (high reward but low test accuracy) or 'collapse' (low entropy)—require deep data inspection.

• Hugging Face
• Reinforcement Learning
• Large Language Models (LLMs)
• Policy Optimization
• Agentic Systems
• Machine Learning Pipelines
• Training Agents Repo
• HF YouTube Channel

Watch on YouTube Full article

Jul 28, 2026 · 53:25

Doors of (AI)pportunity: The Front and Backdoors of LLMs - Kasimir Schulz & Kenneth Yeung

The talk provides an advanced deep dive into LLM security vulnerabilities, moving beyond simple prompt injection attacks. Speakers demonstrate sophisticated methods including Knowledge Return Oriented Prompting (KROP), Policy Puppetry, and direct manipulation of the model's computational graph to implant backdoors. The discussion highlights that even seemingly safe formats (like ONNX) can be exploited if the underlying architecture is compromised, posing significant supply chain risks for AI-driven applications.

Key takeaways

1. Multiple Layers of Defense are Required 35:35

   System prompts alone are insufficient protection. Developers must implement multiple layers of defense to mitigate attacks, as demonstrated by the failure of simple system prompt rules against advanced exploits.

2. Indirect Prompt Injection is a Major Threat 43:25

   Attacks do not require direct user interaction. Compromise can occur through injected text in external sources like customer reviews, emails (especially those processed by Retrieval Augmented Generation/RAG systems), or calendar invites.

3. Model Backdoors are Difficult to Detect 50:15

   Backdoors can be implanted into the computational graph of a model (e.g., changing an animal classification logic based on a specific color trigger). These backdoors can remain dormant until a precise trigger condition is met, making them highly effective and difficult to detect.

• NDC Conferences
• AI Security
• Large Language Models (LLMs)
• Cybersecurity
• Prompt Engineering
• Machine Learning Architecture
• Hugging Face
• Netron.app

Watch on YouTube Full article

Jul 28, 2026 · 36:31

How to Break AI Systems (Before Someone Else Does) - Gary Lopez - NDC Toronto 2026

This talk provides an advanced overview of AI system vulnerabilities, arguing that Large Language Models (LLMs) are fundamentally different from traditional software because they predict the next token rather than executing code with clear separation. The presentation details various attack vectors—including direct and indirect prompt injections, encoding bypasses, and agentic risks—and outlines modern defense strategies like 'Spotlighting' to help developers build more resilient AI applications.

Key takeaways

1. LLMs Fail Due to Data/Code Ambiguity 18:03

   Unlike traditional software with clear separation between executable code and data, LLMs treat all inputs (user prompts, system instructions, external data) as tokens for prediction. This fundamental flaw allows attackers to bypass safeguards.

2. AI Red Teaming is Evolving 5:20

   Modern AI red teaming must consider not only traditional adversarial attacks but also Responsible AI (RAI) challenges, such as hallucinations, bias inference, and knowledge-to-action instructions.

3. Mitigation Requires Multi-Layered Defense 29:50

   Defenses must be applied at multiple points: input filters (before the model receives data), output guards (after generation but before user display), and through techniques like 'Spotlighting' to enforce structural integrity.

• NDC Conferences
• AI Security
• Red Teaming
• Prompt Injection
• Large Language Models (LLMs)
• Data Integrity
• Responsible AI (RAI)
• jailbroken: How does large language model safety training fail
• Crescendo Attack

Watch on YouTube Full article