The Video Signal

Channel

Hugging Face

Digests from Hugging Face

Training Agents 4: From reward functions to environments. thumbnail

· 1:13:40

Training Agents 4: From reward functions to environments.

This session details the evolution of agent training from simple functional reward signals to complex, stateful environments. The core concept is the `reset()/step()` contract, which allows agents to interact with a simulated world (the environment) over a sequence of actions. The discussion covers the OpenM framework, which standardizes environment definition, and its integration with TRL (Transformer Reinforcement Learning) using isolated compute environments like Hugging Face sandboxes. This enables training sophisticated agents, such as coding agents (OpenCode), on complex, multi-step tasks while maintaining reproducibility and isolation.

Key takeaways

  1. The Shift to Stateful Environments 1:42

    For agents performing sequences of actions (e.g., tool calls, file edits), the reward signal must come from the environment's state after an action, rather than being a simple Python function evaluated once. This requires adopting the standard `reset()/step()` contract, moving from sparse signals to continuous interaction loops.

  2. The OpenM Ecosystem 5:30

    OpenM provides a standardized, containerized way to define any task as an environment. It packages the task, the runtime (compute backend), and the grading components (verifiers, rewards) into a single, shareable unit that can be deployed on various platforms (e.g., Hugging Face Spaces, Kubernetes).

  3. Advanced Agentic Training Loops 9:10

    Training can be managed by two models: the 'White Box' (where the training framework owns the loop) and the 'Black Box' (where the agent/harness owns the loop). The latter requires a 'capture proxy' to intercept agent calls and send the resulting rollout graph back to the trainer for policy updates.

  4. Reproducibility and Isolation 7:30

    Environments are designed to be self-contained applications, often deployed in isolated compute environments (like HF sandboxes). This ensures that training runs are reproducible and prevents the agent from accessing or manipulating the verifiers or task description to 'hack' the reward signal.

Watch on YouTube Full article

Run Local Models in Pi: llama.cpp, GGUF, and the /llama Command thumbnail

· 6:11

Run Local Models in Pi: llama.cpp, GGUF, and the /llama Command

This tutorial provides a complete guide on running large language models (LLMs) locally on a Raspberry Pi using `llama.cpp`. The process emphasizes privacy and offline capability by ensuring that no prompts, code, or data leave the local machine. Key steps include installing `llama.cpp`, selecting an optimal GGUF model (like Qwen3 8B) based on hardware compatibility, and loading/running the model via the `/llama` command.

Key takeaways

  1. Local Model Operation

    Running models locally with `llama.cpp` ensures that all data processing remains entirely within the machine, eliminating reliance on third-party APIs for prompts, code, or data (0:15).

  2. Installation and Setup 2:29

    Install `llama.cpp` using the provided installer link (`llama.app`) to establish the local server environment, allowing subsequent model interaction via the `llama serve` command (0:59).

  3. Model Selection and Quantization 3:35

    To select an optimal model, use the hardware compatibility feature on sites like `llama.app`. This tool recommends the best quantization level (e.g., 4-bit) for specific hardware (M4 Max), which is crucial for performance (2:30).

  4. Running Models via Pi 6:00

    After downloading a model ID and selecting the appropriate quantization (e.g., Q4), models can be loaded and interacted with directly using the `/llama` command within the local environment (3:30).

Watch on YouTube Full article

We shipped 207 WebGPU Kernels for Browser AI thumbnail

· 11:44

We shipped 207 WebGPU Kernels for Browser AI

Hugging Face has released over 207 open-source WebGPU kernels for browser AI and a new library, `@huggingface/kernels`, to manage their execution. This system allows developers to run complex machine learning operations (like attention mechanisms or large matrix calculations) directly on the GPU in the browser using JavaScript APIs. The core innovation lies in generating optimal WebGPU shaders (WGSL) via Jinja templates, ensuring that the kernel is tailored to the specific data types and workgroup sizes of the user's device for maximum performance.

Key takeaways

  1. WebGPU Kernels for Browser AI

    Over 207 open-source WebGPU kernels are available on the Hugging Face Hub, enabling high-performance computation directly in the browser.

  2. The @huggingface/kernels Library

    This new library abstracts the complexity of GPU interaction, allowing users to load, validate, and execute various kernels (e.g., `add`) using simple asynchronous JavaScript functions.

  3. Performance Demonstration 5:58

    Demos show significant performance gains: an attention mechanism is implemented in ~20 lines of JS, and a 1M+ cell wave animation runs at up to 60 FPS (capped by `requestAnimationFrame`), vastly outperforming plain JavaScript calculations.

Watch on YouTube Full article

Hugging Face Journal Club: AI Research Preference Models thumbnail

· 34:48

Hugging Face Journal Club: AI Research Preference Models

This discussion summarizes Meta's research on Research Preference Models (RPMs), which utilize Large Language Models (LLMs) to predict the success of machine learning experiments. Given that ML evaluations are computationally expensive (potentially taking days or weeks on GPUs), RPMs guide autonomous agents by selecting the most promising candidates for evaluation, drastically reducing required compute time while maintaining high performance. The process involves complex tree search mechanisms and can be enhanced through ensembling multiple LLM judges.

Key takeaways

  1. RPM Goal: Reducing Compute Budget 2:35

    The primary goal is to avoid evaluating all possible ML candidates, which consumes excessive compute resources. RPMs select the most promising experiments (mutations) to evaluate next, reducing required time from potentially days down to hours while achieving performance comparable to an 'Oracle' [0:02:35].

  2. RPM Types and Functionality 2:58

    Two main types are discussed: the Inference-only RPM (using a frozen LLM to reason over plans/code) and the Agentic RPM, which can run small-scale pilot experiments to further refine predictions [0:02:58].

  3. The Search Process (Tree Traversal) 3:45

    The process is modeled as a tree search, starting from a root node (initial experiment). Candidates are generated as children nodes; the RPM scores these candidates, and the agent selects the best one to explore next. This mechanism resembles Monte Carlo Tree Search (MCTS) [0:03:45].

  4. Ensembling for Robustness 8:13

    To improve reliability, the research suggests evaluating candidates using ensembles of multiple frontier models (e.g., GPT-5 Opus). Techniques include majority vote and an LLM arbiter ensemble to mitigate issues like reward hacking [0:08:23].

Watch on YouTube Full article

Hugging Face Journal Club: Training AI Scientists to Replicate Research thumbnail

· 34:41

Hugging Face Journal Club: Training AI Scientists to Replicate Research

The discussion summarizes research on Faraday-27B, a model trained by Inherent designed for scientific replication—the ability to reproduce results from redacted ML/AI papers. The system uses Reinforcement Learning (RL) and integrates CodeX as a tool, allowing the agent to execute code within a simulated environment. Key methodological advances include using sophisticated rubric-based judges (generated via Claude) instead of simple verifiers, employing multi-rollout averaging to mitigate variance, and implementing weighted credit assignment across the agent's steps.

Key takeaways

  1. Scientific Replication Task

    The model is tasked with replicating missing figures from redacted ML/AI papers. This process requires the agent to use tools (like CodeX) and execute code in a simulated environment, moving toward full automation of AI R&D.

  2. Advanced Judging Mechanism 0:01

    Instead of simple verification, the system uses a rubric-based judge (generated by Claude) that assigns fine-grained points for correct reasoning, figure accuracy, and code writing. This process involves averaging judgments across multiple rollouts to prevent reward hacking.

  3. Performance & Scaling 0:02

    The trained Faraday model demonstrated strong performance, sometimes outperforming much larger models like Claude and GPT-5. Furthermore, the system showed generalization even when given increased compute resources (e.g., scaling up to 8 hours/8 B300s).

Watch on YouTube Full article

Hugging Face Journal Club: Direct On-Policy Distillation thumbnail

· 33:25

Hugging Face Journal Club: Direct On-Policy Distillation

The discussion details a novel technique called Direct On-Policy Distillation for achieving weak-to-strong generalization in large language models. This method proposes an efficient alternative to expensive full Reinforcement Learning (RL) training by leveraging the policy shift observed when training a small model with RL. Specifically, it uses this policy shift as a dense reward signal to train and update a much larger target model (student), significantly reducing computational costs compared to direct RL on the large model.

Key takeaways

  1. Weak-to-Strong Generalization via Policy Shift

    Instead of directly training a large model with expensive RL, this method measures how an RL run changes a small model's policy (the 'policy shift'). This shift is then used as a dense reward signal to distill knowledge into the larger target model.

  2. Efficiency Gains

    The technique offers substantial cost savings. For example, training a 7B model via RL might take 320 hours, while using distillation from a 1.5B model's policy shift can reduce the estimated time to around 164 hours.

  3. Methodological Blurring

    The process blurs the line between traditional RL and knowledge distillation by combining two types of losses: the policy shift signal (from RL) and a standard KL term, making the overall training setup highly efficient.

Watch on YouTube Full article

Prompt Caching Explained: Stop Overpaying for AI Agents thumbnail

· 17:16

Prompt Caching Explained: Stop Overpaying for AI Agents

Prompt caching is essential for managing costs in long-running AI agent sessions. Instead of paying full price for re-sending entire conversation histories (context windows) on every turn—which can lead to exponential cost increases—proper prompt caching ensures that the LLM only charges a discounted rate for tokens it has seen before. This requires designing an agent harness that correctly preserves reusable prompt prefixes and understands provider-specific API behaviors.

Key takeaways

  1. Cache Inputs, Not Outputs

    Prompt caching stores and reuses inputs (the conversation history/prompt), not the LLM's outputs. Caching outputs is generally not useful for LLMs.

  2. Cost Escalation Risk 0:23

    Without caching, sending a growing context window (e.g., 51k tokens, then 55k tokens) repeatedly leads to exponentially increasing costs, making long sessions prohibitively expensive.

  3. Cache Expiry is Critical 2:03

    The cache has an expiration time (e.g., OpenAI OAuth API: one hour; Anthropic: five minutes by default). The agent harness must account for this expiry to avoid paying full price again.

  4. System Prompts Must Be Static 8:00

    To prevent cache invalidation, do not include dynamic elements like timestamps or current working directories within the system prompt. Keep conversation history append-only.

Watch on YouTube Full article

How AI agents reproduced ICML 2026 papers thumbnail

· 26:37

How AI agents reproduced ICML 2026 papers

The ICML 2026 Agents Reproduction Challenge was a large-scale community effort involving over 1,200 participants and AI agents attempting to reproduce claims from accepted machine learning papers. The initiative demonstrated the potential for automated reproducibility testing in academic research, finding that while a majority of papers were reproducible (some fully, some via smaller scale experiments), significant flaws were also identified. Key technical takeaways include the use of specialized tools like `tracko` and Hugging Face infrastructure to create fully auditable, machine-readable log books for every reproduction attempt.

Key takeaways

  1. Scale of Reproduction Effort 4:18

    The challenge involved 1,200+ participants attempting to reproduce claims from a subset of ICML 2026 papers. A total of 2,200 unique papers were attempted, resulting in approximately 35,000 different claims being judged (Timestamp: ~4:18).

  2. Reproducibility Success Rate 12:34

    A majority of the papers looked at were reproducible. Specifically, over 2,000 papers had at least one major claim independently verified (Timestamp: ~6:34).

  3. Identification of Flaws and Contested Claims 13:10

    The community found that about 23% of papers could not be fully reproduced as claimed, leading to at least 496 contested or falsified claims. Furthermore, 49 papers were almost fully falsified (Timestamp: ~8:15).

  4. Best Practices in Agent Use 15:42

    The 'Best Human in the Loop' award highlighted that effective reproduction requires human intervention to guide agents, especially when evaluating qualitative results (e.g., building a UI to compare quantized images) (Timestamp: ~10:35).

Watch on YouTube Full article

Hugging Face Journal Club: Scaling Laws for Pre-training & RL thumbnail

· 30:57

Hugging Face Journal Club: Scaling Laws for Pre-training & RL

The discussion analyzes a paper proposing a joint scaling law for pre-training and Reinforcement Learning (RL), which models how compute allocation across these stages impacts downstream task performance. Key findings suggest that increasing compute allocated to pre-training significantly boosts the model's ability on specific tasks (Pass 1) but has diminishing returns on generalized capability (Pass K). The analysis highlights the importance of optimizing the trade-off between SFT and RL compute budgets.

Key takeaways

  1. Pre-training vs. RL Compute Allocation 15:20

    The primary takeaway is that increasing compute allocated to pre-training leads to higher performance on downstream tasks (Pass 1). Conversely, while RL improves Pass 1, the model's generalized capability (Pass K) remains relatively stable regardless of the pre-training scale.

  2. Scaling Laws and Model Size 24:45

    When fixing the total compute budget, training smaller models for longer is generally more effective than attempting to train larger models, contradicting simple Chinchilla scaling assumptions in certain contexts.

Watch on YouTube Full article

Tau Crash Course: The Python Port of Pi thumbnail

· 25:04

Tau Crash Course: The Python Port of Pi

Tau is a Python-based coding agent harness designed as an architectural port of Pi, aimed at standardizing and enhancing reliable AI workflow development. It features a Textual TUI for interaction and maintains core functionality like session management, tool calling (read, write, edit, bash), and skill invocation. Key technical aspects include the ability to store sessions in a JSONL format for easy analysis and integration into larger build pipelines.

Key takeaways

  1. Architectural Porting

    Tau is a coding harness developed entirely in Python and functions as an architectural port of Pi, ensuring core functionality remains consistent while updating the user interface.

  2. TUI Framework

    The Terminal User Interface (TUI) for Tau is built using Textual, which results in some differences from Pi's native TUI but maintains similar command structures.

  3. Session Management and Tree Structure 13:39

    Sessions are stored as a tree structure, where every message points to its parent ID. This allows users to fork conversations or navigate history using the `/tree` command.

  4. Data Export for Analysis

    Sessions can be exported into a JSONL format (JSON Lines), making them easily parsable for external analysis, skill testing, and integration with other agents.

Watch on YouTube Full article

Hugging Face Journal Club: Kimi K3 thumbnail

· 41:28

Hugging Face Journal Club: Kimi K3

The discussion summarizes the Kimi K3 tech report, detailing a highly complex and scaled training pipeline for large language models. Key innovations include specialized agentic structures (general, deep research, coding), multi-tier on-policy distillation across nine domain experts, and advanced infrastructure techniques like partial rollout schemes and dynamic resource scheduling. The talk emphasizes that the model's performance is achieved through sophisticated engineering efforts rather than a single breakthrough concept.

Key takeaways

  1. Multi-Tier Expert Specialization 2:00

    The training pipeline involves generating nine specialized domain experts (3 reasoning levels: low, high, max) for three general domains (general tasks, deep research, coding). These are combined using multi-tier on-policy distillation into a single student model.

  2. Partial Rollout Scheme 4:08

    A novel RL technique where the system samples $k$ rollouts from $n$ prompts. It uses an additional parameter $\lambda$ to measure completed rollouts within a budget, updating weights on that subset and pushing incomplete rollouts into subsequent iterations.

  3. Extensible Chat Template 6:42

    The Kimi K3 model utilizes an 'extensible token markup language' for its chat template. This design aims to be easy to learn during SFT while remaining flexible enough to accommodate future modalities without re-engineering the core template.

  4. Advanced Infrastructure Scaling 30:35

    The system employs sophisticated resource management, including a dynamic rollout auto throttling scheduler based on KV cache pressure and a method for collocating RL training and inference by automatically balancing GPU resources between the two tasks.

Watch on YouTube Full article

Training Agents 3: Reinforcement Learning thumbnail

· 1:17:20

Training Agents 3: Reinforcement Learning

This session introduces Group Relative Policy Optimization (GRPO), a reinforcement learning (RL) method that advances agent training beyond mere imitation (SFT/Distillation). GRPO trains models by sampling multiple completions per prompt and using the group's relative scores—calculated via a reward function—as the primary training signal. This approach is highly effective for complex tasks, allowing agents to learn from their own varied trajectories in an iterative loop.

Key takeaways

  1. GRPO Mechanics 20:30

    GRPO calculates advantages relative to the group average (Reward - Group Average / Group Standard Deviation). This method eliminates the need for a separate value model, reward model, or critic, simplifying the RL loop. The process involves generating multiple rollouts, scoring them with a verifiable Python function (the reward function), and updating the policy based on these relative advantages.

  2. Training Pipeline Progression 5:05

    The training pipeline typically progresses from Supervised Fine-Tuning (SFT) for dense, off-policy signals, to Distillation for richer online rollouts, and finally to RL/GRPO for sparse, on-policy learning. SFT is often used first to bootstrap the model's understanding of the task structure.

  3. Reward Function Design 26:40

    Defining a verifiable reward function is critical; it acts as a 'contract' defining success. Rewards can be composed of multiple components, such as a format check (e.g., ensuring JSON structure) and an accuracy check (e.g., passing unit tests). For agentic tasks, using test suites or compiling code provides robust signals.

  4. Interpreting Training Curves 35:00

    Monitoring training curves via tools like Track.io is essential for debugging. Key metrics include the reward (should rise), entropy (should remain stable/flat, not dive or spike), and completion length. Failure modes—such as 'reward hacking' (high reward but low test accuracy) or 'collapse' (low entropy)—require deep data inspection.

Watch on YouTube Full article

The Hugging Face Hub for Enterprise & Academia thumbnail

· 6:44

The Hugging Face Hub for Enterprise & Academia

The paid organizational plan for the Hugging Face Hub provides advanced features critical for enterprise and academic use cases, focusing on enhanced collaboration, robust security, and strict governance. Key upgrades include private workspaces with built-in versioning and lineage tracking, centralized identity management via SSO (SAML/OIDC), IP allowlisting, audit logs, and streamlined billing through prepaid credits that separate storage costs from compute usage.

Key takeaways

  1. Private Collaboration Workspace 0:25

    Organizations gain a single shared namespace for models, datasets, and Spaces, featuring built-in versioning and lineage tracking. Resource Groups allow granular access control to repositories (0:25).

  2. Advanced Data Interaction 0:41

    The Dataset Viewer allows users to query private data using an agent interface (e.g., SQL console), enabling natural language queries against datasets (0:41).

  3. Enterprise Security & Compliance 2:10

    Security features include Single Sign-On (SSO) via SAML/OIDC, centralized token management, IP allowlisting for corporate network restriction, and the option to choose data storage regions (e.g., within the EU). The infrastructure is SOC 2 Type II certified (1:30).

  4. Governance and Automation 2:32

    Features like audit logs, service accounts for CI/CD workflows, SCIM provisioning for user lifecycle syncing, and gated access flows ensure controlled adoption of open-weight models at scale (1:52).

Watch on YouTube Full article

Local AI 201 thumbnail

· 53:22

Local AI 201

The session provides an advanced deep dive into local AI deployment, emphasizing that successful LLM inference is not determined by hardware capacity alone. Instead, it requires selecting a balanced stack comprising the right model, quantization level, and specialized inference engine (e.g., VLLM, llama.cpp) for the specific use case—whether single-user chat or high-concurrency agentic workflows. Key performance metrics like memory bandwidth are shown to be more critical than raw memory capacity when scaling up requests.

Key takeaways

  1. Start with the Use Case, Not the Hardware 2:09

    When designing a local AI solution, always begin by defining the required use case (e.g., single-user chatbot vs. 50-person agentic workflow). The hardware, model, and engine stack must then be selected to support that specific requirement.

  2. Memory Bandwidth is Critical for Throughput 4:08

    For serving multiple requests (high throughput), memory bandwidth is often a more critical bottleneck than total memory capacity. For example, the RTX 5090 was shown to achieve significantly higher performance due to its high bandwidth compared to other devices.

  3. Engine Selection Dictates Performance Under Load 5:41

    The choice of inference engine (e.g., VLLM vs. llama.cpp) and kernel optimization is paramount. Improperly selecting an engine can severely limit performance, causing a high-bandwidth device to perform worse than a lower-bandwidth machine under load.

  4. Local AI Offers Superior Privacy and Control 7:30

    Running LLMs locally provides massive advantages in security, privacy, and control compared to relying on third-party cloud APIs. This allows users to fully tune the stack for long-term stability.

Watch on YouTube Full article

Hugging Face Journal Club: AsyncOPD and How Stale Can On-Policy Distillation Be? thumbnail

· 30:27

Hugging Face Journal Club: AsyncOPD and How Stale Can On-Policy Distillation Be?

The discussion details Asynchronous On-Policy Distillation (AsyncOPD), a method designed to significantly boost training throughput by making the distillation process fully asynchronous. While conventional methods are synchronous and suffer from GPU blocking during backpropagation, AsyncOPD continuously generates rollouts from policies while simultaneously scoring them with a teacher model. This approach achieves substantial speedups (1.5x to 2.7x) compared to synchronous methods, though it introduces complexity related to maintaining stability when student rollouts become significantly off-policy.

Key takeaways

  1. AsyncOPD for Throughput Gains 23:23

    By decoupling the generator (student policy), scorer (teacher model), and backpropagator, AsyncOPD eliminates GPU blocking inherent in synchronous distillation. This allows continuous operation, leading to throughput improvements of 1.5x to 2.7x on various math benchmarks.

  2. Addressing Cache Misses via Monte Carlo Sampling 30:07

    When calculating Reverse KL divergence using Top-K logits, cache misses can occur because the required log probabilities for the loss calculation may not have been stored during the initial sampling phase. MC sampling is proposed as a solution to estimate the loss accurately by storing and correcting estimates using important sampling.

  3. Trade-offs in Off-Policy Distillation 20:50

    While fully asynchronous methods offer high throughput, they require careful handling of off-policy rollouts. The stability and accuracy are dependent on the degree of staleness allowed (the difference between the current policy and the teacher's distribution).

Watch on YouTube Full article

New Model: Inkling by Thinking Machine on Hugging Face thumbnail

· 36:00

New Model: Inkling by Thinking Machine on Hugging Face

Thinking Machines announced Inkling, a massive open model with nearly one trillion parameters. It is designed as a natively multimodal architecture, accepting image, text, and audio inputs simultaneously. Key technical features include a Mixture-of-Experts (MoE) structure (975B total / 41B active params), a 1M token context window, and advanced deployment support across multiple frameworks like `transformers`, SGLang, vLLM, and `llama.cpp`. The model is available in BF16 and NVFP4 formats, facilitating high-performance inference on various hardware setups.

Key takeaways

  1. Multimodal Capability & Scale 3:50

    Inkling is a true multimodal model that natively processes image, text, and audio inputs using a single architecture. It boasts an immense 1M token context window and was trained on 45T tokens.

  2. Architectural Innovations 5:10

    The model utilizes relative attention (replacing RoPE) and a specialized SConv layer to efficiently aggregate hidden states, making it highly efficient for multimodal tasks compared to previous models.

  3. Deployment Flexibility 7:10

    Inkling supports multiple deployment paths: BF16 (requiring ~2TB VRAM) and NVFP4 (600GB VRAM). It provides day-zero support for `transformers`, SGLang, vLLM, and `llama.cpp` (including GGUF quants via Unsloth), enabling diverse inference environments.

Watch on YouTube Full article

Build Small Winners Reveal thumbnail

· 53:45

Build Small Winners Reveal

The Build Small Hackathon winners reveal celebrated projects that emphasize building highly functional applications using small, efficient models and local/on-device processing. The community demonstrated a strong focus on 'local first' AI solutions, achieving impressive metrics like 64% of apps running fully offline. Winning projects showcased practical utility—such as scam defense (Jawbreaker) or specialized coaching (Posify)—proving that powerful, real-world impact can be achieved with tiny model footprints.

Key takeaways

  1. Focus on Local and Small Models 18:05

    The hackathon emphasized building small apps using sub-32 billion parameter models. Key metrics showed that 64% of submissions ran fully offline, highlighting the viability of 'local first' AI architectures.

  2. High Utility in Niche Applications 27:20

    Winning projects demonstrated significant real-world utility. Examples include Jawbreaker (a private scam defense tool for suspicious text/emails) and Agenda Parser (breaking down dense local government agendas), proving the value of small, focused solutions.

  3. Advanced Technical Implementation 30:30

    Winners utilized advanced techniques like WebGPU for browser-based real-time games (Parry) and quantization methods (GGUF) to drastically reduce cold boot times, demonstrating high technical polish.

  4. Open Source Model Ecosystem 22:20

    The most popular model family used was Qwen (288 spaces), followed by MiniCPM OpenBMB and Nematron, confirming the community's reliance on diverse open-source models.

Watch on YouTube Full article