Large Language Models — The Video Signal

Sep 21, 2026 · 2:21

Understand the Gemma 4 model family

The Gemma 4 family is a set of multimodal, open-source large language models available in five sizes across four architectures. The models range from efficient, dense, on-device options (E2B, E4B) utilizing Per-Layer Embeddings (PLE), to advanced architectures like the encoder-free 12B model, the Mixture-of-Experts (MoE) 26B model, and the highly capable 31B dense model. Each size is optimized for specific use cases, including dedicated vision and audio processing.

Key takeaways

E2B and E4B Models
These smaller, dense models are optimized for on-device usage and utilize Per-Layer Embeddings (PLE), which are lookup tables for processing queries. They process audio and images using dedicated encoders.
12B Model Architecture
This model is suitable for high-end laptops and employs an encoder-free method, removing dedicated encoders (e.g., audio encoder) and directly projecting audio to the LLM.
26B Model (MoE)
This Mixture-of-Experts (MoE) model uses 26 billion parameters but only activates four billion at any given time ('A' for active). It features a larger vision encoder, making it ideal for difficult vision tasks.
31B Model
This is the most capable dense model in the Gemma 4 family. It utilizes a larger vision encoder and is presented as the top-tier model.

Watch on YouTube Full article

Sep 14, 2026 · 9:20

BF16 vs NVFP4 with Nemotron 3.5 Lightning

This technical deep dive compares two model quantization formats, BF16 and NVFP4, using Nemotron 3.5 Lightning. NVFP4 is presented as a highly efficient, low-precision format that significantly reduces memory footprint and increases throughput for inference. While NVFP4 is recommended for token generation, BF16 or full precision checkpoints are advised for model customization, fine-tuning, or training stability. The technology has expanded, allowing NVFP4 to run efficiently on architectures like Hopper and Ampere, not just Blackwell.

Key takeaways

NVFP4 for Inference Efficiency
Using NVFP4 drastically reduces the memory footprint required to store model weights compared to BF16, making deployment on smaller hardware more feasible. It is recommended for high-speed token generation.
BF16 for Training Stability 3:56
While NVFP4 is ideal for inference, BF16 or full precision checkpoints should be used if the developer plans to customize the model, fine-tune weights, or use custom quantization algorithms, as this ensures better training stability.
Minimal Accuracy Loss 2:30
Despite the quantization from BF16 to NVFP4, the accuracy degradation is reported to be very small (e.g., 99.99% retained), thanks to intrinsic properties of model weights.
Broad Hardware Compatibility 5:40
The ability to run NVFP4 checkpoints is no longer limited to the newest hardware (Blackwell); it is now supported across a wide range of GPUs, including Hopper and Ampere.

Watch on YouTube Full article

Sep 4, 2026 · 0:56

Celebrating one billion Gemma downloads

Google Developers celebrated reaching one billion downloads for the Gemma model family. The discussion highlighted significant advancements in multimodal AI capabilities, specifically noting that Gemma 4 supports video and audio understanding. For developers, key takeaways include utilizing Unsloth Desktop—a local coding agent—for development and fine-tuning, and keeping an eye on future platforms like GenieX, which is designed to integrate highly requested models like Gemma.

Key takeaways

Gemma's Multimodal Capabilities
The latest model in the family, Gemma 4, represents a major breakthrough by supporting both video and audio understanding.
Local Development Tools
Unsloth Desktop was launched as a local coding agent, allowing developers to run models and perform tasks entirely offline. Users can also fine-tune models locally.
Future Platform Roadmap
A platform called GenieX is under development, positioning Gemma as one of the most requested models for future enterprise integration. The team expressed excitement for upcoming versions, including Gemma 5 and Gemma 6.

Watch on YouTube Full article

Aug 17, 2026 · 35:46

How To Turn Evals Into A Better Model

This session details advanced strategies for improving Large Language Model (LLM) performance, arguing that optimizing the evaluation environment (Evals) is often more impactful than immediate fine-tuning. The core components of an eval are the Task Set, the Harness, and the Scoring Function. Furthermore, it provides a deep dive into Reinforcement Learning (RL), outlining its architecture—including inference engines (VLM, SGLang), orchestrators, and trainers—and warning about common pitfalls like reward hacking.

Key takeaways

Prioritize Eval Improvement Over Fine-Tuning 9:39
Before fine-tuning a model, thoroughly audit the evaluation setup. Improvements can often be found by adjusting sampling parameters (e.g., using temperature > 0), swapping harnesses (like Pi for open-source control), or increasing resource allocation/timeouts. [0:08:19]
Understand the Three Parts of an Eval 5:15
Every evaluation consists of three parts: the Task Set (data, prompts, tools); the Harness (the program loop driving LLM interaction with an environment); and the Scoring Function/Reward Function (which can be deterministic or use a judge LLM). [0:00:55]
Reinforcement Learning (RL) is for System Improvement 13:24
RL is a powerful, advanced training algorithm used to improve model capabilities on specific tasks by learning from trial and error. It should be considered the last step after optimizing the eval environment. [0:13:24]
Beware of Reward Hacking 22:24
RL is highly sensitive to weak or poorly designed evals. Models may learn to optimize for a proxy metric (reward hacking) rather than solving the true underlying task, necessitating careful evaluation design and red teaming. [0:22:24]

Watch on YouTube Full article

Jul 29, 2026 · 41:28

Hugging Face Journal Club: Kimi K3

The discussion summarizes the Kimi K3 tech report, detailing a highly complex and scaled training pipeline for large language models. Key innovations include specialized agentic structures (general, deep research, coding), multi-tier on-policy distillation across nine domain experts, and advanced infrastructure techniques like partial rollout schemes and dynamic resource scheduling. The talk emphasizes that the model's performance is achieved through sophisticated engineering efforts rather than a single breakthrough concept.

Key takeaways

Multi-Tier Expert Specialization 2:00
The training pipeline involves generating nine specialized domain experts (3 reasoning levels: low, high, max) for three general domains (general tasks, deep research, coding). These are combined using multi-tier on-policy distillation into a single student model.
Partial Rollout Scheme 4:08
A novel RL technique where the system samples $k$ rollouts from $n$ prompts. It uses an additional parameter $\lambda$ to measure completed rollouts within a budget, updating weights on that subset and pushing incomplete rollouts into subsequent iterations.
Extensible Chat Template 6:42
The Kimi K3 model utilizes an 'extensible token markup language' for its chat template. This design aims to be easy to learn during SFT while remaining flexible enough to accommodate future modalities without re-engineering the core template.
Advanced Infrastructure Scaling 30:35
The system employs sophisticated resource management, including a dynamic rollout auto throttling scheduler based on KV cache pressure and a method for collocating RL training and inference by automatically balancing GPU resources between the two tasks.

Watch on YouTube Full article

Jul 26, 2026 · 17:31

The Messy Reality of Scale: Synthetic Data and Pre-Training — Marah Abdin & Robert McHardy, poolside

The presentation details how poolside scaled their LLM training from smaller models (Laguna M/XS) to a massive 118 billion parameter model for agentic coding. The core strategy involves creating sophisticated synthetic data pipelines—which complement organic data by exposing implicit rationale and structure—and implementing extreme rigor in the pre-training process. Key technical challenges addressed include catching silent failures like broken GPUs, numerical precision loss during tensor parallel accumulation (BF16 to FP32 fixes), and race conditions via weight hashing checks. The resulting model, Laguna S, demonstrates competitive performance against industry leaders on coding benchmarks.

Key takeaways

Synthetic Data for Feature Extraction 10:37
Synthetic data is used not to replace organic data but to complement it by extracting and projecting implicitly hidden features (e.g., implicit rationale, planning, structure). Pipelines are modular, allowing complex tasks to be broken down into stages like multi-stage workflows, cross-domain porting (e.g., math problems to code), or multi-turn chats.
Rigorous Training Validation: Trust Nothing 15:03
To ensure correctness at scale, the team implemented model replica hashing checks; if hashes over weights do not match across distributed replicas, training is immediately halted. Failures observed include silent data corruption from broken GPUs and gradient corruption due to race conditions.
Scaling Success: The 118B Parameter Model
The final model, Laguna S (118B parameters), was trained on 30 trillion tokens across 4,000 GPUs. Early results show strong performance in agentic coding benchmarks (e.g., SWEETbench), outperforming models like GLM 4.5 Air.

Watch on YouTube Full article

Jul 22, 2026 · 10:09

How are large language models trained?

Large Language Models (LLMs) undergo two core training phases: pre-training and post-training. Pre-training uses next token prediction on massive datasets to build foundational language knowledge. Post-training refines this model for safety, helpfulness, and instruction following using techniques like Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). The process requires complex engineering at scale, including managing petabytes of data and utilizing advanced evaluation methods like auto-raters.

Key takeaways

Pre-training: Foundational Knowledge
The initial phase trains the model using next token prediction (a self-supervised task). This allows the LLM to implicitly learn language patterns and world knowledge without manual labeling. Scaling this process is a massive engineering challenge, requiring complex parallelism across thousands of interconnected GPUs, automated recovery systems, and pipelines for petabytes of data.
Post-training: Alignment and Refinement 5:40
This phase focuses on aligning the model to be safe, helpful, and accurate. It primarily involves Supervised Fine-Tuning (SFT) using gold standard examples, or Reinforcement Learning (RL), where a separate Reward Model grades responses for quality, pushing the LLM toward desired behaviors.
Evaluation and Measurement 7:40
Measuring model success requires defining clear criteria. While pre-training measures next word prediction accuracy, post-training focuses on alignment (e.g., helpfulness). Evaluation is scaled using a second LLM—an auto rater—to score responses against defined criteria.

Watch on YouTube Full article

Jul 16, 2026 · 36:00

New Model: Inkling by Thinking Machine on Hugging Face

Thinking Machines announced Inkling, a massive open model with nearly one trillion parameters. It is designed as a natively multimodal architecture, accepting image, text, and audio inputs simultaneously. Key technical features include a Mixture-of-Experts (MoE) structure (975B total / 41B active params), a 1M token context window, and advanced deployment support across multiple frameworks like `transformers`, SGLang, vLLM, and `llama.cpp`. The model is available in BF16 and NVFP4 formats, facilitating high-performance inference on various hardware setups.

Key takeaways

Multimodal Capability & Scale 3:50
Inkling is a true multimodal model that natively processes image, text, and audio inputs using a single architecture. It boasts an immense 1M token context window and was trained on 45T tokens.
Architectural Innovations 5:10
The model utilizes relative attention (replacing RoPE) and a specialized SConv layer to efficiently aggregate hidden states, making it highly efficient for multimodal tasks compared to previous models.
Deployment Flexibility 7:10
Inkling supports multiple deployment paths: BF16 (requiring ~2TB VRAM) and NVFP4 (600GB VRAM). It provides day-zero support for `transformers`, SGLang, vLLM, and `llama.cpp` (including GGUF quants via Unsloth), enabling diverse inference environments.

Watch on YouTube Full article