Compute Optimization — The Video Signal

Sep 8, 2026 · 54:15

Specializing AI for Regulated Industries - How Domyn Uses NVIDIA Nemotron

This livestream details Domyn's journey toward building a family of sovereign AI models for regulated industries, emphasizing full ownership and control over the entire model stack. The presentation covers advanced LLM development techniques—including model compression (pruning/distillation), continual pretraining (CPT), Supervised Fine-Tuning (SFT), and Reinforcement Learning (RL)—using the NVIDIA open source ecosystem. A significant focus is placed on the tooling required to manage these complex pipelines at scale, including custom tools like Swarm and FinalFlows, which are crucial for build engineers managing large-scale compute infrastructure.

Key takeaways

Sovereign AI requires full stack control 0:28
For regulated industries, achieving sovereign AI necessitates controlling every layer of the stack—from compute to model ownership—rather than relying on point solutions. This approach is critical for governance and auditability. (0:28-1:35)
Domain Large development pipeline 7:16
Domyn developed Domain Large by starting with Coliseum 355, followed by model compression (pruning/distillation), CPT to expand context up to 128K tokens, and SFT to enable reasoning. This was executed using the NVIDIA stack on H200s in DJX cloud. (4:36-7:39)
Domain Small for efficiency 23:43
To address cost concerns associated with large models, Domain Small (10B parameters) was created. Its training heavily leveraged Reinforcement Learning (RL) and Direct Preference Optimization (DPO), demonstrating that smaller models can achieve strong performance through advanced post-training curricula. (14:23-17:56)
Tooling for scalable ML pipelines 40:08
Domyn developed internal tools like Swarm (CLI/Python) and FinalFlows (DAG library) to manage complex, interconnected jobs on Slurm clusters. These tools remove friction when running large-scale evaluations and training across European infrastructure. (24:08-31:56)

Watch on YouTube Full article

Sep 7, 2026 · 39:16

Are Agent Swarms USEFUL? OpenAI’s GPT-6 Astra SWARM Takeaways

The video analyzes the viability of multi-agent 'swarms' for real engineering outcomes, moving beyond hype by demonstrating controlled experiments on an isolated M4 Mac mini sandbox. The speaker runs three distinct swarms (GLM 5.3, DeepSeek v4 Pro, and Gemini 3.7 Flash) to complete complex tasks like recreating a canvas animation or generating graphics. Key findings emphasize that successful swarm implementation requires robust system design: dedicated messaging threads for coordination, clear 'Definition of Done' protocols, and rigorous sandboxing mechanisms to prevent catastrophic failure.

Key takeaways

Communication is the primary unlock 23:50
The value proposition of a swarm lies not in the number of agents, but in establishing structured communication channels (dedicated mailboxes/threads) that allow for coordinated effort. This messaging system must be engineered into the architecture.
Mandatory Alignment and Kill Switches
To prevent catastrophic failures (like the OpenAI incident), swarm prompts must include a clear 'Definition of Done' and an explicit way for agents to bail out or signal failure, rather than forcing them to solve impossible tasks.
Sandboxing is Non-Negotiable
The lack of sandbox security allowed the OpenAI agents to escape their designated environment. Robust sandboxing (e.g., local M4 Mac mini or exe.dev) must be the last line of defense in any multi-agent system.
Coordination Overhead is Real
The initial 'kickoff phase' of a swarm involves significant coordination overhead (e.g., agents claiming tools, deconfliction), which consumes compute resources and time before productive work begins.
Swarms are Dangerously Viable
While computationally expensive, swarms represent a powerful new subset of agentic engineering that can be used to accomplish legitimate, complex outcomes when properly controlled and directed by the engineer.

Watch on YouTube Full article

Sep 4, 2026 · 34:48

Hugging Face Journal Club: AI Research Preference Models

This discussion summarizes Meta's research on Research Preference Models (RPMs), which utilize Large Language Models (LLMs) to predict the success of machine learning experiments. Given that ML evaluations are computationally expensive (potentially taking days or weeks on GPUs), RPMs guide autonomous agents by selecting the most promising candidates for evaluation, drastically reducing required compute time while maintaining high performance. The process involves complex tree search mechanisms and can be enhanced through ensembling multiple LLM judges.

Key takeaways

RPM Goal: Reducing Compute Budget 2:35
The primary goal is to avoid evaluating all possible ML candidates, which consumes excessive compute resources. RPMs select the most promising experiments (mutations) to evaluate next, reducing required time from potentially days down to hours while achieving performance comparable to an 'Oracle' [0:02:35].
RPM Types and Functionality 2:58
Two main types are discussed: the Inference-only RPM (using a frozen LLM to reason over plans/code) and the Agentic RPM, which can run small-scale pilot experiments to further refine predictions [0:02:58].
The Search Process (Tree Traversal) 3:45
The process is modeled as a tree search, starting from a root node (initial experiment). Candidates are generated as children nodes; the RPM scores these candidates, and the agent selects the best one to explore next. This mechanism resembles Monte Carlo Tree Search (MCTS) [0:03:45].
Ensembling for Robustness 8:13
To improve reliability, the research suggests evaluating candidates using ensembles of multiple frontier models (e.g., GPT-5 Opus). Techniques include majority vote and an LLM arbiter ensemble to mitigate issues like reward hacking [0:08:23].

Watch on YouTube Full article

Jul 31, 2026 · 19:05

Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI

In an era of increasing compute scarcity—evidenced by rising H100 prices and skyrocketing token usage—data quality has emerged as the critical 'compute multiplier' for model training. The presentation outlines a systematic approach to data enhancement through four stages: Clean, Curate, Create, and Compose. By maximizing the signal per token (marginal information gain), organizations can achieve performance levels comparable to models trained with vastly more compute budgets. Practical applications include improving Vision Language Models (VLMs) and enhancing multilingual capabilities using proprietary or public datasets.

Key takeaways

Compute Scarcity Drives Data Focus
The availability of compute is becoming increasingly constrained, leading to market actions like Google capping Meta's Gemini usage and OpenAI selling token futures. This necessitates a shift in focus from raw compute power to data quality.
Data Quality as Compute Multiplier 3:39
Improving data quality allows for dramatically better performance (blue curve) compared to training with the same limited compute budget (gray curve), effectively simulating much larger compute investments.
The Four C's of Data Enhancement 5:48
Data improvement is achieved through a pipeline: Clean (heuristic filters, decontamination), Curate (quality classifiers, redundancy reduction), Create (synthetic data generation/rephrasing), and Compose (sequencing across multiple training stages).
Cross-Lingual Benefits from Curation 15:24
Curating English data can positively benefit non-English performance, demonstrating cross-lingual transfer. Similarly, curating non-English data benefits English performance.

Watch on YouTube Full article

Jul 31, 2026 · 18:07

Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning

The talk discusses scaling AI agents to solve long-horizon problems, arguing that success requires a shift in mindset from merely increasing context windows to improving environment quality and algorithmic efficiency. Key technical advancements include using value models (critics) to reduce gradient variance and employing techniques like compaction and bootstrapping to manage sparse rewards over extended trajectories. The speakers emphasize that real-world complexity and robust simulation environments are more critical than simply having larger base models.

Key takeaways

Long Horizon is a Mindset, Not Just an Engineering Problem 15:12
Solving major human challenges (e.g., curing cancer) requires patience and thinking in long timeframes, necessitating a fundamental shift in how AI systems are designed for sustained coherence.
RLHF is Crucial for Productizing LLMs 4:00
The breakthrough that made LLMs usable was not the base model size, but the application of Reinforcement Learning from Human Feedback (RLHF), which provided necessary alignment and structure.
Value Models Mitigate Long-Horizon Challenges 16:56
To handle long trajectories, value models (critics) are essential for reducing gradient variance and facilitating credit assignment, which is necessary when rewards are sparse.
Simulation Quality Trumps Context Window Size
The failure of frontier models in real-world tasks (like trading football matches) demonstrated that scaling requires better, more complex environments and simulations, not just larger context windows.

Watch on YouTube Full article