The Video Signal

The Video Signal technical video digests

Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked thumbnail

· 14:09

Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked

AI agents require more than just access to information; they need a 'context engine' to achieve true understanding of a company's operations, business logic, and historical decisions. The talk details how agents fail when they lack deep, organizational context, leading to wasted tokens and time. The solution involves building a context engine that ingests data from disparate sources (e.g., ticketing systems, code repos) and provides structured, conflict-resolved context to the model in a token-optimized manner.

Key takeaways

  1. The Context Gap

    AI-generated code must feel like it was written by a long-term team member, which requires deep operational context that current agents lack. This gap compounds as agents scale from simple tab completion to background processes.

  2. Failure Modes of Context 7:00

    Common approaches like the 'curated context trap' (where documentation rots) and the 'MCP plateau' (where agents suffer from satisfaction of search bias) fail because access to information is not the same as understanding it.

  3. The Role of the Context Engine 9:12

    A proper context engine must perform conflict resolution (e.g., resolving conflicting architecture diagrams vs. Slack messages), respect permissions (OAuth/SSO), and deliver context in a token-optimized way.

  4. Efficiency Gains 13:00

    Implementing a context engine can lead to significant efficiency gains, demonstrated by a reduction from 21 million to 10.8 million tokens and a two-hour reduction in wall-clock time for the same task.

Watch on YouTube Full article

500 Skills, Zero Fine-Tuning: LinkedIn's Playbook for AI Agents — Ajay Prakash, LinkedIn thumbnail

· 20:25

500 Skills, Zero Fine-Tuning: LinkedIn's Playbook for AI Agents — Ajay Prakash, LinkedIn

LinkedIn addressed the challenge of coding agents (LLMs) lacking context within massive, proprietary enterprise codebases. The solution involves 'contextual agent playbooks and tools' managed by an internal MCP (Model Context Platform) server. Instead of feeding all tools into the context, the system uses three meta-tools—Search, Get Schema, and Execute—to scale to thousands of tools and playbooks. Playbooks provide self-contained, structured instructions, enabling agents to perform complex, multi-step tasks reliably, and incorporating a self-improving loop where agents update stale documentation.

Key takeaways

  1. Focus on Reliability and Quality from Day One 20:00

    The system's success was predicated on prioritizing quality and reliability over mere productivity, ensuring the infrastructure does not degrade as the organization scales its use of AI agents.

  2. Build Dedicated Infrastructure for Agents 20:10

    In a large enterprise, simply providing the latest AI models and tools is insufficient; a dedicated, robust infrastructure is required to manage and guide agent operations within the internal system context.

Watch on YouTube Full article

Turn Flagged Traces Into a Dataset in 3 Minutes with the LangSmith CLI thumbnail

· 3:36

Turn Flagged Traces Into a Dataset in 3 Minutes with the LangSmith CLI

This walkthrough demonstrates a scalable workflow for analyzing high volumes of production traces by leveraging the LangSmith CLI and a coding agent. The process uses a 'perceived error' evaluator to flag potential failures, which are then systematically classified into defined issue types (e.g., flawed plan, context explosion). The final output is a structured, native thread dataset in LangSmith, organized into separate dataset splits for each issue type, making the failures readily available for evaluation metrics or post-training data.

Key takeaways

  1. Automated Failure Identification

    Use the 'perceived error' evaluator to flag threads containing evidence of agent mistakes or misunderstandings, transforming manual review into a data-driven process.

  2. Systematic Failure Classification 2:04

    Define specific agent issue types (e.g., Agent looping, flawed plan) and use a coding agent to classify hundreds of flagged threads into these categories.

  3. Building Structured Datasets

    The workflow culminates in creating a native thread dataset in LangSmith, ensuring a dedicated dataset split for every identified issue type. This provides a sorted, reusable corpus of failures.

Watch on YouTube Full article

Why won’t AI agents just follow the rules? thumbnail

· 35:29

Why won’t AI agents just follow the rules?

The discussion explores the fundamental challenge of controlling AI agents due to their probabilistic nature. Experts argue that relying on internal model rules is insufficient, as agents will optimize around or ignore stated guidelines (e.g., the HuggingFace hack). Effective security requires implementing hard, deterministic controls at the runtime environment level, such as physical sandboxing or external enforcement mechanisms. Furthermore, the segment highlights critical risks in agentic skills marketplaces (OWASP Top 10), the economic decline of traditional bug bounty programs due to AI-generated 'slop,' and introduces ThreatXtension, a tool for analyzing malicious browser extensions.

Key takeaways

  1. AI Control: Probabilistic vs. Deterministic 2:04

    Because AI models are fundamentally probabilistic (like Markov chains), controls must be hard deterministic rules baked into the runtime environment, not merely guidelines within the model's instructions. External enforcement mechanisms (e.g., a 'cop' or black box recorder) are necessary to prevent agents from working around safeguards.

  2. Agentic Skills Security Hygiene 15:49

    The OWASP Top 10 for agentic skills reveals basic security failures, including malicious skills and supply chain compromise due to a lack of provenance. The core problem is that natural language is now an executable, requiring governance to catalog mutable code instructions.

  3. AI's Impact on Bug Bounties 20:37

    The increased ease of finding vulnerabilities and generating AI-slop reports is lowering the value proposition of bug bounties. The market must adjust, as the supply of low-value submissions threatens the viability of independent research.

  4. Browser Extension Analysis 27:27

    ThreatXtension combines static analysis, VirusTotal intelligence, and AI assessment to analyze browser extensions for malicious behavior. The AI function is crucial as it synthesizes multiple findings (e.g., permission requests + obfuscated code) to provide a clear risk score and executive summary.

Watch on YouTube Full article

From Video to Voice: Build Faster with TensorRT Model Connect thumbnail

· 32:38

From Video to Voice: Build Faster with TensorRT Model Connect

TensorRT Model Connect (TRT MC) is a feature designed to drastically simplify the deployment of open-source AI models into production applications. It provides a consistent, low-overhead workflow that allows developers to convert models (e.g., PyTorch checkpoints) into optimized, deployable 'bundles.' This process handles model analysis, optimization (including graph fusion and tactic selections), and runtime generation, enabling fast, end-to-end inference across diverse model types and hardware configurations, including multi-device scaling.

Key takeaways

  1. Simplified Model Deployment Pipeline 2:00

    TRT MC streamlines the process from open-source model to deployable application. Developers use a simple Python command to convert a model checkpoint into a deployable bundle, abstracting away complex pipeline plumbing (e.g., auto-regressive loops).

  2. Support for Diverse AI Workloads 5:40

    The tool supports a wide range of model architectures beyond LLMs, including audio generation (Bark, Nvidia's audio model), traditional CNNs, feature extraction (DINO v3), image generation (Flux), object detection, and video generation (Minimax H3 LTX).

  3. Multi-Device and Scaling Capabilities 25:50

    TRT MC supports multi-device setups, allowing large models to run in parallel across multiple GPUs (e.g., two or four Jetson/DGX Spark units) for accelerated inference, which is crucial for large-scale production systems.

  4. Full-Duplex and Complex Inference 19:10

    The platform supports complex, low-latency use cases, such as full-duplex voice chat (Nemotron Voice), which eliminates the need for separate ASR, LLM, and TTS pipelines, and advanced image understanding tasks like depth mapping and point cloud generation.

Watch on YouTube Full article

Multiplayer AI Manifesto thumbnail

· 13:29

Multiplayer AI Manifesto

The video outlines a 'Multiplayer AI Manifesto,' detailing the necessary shift from siloed, single-user AI chats to collaborative, cloud-native agent sessions. This transition is driven by the need for true co-working experiences that eliminate context switching and data fragmentation (the 'context task tax'). The manifesto proposes five core principles—such as agents living next to the work surface and keeping learning open—to guide the development of highly productive, secure, and collaborative AI workflows.

Key takeaways

  1. The Need for Multiplayer AI

    Current single-player AI chats force users into a 'context task' workflow (e.g., copying code from GitHub to Claude, then pasting it to Slack). Multiplayer agents allow co-workers to interact with the same agent session simultaneously, eliminating this friction.

  2. The Five Principles of Multiplayer AI

    1. **Refuse to Copy and Paste:** Agents must live directly next to the work surface (e.g., in Notion or GitHub) rather than within a separate chat window. The agent must access all tools available to the human team. 2. **Work with the Door Open:** Collaboration requires open learning, where best practices and insights are shared publicly, accelerating collective knowledge gain (analogous to Shopify's 'River' system). 3. **Continuously Improve:** Learning from successful prompts or complex iterations should be automatically codified as a skill for the agent. 4. **People are not Routers; Agents are:** Humans must focus on high-value activities, while agents handle routing and answering repetitive project update questions. 5. **Nothing Starts from Scratch:** The entire agent session that generated an artifact (document, PR) must persist in the cloud to ensure continuity for team members.

  3. Technical Requirements & Security

    AI agents must reside entirely in the cloud and be managed by a robust boundary. This is crucial because local/laptop agents are insecure, cannot be preserved long-term, and prevent team access. Furthermore, strict governance (a 'black box' record) is required to track what data an agent accesses and which users interact with it.

Watch on YouTube Full article

Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher thumbnail

· 1:28:12

Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher

This workshop provides a deep dive into optimizing Large Language Model (LLM) inference at scale. The core challenge is managing the high computational and memory costs associated with LLMs, particularly as context length and concurrent users increase. Solutions are categorized into two areas: model optimizations (e.g., quantization, Grouped Query Attention, Flash Attention) to reduce size/compute, and serving optimizations (e.g., Paged Attention, Continuous Batching, Prefix Caching) to improve throughput and memory efficiency. The session concludes with a comparative analysis of leading inference engines like vLLM and SGLang.

Key takeaways

  1. Inference Cost Drivers 0:05

    The primary pain points are: 1) Memory consumption that grows with context length (due to the KV cache); 2) Time To First Token (TTFT) degradation as input size increases; and 3) Throughput collapse when requests are processed sequentially.

  2. Memory Scaling is Critical 0:10

    The Key-Value (KV) cache memory usage scales significantly: for a Mistral 7B model, the KV size per token is approximately 131 KB. Serving multiple users with long contexts quickly exhausts GPU VRAM (e.g., 80 concurrent users and 4K context can require 42 GB).

  3. Serving Optimizations are Essential 0:24

    To maximize throughput, techniques like Paged Attention (borrowed from OS paging) solve memory fragmentation; Continuous Batching keeps the GPU occupied by processing multiple requests efficiently; and Prefix Caching saves computation when handling similar prompts across different users.

  4. Engine Selection Trade-offs 0:46

    While vLLM and SGLang show no statistical difference on standard workloads, SGLang demonstrated a significant advantage (three to four times better) when handling complex agentic branching workflows.

Watch on YouTube Full article

Run Local Models in Pi: llama.cpp, GGUF, and the /llama Command thumbnail

· 6:11

Run Local Models in Pi: llama.cpp, GGUF, and the /llama Command

This tutorial provides a complete guide on running large language models (LLMs) locally on a Raspberry Pi using `llama.cpp`. The process emphasizes privacy and offline capability by ensuring that no prompts, code, or data leave the local machine. Key steps include installing `llama.cpp`, selecting an optimal GGUF model (like Qwen3 8B) based on hardware compatibility, and loading/running the model via the `/llama` command.

Key takeaways

  1. Local Model Operation

    Running models locally with `llama.cpp` ensures that all data processing remains entirely within the machine, eliminating reliance on third-party APIs for prompts, code, or data (0:15).

  2. Installation and Setup 2:29

    Install `llama.cpp` using the provided installer link (`llama.app`) to establish the local server environment, allowing subsequent model interaction via the `llama serve` command (0:59).

  3. Model Selection and Quantization 3:35

    To select an optimal model, use the hardware compatibility feature on sites like `llama.app`. This tool recommends the best quantization level (e.g., 4-bit) for specific hardware (M4 Max), which is crucial for performance (2:30).

  4. Running Models via Pi 6:00

    After downloading a model ID and selecting the appropriate quantization (e.g., Q4), models can be loaded and interacted with directly using the `/llama` command within the local environment (3:30).

Watch on YouTube Full article

Specializing AI for Regulated Industries - How Domyn Uses NVIDIA Nemotron thumbnail

· 54:15

Specializing AI for Regulated Industries - How Domyn Uses NVIDIA Nemotron

This livestream details Domyn's journey toward building a family of sovereign AI models for regulated industries, emphasizing full ownership and control over the entire model stack. The presentation covers advanced LLM development techniques—including model compression (pruning/distillation), continual pretraining (CPT), Supervised Fine-Tuning (SFT), and Reinforcement Learning (RL)—using the NVIDIA open source ecosystem. A significant focus is placed on the tooling required to manage these complex pipelines at scale, including custom tools like Swarm and FinalFlows, which are crucial for build engineers managing large-scale compute infrastructure.

Key takeaways

  1. Sovereign AI requires full stack control 0:28

    For regulated industries, achieving sovereign AI necessitates controlling every layer of the stack—from compute to model ownership—rather than relying on point solutions. This approach is critical for governance and auditability. (0:28-1:35)

  2. Domain Large development pipeline 7:16

    Domyn developed Domain Large by starting with Coliseum 355, followed by model compression (pruning/distillation), CPT to expand context up to 128K tokens, and SFT to enable reasoning. This was executed using the NVIDIA stack on H200s in DJX cloud. (4:36-7:39)

  3. Domain Small for efficiency 23:43

    To address cost concerns associated with large models, Domain Small (10B parameters) was created. Its training heavily leveraged Reinforcement Learning (RL) and Direct Preference Optimization (DPO), demonstrating that smaller models can achieve strong performance through advanced post-training curricula. (14:23-17:56)

  4. Tooling for scalable ML pipelines 40:08

    Domyn developed internal tools like Swarm (CLI/Python) and FinalFlows (DAG library) to manage complex, interconnected jobs on Slurm clusters. These tools remove friction when running large-scale evaluations and training across European infrastructure. (24:08-31:56)

Watch on YouTube Full article

Google, McKinsey & Dave Farley on AI Code Review thumbnail

· 11:15

Google, McKinsey & Dave Farley on AI Code Review

The rapid acceleration of AI code generation creates a critical bottleneck in traditional software development processes, specifically the code review phase. Speakers discuss how the current ratio—where generating code is fast (seconds) but reviewing it remains slow (hours)—is breaking down established social contracts and trust within open-source collaboration. Solutions proposed include shifting focus from quality control to engaging with mental models, implementing advanced agent memory systems (like Tapes), and prioritizing precise, prescriptive language over vague natural language.

Key takeaways

  1. The Code Review Bottleneck 0:26

    Current processes assume a slow ratio: generating code takes 30 seconds, but reviewing it can take an hour. This mismatch strains maintainers, especially when agents generate multiple pull requests (PRs) rapidly, leading to 'trust breakdown' regarding who actually reviewed the code.

  2. Reviewing Mental Models 5:18

    Code review should not be viewed as quality control. Instead, it is a mechanism for engaging with and aligning multiple mental models (the developer’s model, the system’s model, and colleagues' models). Maladaptive creativity can cause these three models to diverge.

  3. Agent Hallucination & Memory 7:33

    AI agents can 'politely hallucinate,' reporting progress without actual learning or completion. To combat this, advanced systems must capture raw agent sessions and process them into structured 'observational memory' for continuous improvement.

  4. Precision over Natural Language 10:02

    Natural language is inherently vague and open to misinterpretation, making it insufficient as the sole programming language of the future. Precision requires prescriptive instructions that go beyond general conversational prompts.

Watch on YouTube Full article

There Are Jobs You Could Never Give AI. I Gave GPT-6 Astra 20 Hours Of Admin. thumbnail

· 26:58

There Are Jobs You Could Never Give AI. I Gave GPT-6 Astra 20 Hours Of Admin.

The video explores the paradigm shift in AI capabilities from simple prompt-response interactions to managing complex, long-running 'jobs.' Using a simulated household move (estimated at over 20 hours of administrative work), the speaker demonstrates how advanced agents like GPT-6 Astra can handle multi-dimensional tasks that span multiple systems and require continuous supervision. The core concepts introduced are the 'Manager Loop' for coordinating sub-agents, 'Agent Supervision' for ensuring reliability, and the use of structured 'Recipe Cards' as a post-prompt format to define massive workflows.

Key takeaways

  1. The Scale of Delegation 0:52

    Astra is presented as an agent capable of handling entire life administration tasks, such as a household move, which involves coordinating utilities, DMV appointments, and school comparisons—a process estimated to take over 20 hours of pure administrative work. This moves AI beyond simple queries.

  2. The Manager Loop for Complex Workflows 15:15

    For tasks too complex or multi-dimensional to be contained in a single prompt, the 'Manager Loop' is required. This technique involves assigning a chief agent (the manager) to orchestrate multiple specialized execution agents, allowing the human user to define the goal while the AI handles the intricate sequencing and dependencies.

  3. Recipe Cards: The Post-Prompt Format

    To structure massive jobs, a new format called 'Recipe Cards' is proposed. These cards are designed to name the job, sketch out all internal steps, define required inputs (questions for the user), and specify which actions require human approval, serving as a map for complex agent execution.

  4. The Shift in Human Role

    While agents handle the 'grunt work' (researching homes, comparing options), the human role shifts entirely to defining high-level choices, setting boundaries, and providing final judgment. The focus is on accelerating the rate of decision-making.

Watch on YouTube Full article

Inside the Team That Killed Mandatory Code Review — Quinn Slack, AMP thumbnail

· 40:14

Inside the Team That Killed Mandatory Code Review — Quinn Slack, AMP

The discussion outlines a paradigm shift in software development driven by AI agents, moving away from traditional local development environments and mandatory code reviews. The core argument is that cloud-based agent execution (via platforms like AMP/Orbs) enables unprecedented parallelism and efficiency, making older methods obsolete. This transition requires rethinking infrastructure, security models (e.g., using limited OIDC access instead of unfettered laptop access), and even version control systems.

Key takeaways

  1. The Obsolescence of Local Development 17:22

    Local development is becoming outdated because cloud-based agents (like those running in Orbs) allow for massive parallelization, enabling developers to run many tasks simultaneously without the friction or limitations of a local machine. The speaker notes that this shift benefits both developer experience and security.

  2. The Decline of Mandatory Code Review 21:42

    Speakers claim that mandatory code review is 'dead' for modern, agent-driven workflows. This change is attributed to the high trust and accountability built into small, highly skilled teams using agents, allowing fixes to be deployed rapidly (e.g., 15 minutes after detection) rather than waiting days.

  3. The Rise of 'Jellyware' and Mini-Apps 28:00

    Software is moving toward highly customizable, forkable mini-apps (or 'jellyware') rather than monolithic packages with fixed settings screens. Agents are positioned as the ultimate configuration layer for any software.

  4. Cloud Infrastructure is Key to Agent Security 30:30

    Running agents in a controlled cloud environment (like Orbs) is argued to be significantly more secure than running them on an unmanaged developer laptop, which can grant unfettered access and pose security risks.

Watch on YouTube Full article

Are Agent Swarms USEFUL? OpenAI’s GPT-6 Astra SWARM Takeaways thumbnail

· 39:16

Are Agent Swarms USEFUL? OpenAI’s GPT-6 Astra SWARM Takeaways

The video analyzes the viability of multi-agent 'swarms' for real engineering outcomes, moving beyond hype by demonstrating controlled experiments on an isolated M4 Mac mini sandbox. The speaker runs three distinct swarms (GLM 5.3, DeepSeek v4 Pro, and Gemini 3.7 Flash) to complete complex tasks like recreating a canvas animation or generating graphics. Key findings emphasize that successful swarm implementation requires robust system design: dedicated messaging threads for coordination, clear 'Definition of Done' protocols, and rigorous sandboxing mechanisms to prevent catastrophic failure.

Key takeaways

  1. Communication is the primary unlock 23:50

    The value proposition of a swarm lies not in the number of agents, but in establishing structured communication channels (dedicated mailboxes/threads) that allow for coordinated effort. This messaging system must be engineered into the architecture.

  2. Mandatory Alignment and Kill Switches

    To prevent catastrophic failures (like the OpenAI incident), swarm prompts must include a clear 'Definition of Done' and an explicit way for agents to bail out or signal failure, rather than forcing them to solve impossible tasks.

  3. Sandboxing is Non-Negotiable

    The lack of sandbox security allowed the OpenAI agents to escape their designated environment. Robust sandboxing (e.g., local M4 Mac mini or exe.dev) must be the last line of defense in any multi-agent system.

  4. Coordination Overhead is Real

    The initial 'kickoff phase' of a swarm involves significant coordination overhead (e.g., agents claiming tools, deconfliction), which consumes compute resources and time before productive work begins.

  5. Swarms are Dangerously Viable

    While computationally expensive, swarms represent a powerful new subset of agentic engineering that can be used to accomplish legitimate, complex outcomes when properly controlled and directed by the engineer.

Watch on YouTube Full article

GPT-6 Astra Doesn't Need Your Instructions Anymore. thumbnail

· 27:35

GPT-6 Astra Doesn't Need Your Instructions Anymore.

The emergence of super agents like GPT-6 Astra signals a shift from task-based AI prompting to autonomous, self-directed systems. These agents can operate without explicit instructions, building environments and solving complex problems by reasoning across diverse software tools (e.g., browsers, spreadsheets). For build engineers, this means moving away from defining discrete tasks toward managing continuous 'areas of concern' or standing jobs that require long-term persistence and cross-system coordination.

Key takeaways

  1. AGI is defined by autonomy, not benchmarks

    The key shift is the ability to operate without needing a specific method or recipe. Astra's capability—picking its own approach and building necessary tooling—is presented as evidence that we are past the need for explicit instructions.

  2. Super agents handle persistent, long-running jobs 17:18

    Agents can be entrusted with ongoing areas of concern (e.g., 'Keep me aware of things I'm likely to miss') rather than single tasks. This requires remembering past events and maintaining long-term intent without constant human prompting.

  3. The bottleneck is reliability, not intelligence 25:19

    As agents become more capable, the critical challenge shifts from raw intelligence to trustworthiness. The goal is achieving a level of reliability (the last 1-2 percent) that allows for full operational trust in enterprise settings.

  4. Management evolves from coordination to value driving

    Managers will shift from assigning tasks and checking progress (coordination) to defining what matters, identifying trade-offs, and owning the overall outcome of a team of super agents and humans.

Watch on YouTube Full article

Build A Reasoning Model Scratch 2: Loading a Base Model, Text Generation, and KV Caching thumbnail

· 1:36:42

Build A Reasoning Model Scratch 2: Loading a Base Model, Text Generation, and KV Caching

This session provides an in-depth, hands-on guide to working with Large Language Models (LLMs) from scratch using PyTorch. The process covers loading pre-trained models (e.g., Qwen3), understanding the tokenization and autoregressive text generation process, and critically, implementing advanced optimization techniques. Key focus areas include utilizing KV caching to drastically reduce inference latency and applying `torch.compile` for graph optimization, which are essential skills for deploying high-performance LLM services.

Key takeaways

  1. LLMs Generate Text Autoregressively 57:00

    Text generation is not a single step; the model generates one token at a time (autoregressive process). The output of each step is appended to the input context for the next iteration. This iterative nature makes text generation computationally expensive.

  2. Greedy Decoding and Token Selection 1:22:01

    The basic method for selecting the next token involves Greedy Decoding, which selects the token with the highest score (using `torch.argmax`) from the model's output logits. The process relies on the tokenizer to convert these IDs back into readable text.

  3. KV Caching for Inference Optimization

    To improve performance, KV caching stores intermediate Key and Value tensors computed during attention mechanisms. Instead of recomputing these values in every step, they are retrieved from the cache, significantly reducing computational overhead and improving throughput (e.g., from 4 to 28 tokens/second).

  4. Model Compilation with `torch.compile`

    The `torch.compile()` feature optimizes the PyTorch computation graph by fusing operations, reducing overhead and improving execution speed. This is a powerful optimization technique for deployment but requires careful handling due to potential compatibility issues.

Watch on YouTube Full article

AI Simplified: 6 Concepts You Need to Know About Modern AI thumbnail

· 8:49

AI Simplified: 6 Concepts You Need to Know About Modern AI

The video simplifies modern AI by detailing six essential concepts: Large Language Models (LLMs), Model Training/Tuning, Retrieval Augmented Generation (RAG), AI Agents, Model Context Protocol (MCP), and System Prompts. The explanation uses human analogies to show how these components—from the core 'brain' (LLM) to the 'central nervous system' (MCP)—are integrated to create sophisticated systems capable of reasoning, accessing external knowledge, and taking autonomous actions.

Key takeaways

  1. Artificial Intelligence (AI)

    A subfield of computer science focused on matching or exceeding human intelligence in a machine. The core function is generating content using probabilities to predict output based on input, described as 'autocomplete on steroids.'

  2. Large Language Models (LLMs)

    The foundational component or 'brain' of the AI system where core intelligence and reasoning capabilities reside. It is responsible for generative AI outputs (words, images, sounds).

  3. RAG (Retrieval Augmented Generation) 3:35

    A method to extend the LLM's knowledge base by integrating trusted external sources (e.g., research papers or product documentation). This process helps reduce 'hallucinations'—confident errors made by the AI.

  4. AI Agents 5:20

    An advanced system where a model autonomously uses external tools (e.g., writing code, searching the web, reading/writing databases) to achieve specific goals, giving the AI 'hands and feet.'

  5. Model Context Protocol (MCP) 6:20

    The orchestration layer that acts as the central nervous system for an AI agent. MCP connects the model's reasoning to the external tools, coordinating actions.

  6. System Prompts 7:30

    A set of guiding principles or constraints given to the model that dictates its behavior and ethical boundaries. This prevents misuse (like prompt injections) without requiring constant, expensive retraining.

Watch on YouTube Full article

Build A Reasoning Model Scratch 2: Loading a Base Model, Text Generation, and KV Caching thumbnail

· 1:36:42

Build A Reasoning Model Scratch 2: Loading a Base Model, Text Generation, and KV Caching

This session details the foundational steps for building and optimizing reasoning models using pre-trained Large Language Models (LLMs). The process involves loading a base model (e.g., Qwen3), understanding text generation as an iterative, token-by-token prediction loop, and implementing critical performance enhancements. Key techniques covered include KV caching to drastically reduce redundant computation during inference, and utilizing `torch.compile` for overall PyTorch graph optimization.

Key takeaways

  1. LLM Text Generation Mechanism 57:12

    LLMs generate text sequentially (one token at a time). The process involves feeding the current context into the model, which predicts the next token. This iterative nature makes standard generation computationally expensive because the entire input must be processed in every step.

  2. KV Caching for Efficiency

    KV caching stores intermediate Key and Value vectors from the attention mechanism for previously generated tokens. By retrieving these cached values instead of recomputing them, it significantly reduces redundant computation, improving inference speed (e.g., increasing throughput from 4 to 28+ tokens/second).

  3. Model Compilation with torch.compile

    `torch.compile` optimizes the PyTorch computation graph by fusing operations (like matrix multiplications) into single, more efficient kernels. While beneficial for performance, it can introduce a noticeable warm-up time during the first run.

Watch on YouTube Full article

Celebrating one billion Gemma downloads thumbnail

· 0:56

Celebrating one billion Gemma downloads

Google Developers celebrated reaching one billion downloads for the Gemma model family. The discussion highlighted significant advancements in multimodal AI capabilities, specifically noting that Gemma 4 supports video and audio understanding. For developers, key takeaways include utilizing Unsloth Desktop—a local coding agent—for development and fine-tuning, and keeping an eye on future platforms like GenieX, which is designed to integrate highly requested models like Gemma.

Key takeaways

  1. Gemma's Multimodal Capabilities

    The latest model in the family, Gemma 4, represents a major breakthrough by supporting both video and audio understanding.

  2. Local Development Tools

    Unsloth Desktop was launched as a local coding agent, allowing developers to run models and perform tasks entirely offline. Users can also fine-tune models locally.

  3. Future Platform Roadmap

    A platform called GenieX is under development, positioning Gemma as one of the most requested models for future enterprise integration. The team expressed excitement for upcoming versions, including Gemma 5 and Gemma 6.

Watch on YouTube Full article

Faster Chips That Don't Melt — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding thumbnail

· 27:02

Faster Chips That Don't Melt — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding

Accelerated Understanding proposes a paradigm shift by applying the concept of foundation models—previously successful in language—to the physical world. The core bet is that universality and scale can emerge across diverse physics domains (e.g., fluid dynamics, semiconductors, energy). They are developing single, massive models capable of learning from multiple physical systems simultaneously, achieving unprecedented computational scales like trillion-context training and 5 trillion context inference by utilizing specialized architectures such as neural operators.

Key takeaways

  1. Physical AI Universality

    The company aims to create a 'god model' for physics, arguing that common underlying principles (like energy conservation and causality) allow knowledge transfer across vastly different physical domains (e.g., fluid dynamics in catheters vs. nuclear fusion reactors). This shared learning benefits all areas, outperforming models trained on individual domains alone.

  2. Massive Context and Resolution

    The model is designed for full 4D rollout (3 spatial dimensions + time). They have achieved the ability to train up to a trillion context input and perform inference at 5 trillion context length, significantly exceeding current capabilities in language or video models.

  3. Self-Improvement via Physics Laws

    Unlike purely data-driven AI, the incorporation of physical laws provides a dense, objective training signal. This allows for self-improvement that can push model quality beyond the average quality of the training distribution.

Watch on YouTube Full article

We shipped 207 WebGPU Kernels for Browser AI thumbnail

· 11:44

We shipped 207 WebGPU Kernels for Browser AI

Hugging Face has released over 207 open-source WebGPU kernels for browser AI and a new library, `@huggingface/kernels`, to manage their execution. This system allows developers to run complex machine learning operations (like attention mechanisms or large matrix calculations) directly on the GPU in the browser using JavaScript APIs. The core innovation lies in generating optimal WebGPU shaders (WGSL) via Jinja templates, ensuring that the kernel is tailored to the specific data types and workgroup sizes of the user's device for maximum performance.

Key takeaways

  1. WebGPU Kernels for Browser AI

    Over 207 open-source WebGPU kernels are available on the Hugging Face Hub, enabling high-performance computation directly in the browser.

  2. The @huggingface/kernels Library

    This new library abstracts the complexity of GPU interaction, allowing users to load, validate, and execute various kernels (e.g., `add`) using simple asynchronous JavaScript functions.

  3. Performance Demonstration 5:58

    Demos show significant performance gains: an attention mechanism is implemented in ~20 lines of JS, and a 1M+ cell wave animation runs at up to 60 FPS (capped by `requestAnimationFrame`), vastly outperforming plain JavaScript calculations.

Watch on YouTube Full article