The Video Signal

Topic

Modal

All digests tagged Modal

Don't Build Agents, Build Environments Instead thumbnail

· 27:19

Don't Build Agents, Build Environments Instead

The talk argues that the most critical challenge in building agentic workflows is not the agent itself, but the underlying environment (the 'background part'). Teams should focus on 'environmental engineering'—designing robust, fast, and persistent development environments ('dev boxes') rather than optimizing the agents. Key architectural shifts involve treating sessions as long-running, stateful processes that require sophisticated isolation between the control plane (planning) and the data plane (execution).

Key takeaways

  1. Focus on Environments, Not Agents

    The difficulty in background agent systems lies in the environment they operate within. The focus should be on building reliable 'dev boxes' that provide necessary resources beyond a bare sandbox.

  2. Jobs vs. Sessions 20:04

    Traditional CI/CD treats runs as stateless 'jobs.' Agents require long-running, resumable 'sessions' that must persist state (e.g., file system snapshots) and potentially outlive the initial machine runtime.

  3. Dev Boxes vs. Sandboxes 3:58

    A sandbox is bare; a 'dev box' must be rich enough to handle complex tasks (e.g., running local servers, taking screenshots, profiling on GPUs), requiring more than just basic Linux access.

  4. Architectural Separation 24:16

    To prevent failure, the system must separate the agent's planning logic (Control Plane) from the risky execution tools (Data Plane). The agent should issue remote calls to a dedicated sandbox rather than running tools collocated with its own process.

Watch on YouTube Full article

How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face thumbnail

· 20:37

How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face

Niels Rogge details how he automated his role at Hugging Face—the 'Google Drive to the hub' team—which focuses on improving the discoverability of machine learning artifacts. He built two systems: an initial deterministic workflow for outreach (using cron jobs and LLM APIs) and a subsequent fully autonomous agent loop for follow-up actions. The architecture leverages modern tooling like Modal, Bash CLI skills, and advanced models (e.g., GLM 5.2) to scale the process of identifying missing artifacts and prompting researchers to publish them on Hugging Face.

Key takeaways

  1. The Problem: Artifact Discoverability

    ML weights and datasets are often published on third-party services (Google Drive, Zenodo) rather than the centralized platform (Hugging Face), hindering discoverability. The goal is to automate outreach to authors.

  2. Initial Automation: Deterministic Workflow 11:43

    The first phase used a deterministic workflow, running as a nightly cron job on GitHub Actions. This approach utilized LLM APIs in predefined steps without an agent framework, offering high predictability and control.

  3. Advanced Automation: Autonomous Agent Loop 15:36

    The follow-up process was automated using a fully autonomous agent loop (e.g., leveraging the Claude agents SDK). This flexible approach allows the agent to use tools and skills, such as Bash and the Hugging Face CLI, to interact with GitHub issues.

Watch on YouTube Full article

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO thumbnail

· 59:10

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO

Modal CTO Akshat Bubna discusses how modern AI workloads—especially those involving agents and elastic inference—exceed the design capabilities of traditional cloud infrastructure like Kubernetes. Modal addresses this by providing a specialized, highly elastic platform built on primitives such as serverless functions, GPU snapshotting, private IPv6 networking, and RDMA support. The discussion highlights a critical shift from optimizing for Developer Experience (DX) to enabling Agent Experience (AX), where the infrastructure must support complex, bursty compute patterns like those found in RL rollouts or custom model deployment across multiple cloud providers.

Key takeaways

  1. Shift from DX to AX 17:33

    Modal is shifting its focus from Developer Experience (DX) to Agent Experience (AX). This shift acknowledges that agents will increasingly operate on infrastructure, making the ability for an agent to self-provision and modify resources via decorators more valuable than manually writing complex YAML files in Kubernetes.

  2. K8s Limitations for AI 4:19

    Traditional orchestration tools like Kubernetes were not designed for highly bursty, compute-heavy workloads common in AI. They struggle with the rapid scaling and specialized resource needs (like accelerators) required by modern models.

  3. Elastic Inference & Autoscaling 21:00

    The platform excels at elastic inference for custom models (e.g., audio, video, robotics). The core challenge is managing unpredictable traffic patterns and scaling rapidly across different regions, a problem Modal addresses through its autoscaling primitives.

  4. Advanced Performance Primitives 27:30

    To achieve frontier-level inference performance, Modal utilizes techniques like speculative decoding (using a smaller 'draft model' to predict tokens ahead of the main model) and DeFlash (a block-based speculator), achieving significant speedups without quality loss.

Watch on YouTube Full article