Distributed Computing — The Video Signal

Sep 14, 2026 · 20:50

Every step you take, every call you make: the reliable agent stack — Giselle van Dongen, Restate

This talk introduces Restate, an open-source framework designed to provide a flexible, durable foundation for building resilient, long-running, and stateful agentic systems. Restate addresses the infrastructure gap in agent development by handling complex concerns like retry logic, recovery, session isolation, and process cancellation. It operates as a server proxying requests to the agent service, using an event journal to ensure that processes can survive crashes, redeploys, and long periods of suspension (e.g., waiting for human approval) without losing state or consuming serverless execution time.

Key takeaways

Durable Execution and Resilience 5:40
Restate enables durable execution, allowing an agent process that runs for extended periods (e.g., a week) to crash and restart exactly at the point of failure, rather than starting over. This is achieved by recording all events in a journal.
Modeling State with Virtual Objects 14:00
Instead of modeling agents as simple workflows, Restate uses 'virtual objects' to model persistent, stateful entities (like a session). These objects have unique IDs and isolated state, allowing multiple agents to interact with a single run without interfering with each other's state.
Advanced Control and Interaction 16:40
The framework supports advanced control primitives, allowing external processes to signal, inject state into, or completely cancel an already running agent loop. This capability is crucial for complex, multi-agent interactions.
Low Latency via Push Model 19:20
Unlike traditional workflow orchestrators that poll for new tasks (pull model), Restate uses an event-driven, push model for invocations. This design significantly lowers latency, achieving low latencies (e.g., 45ms p99) even for multi-step workflows.

Watch on YouTube Full article

Aug 27, 2026 · 30:00

Can LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AI

Multi-GPU AI workloads are increasingly bottlenecked by inter-node and intra-node communication links rather than individual GPU compute power. The speaker introduces ParallelKittens, a set of minimal primitives designed to simplify writing high-performance multi-GPU kernels. A benchmark called ParallelKernelBench tests whether frontier LLMs can generate optimized CUDA kernels for complex real-world patterns (e.g., data parallelism across tensor/expert dimensions). While models show promise in generating correct kernels, their ability to reason about critical architectural trade-offs—such as collective ordering, data partitioning, or choosing between transfer mechanisms (Copy Engine vs. TMA)—remains limited.

Key takeaways

The Bottleneck Shift 17:04
Improvements in compute (e.g., A100 to B200) have outpaced improvements in communication links (Intra-node: 3x; Inter-node: 2x). This forces the bottleneck off the individual GPU and onto the interconnects, causing standard PyTorch/NCCL baselines to fall below 50% of their communication-aware roofline.
Kernel Development Simplification 2:20
The speaker's team developed ParallelKittens, a small set of primitives that adds minimal lines to a single GPU kernel but enables state-of-the-art multi-GPU performance in production environments (e.g., Together AI).
LLM Performance on Kernel Generation 20:40
On the ParallelKernelBench, the best frontier models solved 28 out of 87 problems zero-shot. While scaling up attempts increased correctness to 36/87, the performance gain (speedup) plateaued near 31%. Failures are not syntax but stem from inability to reason about complex trade-offs.

Watch on YouTube Full article

Aug 18, 2026 · 16:55

Infra behind Krea 2: How to train and serve at scale — Gabriel Jorge Menezes, Krea.ai

Gabriel Jorge Menezes details the complex infrastructure required to train and serve Krea 2, a diffusion transformer model trained from scratch on thousands of GPUs. The system addresses challenges like silent failures at scale, GPU thermal throttling, and cross-node communication issues by implementing advanced monitoring (tensor core utilization, InfiniBand metrics). For serving, they built a robust architecture using Gang scheduling and Kubernetes features (virtual kubelet, taints/tolerations) to ensure training workloads can utilize the entire cluster while maintaining production uptime through seamless traffic flipping.

Key takeaways

Metrics are essential for large-scale pre-training 9:50
Do not rely on GPU utilization (which is 'a lie'). Instead, monitor tensor core utilization and collect custom metrics like InfiniBand/NVLink errors, as most failures relate to cross-node communication. [5:58], [6:48]
Embrace failure for stability 4:18
When scaling training runs, instead of debugging every crash, it is often more efficient to 'let it crash.' The system should be designed to recover and run successfully on the same nodes over extended periods. [4:18]
Checkpointing must be extremely fast 8:29
To make long training runs survivable, checkpoint aggressively against a high-speed filesystem capable of writing terabytes quickly (e.g., achieving >1 TB/30 seconds). [8:29]
Decouple training and production workloads 11:01
Use a system that allows high-priority training jobs to utilize the entire cluster while seamlessly migrating inference traffic (production) to external providers or other clusters, ensuring zero downtime. [11:01]

Watch on YouTube Full article

Aug 11, 2026 · 31:24

Evolution of agentic surfaces — Gagan Bhat & Isabella Kai He, Anthropic

The evolution of agentic surfaces (from the Messages API to Claude Managed Agents) addresses the challenge that static harnesses become limiting factors as underlying LLMs improve. The core architectural shift is decoupling the agent's 'brain' (the reasoning loop) from its 'hands' (the tool execution environment/sandbox). This separation significantly improves reliability, allows for parallel setup, and boosts performance by achieving up to 60% faster time-to-first-token at P50. Managed Agents provide production-grade infrastructure—including session logging, credential vaults, and self-hosted sandboxes—enabling developers to focus on defining the agent's task and context rather than managing complex operational overhead.

Key takeaways

Harnesses Encode Stale Assumptions 11:42
As models improve (e.g., Opus 4.5 eliminating 'context anxiety'), fixes built into older harnesses become pure overhead, adding latency or discarding cache incorrectly. Managed Agents are designed to be agile and adapt to model evolution.
Decoupling Brain from Hands 20:00
Separating the agent loop (brain) from the tool execution environment (hands/sandbox) improves reliability, allowing the brain to resume from a durable session log even if the hands fail. It also allows model reasoning to start immediately while container setup runs in parallel.
Self-Improving Agents via Dreaming 27:30
The system utilizes durable session logs and memory state, which can be fed into a periodic batch process called 'dreaming.' This process extracts new insights to automatically update the agent's memory, making subsequent sessions smarter.

Watch on YouTube Full article

Aug 10, 2026 · 37:15

Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev)

The video argues that traditional methods of running AI agents—such as allocating a small corner of a local machine or relying solely on containers/CI/CD—create bottlenecks. To achieve true scale and autonomy in an 'AI developer workflow' (ADW), the entire software factory must be moved into dedicated, isolated agent sandboxes (e.g., using exe.dev). This architecture enables complex workflows like running 'Best of N' comparisons across multiple model configurations while maintaining zero blast radius via disposable keys.

Key takeaways

The Bottleneck Problem
If an engineer is 'in the loop,' they are the bottleneck. True scaling requires moving beyond local compute limitations by giving every agent its own isolated computer, achieving isolation, scale, and autonomy.
Three-Tier Architecture for Scale 23:25
The recommended architecture involves an Out-loop orchestrator (on the engineer's machine), an In-sandbox orchestrator (on each VM), and the core Software Factory/ADW agents running inside the sandbox. This allows the top-level agent to kick off work and then go quiet, only requiring human intervention at planning and reviewing stages.
Best of N Pattern 17:05
Sandboxes enable running 'Best of N' patterns by simultaneously executing the same prompt/workflow across multiple agent configurations (e.g., Default, Frontier, Deepest, Open Weights), allowing for comparison and selection of the optimal outcome.
Security and Isolation 27:50
Sandboxes provide critical security by ensuring a 'bounded blast radius.' Agents use ephemeral resources, such as OpenRouter provisioning keys with hard spend caps, which are revoked upon teardown, preventing unauthorized access to production systems (e.g., AWS).

Watch on YouTube Full article

Jul 29, 2026 · 19:09

Wearing the Agent: From Group Chats to Glasses — Sai Krishna Rallabandi

The talk addresses the architectural shift required for agentic systems moving from single-user deployments to complex group chats and wearable devices. The speaker argues that these multi-user environments fundamentally break single-user assumptions, necessitating a complete redesign of memory management and security layers. Key challenges include preventing information leaks when multiple skills collide (security) and efficiently curating evolving conversations over long periods without context bloat (memory).

Key takeaways

Group Settings Break Single-User Assumptions
Agents designed for one user fail in group settings because the complexity of tracking multiple participants, maintaining privacy, and managing shared information requires unique architectural solutions.
Security Must Be Designed at the Action Surface 12:24
Instead of trying to guard all input (which is impossible due to vast surface areas like web pages, emails, and GitHub issues), security must focus on designing a deterministic guard layer that monitors actions—specifically when the agent reads or exports variables (e.g., bash variables) before taking action.
Memory Requires Atomic Information Extraction
Traditional memory methods are insufficient for groups. The system must move beyond storing raw conversation logs and instead extract high-value, atomic facts from the discussion. This requires designing sophisticated auto-raters to determine relevance, hierarchy, and temporal importance.

Watch on YouTube Full article

Jul 24, 2026 · 21:11

Everything Is a Rollout — Alex Shaw + Ryan Marten, Terminal-Bench, Harbor, Laude Institute

Alex Shaw introduces Harbor, a framework designed for evaluating and optimizing AI agents. The talk argues that agent development should be viewed through the lens of machine learning rather than traditional software engineering. This requires treating agent performance as a 'blackbox artifact' and managing it via empirical evaluation—a process formalized by 'rollouts.' Harbor provides the necessary infrastructure (sandboxes, standardized environments) to execute these complex evaluations in parallel.

Key takeaways

Agent Development vs. Software Engineering 5:15
Unlike traditional software engineering where behavior is predictable before execution, agentic coding and AI agents are best treated as blackbox artifacts whose performance requires empirical evaluation (e.g., 'Generated code is best treated as a blackbox artifact').
The Role of Rollouts in Agent Evaluation 10:30
Agent evaluation relies on 'rollouts' within sandboxed environments. This process involves passing the sandbox to the agent, collecting a trajectory, and then passing it to a verifier which produces rewards. Harbor standardizes this universal process.
Harbor as an Interoperable Standard 12:20
Harbor is presented as a common language and open-source framework for specifying agentic environments, allowing interoperability across different agents, models (e.g., GPT 5.5), and sandboxes to maximize data velocity.
Diverse Evaluation Use Cases 17:30
Evaluation can be highly specialized, including assessing how well agents build products (e.g., RampBench), how they use a product's headless mode, or automating internal processes.

Watch on YouTube Full article