The Video Signal

Topic

Serverless Computing

All digests tagged Serverless Computing

Gadgets: Personal app vibe coding that is actually safe — Kenton Varda, Cloudflare thumbnail

· 18:54

Gadgets: Personal app vibe coding that is actually safe — Kenton Varda, Cloudflare

The talk argues that modern personal AI code generation capabilities fundamentally break traditional cloud infrastructure models designed for single-version applications. Kenton Varda introduces 'Gadgets,' a new application paradigm built on Cloudflare Workers. Gadgets allow users' agents to add custom features directly to an app instance (like adding strikethrough formatting or generating complex SVGs) without requiring the core developer to rewrite the entire platform, thus bypassing the limitations of centralized cloud architecture and traditional feature request pipelines.

Key takeaways

  1. Personal AI Codegen Breaks Traditional Cloud Infrastructure

    The current model requires developers to handle all user-requested features (filed in Jira) through massive, multi-year plugin rewrites. This process is slow and often fails. Personal AI agents offer an alternative where users can have their own agent write and add features directly for their specific use case, keeping the core app clean.

  2. The Limitations of Current Web/Cloud Architecture 13:59

    Traditional web apps run on a developer's server, ensuring all users see one 'blessed version.' This centralization prevents user customization. The proposed Gadget model ensures that each gadget is an isolated instance, and sharing/access control is managed by the platform, not the app itself.

  3. Gadgets Security Model 17:05

    The security architecture isolates components: The UI runs in a null origin iframe sandbox with Content Security Policy. Communication is restricted via `postMessage` to the parent frame, which establishes a Cap'n Web RPC session to server code running in a dynamic worker sandbox (durable objects). This prevents XSS bugs from leaking data outside the isolated environment.

Watch on YouTube Full article

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO thumbnail

· 59:10

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO

Modal CTO Akshat Bubna discusses how modern AI workloads—especially those involving agents and elastic inference—exceed the design capabilities of traditional cloud infrastructure like Kubernetes. Modal addresses this by providing a specialized, highly elastic platform built on primitives such as serverless functions, GPU snapshotting, private IPv6 networking, and RDMA support. The discussion highlights a critical shift from optimizing for Developer Experience (DX) to enabling Agent Experience (AX), where the infrastructure must support complex, bursty compute patterns like those found in RL rollouts or custom model deployment across multiple cloud providers.

Key takeaways

  1. Shift from DX to AX 17:33

    Modal is shifting its focus from Developer Experience (DX) to Agent Experience (AX). This shift acknowledges that agents will increasingly operate on infrastructure, making the ability for an agent to self-provision and modify resources via decorators more valuable than manually writing complex YAML files in Kubernetes.

  2. K8s Limitations for AI 4:19

    Traditional orchestration tools like Kubernetes were not designed for highly bursty, compute-heavy workloads common in AI. They struggle with the rapid scaling and specialized resource needs (like accelerators) required by modern models.

  3. Elastic Inference & Autoscaling 21:00

    The platform excels at elastic inference for custom models (e.g., audio, video, robotics). The core challenge is managing unpredictable traffic patterns and scaling rapidly across different regions, a problem Modal addresses through its autoscaling primitives.

  4. Advanced Performance Primitives 27:30

    To achieve frontier-level inference performance, Modal utilizes techniques like speculative decoding (using a smaller 'draft model' to predict tokens ahead of the main model) and DeFlash (a block-based speculator), achieving significant speedups without quality loss.

Watch on YouTube Full article