Topic

Scaling

All digests tagged Scaling

How Anthropic Builds: Lessons from Labs — Mike Krieger, Anthropic thumbnail

· 26:11

How Anthropic Builds: Lessons from Labs — Mike Krieger, Anthropic

Mike Krieger discusses the evolution of building with AI models, emphasizing that ambition requires being 'unreasonable' in tool usage and process design. He details advanced development workflows—such as porting large codebases (e.g., Python to TypeScript) over a weekend using LLMs—and outlines modern organizational structures for research labs (like Anthropic's), which prioritize rapid iteration through 'persevere or pivot' cycles rather than rigid project management. Key technical shifts include moving from task delegation to expressing end-state goals, and utilizing advanced features like tagging for multi-agent, proactive code development.

Key takeaways

  1. Be Unreasonable in AI Usage 4:15

    The first generation of AI products often constrained access to tools and degrees of freedom. To maximize potential, users must be 'unreasonable' in their prompts, moving beyond simple task delegation to defining the desired end state.

  2. The Importance of Intent Over Code Lines 17:17

    In large codebases (e.g., 2,000 line PRs), the bottleneck is no longer review time but human comprehension. The trend is shifting toward sharing 'Claude code artifacts' that explicitly lay out intent, trade-offs, and rationale alongside the code.

  3. Labs Structure for Rapid Iteration 19:13

    Anthropic Labs operates on a two-week 'persevere or pivot' cycle. Teams are assembled around 'bets,' and the structure deliberately avoids tying people to specific projects, allowing for quick disbandment and reassembly.

  4. Scaling Requires Pre-Measurement 23:50

    Drawing from Instagram's scaling experience, it is crucial to pre-measure everything that might be needed (metrics) and implement thoughtful knobs and feature flags for dynamic runtime configuration.

Watch on YouTube Full article

Build a secure computer for your agent thumbnail

· 36:51

Build a secure computer for your agent

As agents evolve beyond simple Q&A to writing and executing complex code (e.g., data analysis, software engineering), they require an isolated, persistent computing environment. The session details the architecture of LangSmith Sandboxes, a solution designed to provide production-ready, secure execution by giving each agent its own disposable computer. This system addresses critical challenges like untrusted model-generated code, container escape vulnerabilities, and scaling limitations inherent in traditional local or vanilla container setups.

Key takeaways

  1. Production-Grade Isolation 20:45

    Sandboxes utilize hardware virtualized microVMs for kernel-level isolation, ensuring that malicious code cannot escape the environment (preventing issues like container escapes). This level of separation is necessary because agents run untrusted, model-generated code.

  2. Scalability and Performance 23:50

    The platform supports scaling from one to thousands of isolated sandboxes in parallel. Benchmarking shows a median spin-up time of approximately one second, making it suitable for high-volume, user-facing applications.

  3. Secure Credential Management 25:20

    The O proxy acts as a man-in-the-middle proxy controlling all egress from the VM. This ensures that credentials never touch the runtime, significantly mitigating risks associated with data exfiltration or malicious network calls.

  4. State Persistence and Resilience 27:10

    Sandboxes support persistent state across long-running, interruptible tasks. Users can snapshot and restore the entire environment (including file system and memory), allowing for rollbacks or forking to test multiple scenarios.

Watch on YouTube Full article