AI Agents for Performance: Ship Faster, Pay Less — Rajat Shah, Netflix
Summary
This talk outlines how Netflix leveraged AI agents to automate performance engineering, addressing the bottleneck of manually identifying and fixing inefficient code patterns in production services. The process involves feeding profiling data (call stacks, CPU usage) into an LLM agent, which identifies suboptimal algorithms (e.g., quadratic-time patterns). The agent then proposes a fix, validates it against a canary deployment using real production traffic, and can even scale the fix across multiple services by identifying common anti-patterns in a centralized catalog. This shifts performance optimization from a reactive, manual effort to a proactive, automated part of the SDLC.
Key takeaways
-
The Performance Bottleneck
4:00
Traditional performance engineering is highly manual and time-consuming (taking minutes just to identify hot paths), making it difficult to scale across large codebases. This bottleneck increases as coding agents write code faster, potentially introducing inefficiencies.
-
AI Agent Workflow for Optimization
12:17
An LLM agent can automate the entire loop: reading profiling data (call stack/CPU time), identifying anti-patterns (like O(N²) loops), locating the code path in the Git repo, proposing a fix, and validating it via canary deployment.
-
Scaling Fixes with Pattern Catalogs
21:20
Instead of fixing one instance, agents can search across multiple services using a centralized 'Pattern/Anti-pattern Catalog' (stored in a Git repo) to find and propose fixes for the same bad pattern repeatedly implemented elsewhere.
-
Shifting Left: Proactive Performance
26:40
The goal is to move performance improvement left in the SDLC. The catalog can be used proactively—at code review or even during code authoring—to suggest optimal patterns before suboptimal code reaches production.
Technical details
-
Profiling Data Structure
320s
Profilers (regardless of language like Java, Python, Go) output similar data: call stack, self CPU, and inclusive CPU time spent on methods. This structured format is suitable for LLM consumption.
-
Anti-Pattern Identification
600s
LLMs can identify common performance anti-patterns (e.g., O(N²) loops, repeated object allocation in loops) purely by analyzing the call stack data, not just reading the source code.
-
Agent Workflow Steps
800s
The agent must: 1) Identify a pattern from profiling data. 2) Search the Git repo for the method definition at the specific production commit. 3) Trace the entire call path. 4) Generate a code review/fix proposal.
-
Validation and Guardrails
1450s
The process requires multiple layers of verification: Unit tests, integration tests (functional correctness), and mandatory canary deployments. The observability report from the canary run provides 'ground truth' (e.g., CPU reduction, latency change) for the agent to make a judgment call.
-
Pattern Catalog Implementation
1300s
The catalog should be a central, ever-growing Git repository containing documented patterns and anti-patterns. Entries should include hints for LLM querying, confirmed services, and confidence levels.
Mentioned resources
- Jeff Dean's Blog Post
- PyTorch torch fix code repo
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.