May Walter - From Blind Spots to Merged PRs: Runtime Intelligence for Continuous Agentic Performance
Summary
The talk details implementing a runtime intelligence layer for coding agents to enable continuous performance optimization in mature codebases. The approach moves beyond reactive bug fixing by running sensors in production to capture deep forensic context (e.g., function execution frequency, failure modes). This allows agents to proactively surface high-ROI fixes—such as N+1 queries or missing database indexes—scored by complexity and impact, enabling tech leads to prioritize improvements before sprint planning.
Key takeaways
-
Automating the Investigation Phase
17:52
Instead of waiting for performance degradation to become a crisis (a 'leaky bucket' problem), the goal is to automate the investigation phase. This allows teams to identify potential optimizations and their estimated impact/cost without dedicating unpredictable engineering time.
-
Prioritizing Impact over Task Completion
25:30
The focus shifts from merely generating pull requests (PRs) to identifying the highest impact, lowest risk changes. The system must provide human-readable context that justifies attention, preventing 'PR fatigue' from agents.
-
Defining Business Context
22:00
Effective agentic workflows require more than just code analysis; they need business context. The system must map production activity (endpoints, event consumers) to function-level context to understand the true impact of a potential fix.
Technical details
-
Runtime Intelligence Layer
420s
A sensor runs with the application in production, capturing metrics like function execution frequency, duration, and failure context. This provides deep forensic context to answer 'why is this slow or why is this failing?'
-
Agentic Workflow Architecture
720s
The solution was designed to be vendor-neutral (compute and harness) while maintaining security. It utilizes triggers like webhooks or scheduled runs (e.g., weekly via GitHub Actions Workflows). The stack used in the case study included Claude, GitHub, and Slack.
-
Data Analysis & Scoring
1240s
The system analyzes production context to find anti-patterns and opportunities. It scores these findings based on impact and risk, specifically targeting issues like N+1 queries or missing database indexes.
-
Contextualization (Prod-to-Code)
1320s
The 'prod to code' mapping connects service/endpoint level metrics (e.g., P90 latency) down to specific functions and methods, allowing agents to reason over the connection between production behavior and local code structure.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.
