# From Signal to PR: Anatomy of a Self-Improving Agent — Jason Lopatecki, Arize

## Executive summary

The future of observability is shifting from human-driven dashboards to machine-readable telemetry that powers autonomous AI agents. Arize's Signal automates the debugging process by pulling deep production traces and logs (sometimes ten megabytes) directly into the repository as files. This allows coding harnesses, like Claude Code, to understand the exact code path taken during an error, enabling agents to propose fixes and creating a continuous loop where systems can autonomously improve themselves. The human role is evolving from responder to reviewer.

## Key takeaways

- Observability Shift (2.0): Observability is moving beyond UI clicks and graphs; it's becoming a 'smoke'—telemetry data that agents can read to debug software, allowing for continuous automated fixing.
- The Key Unlock: Traces on Filesystem: The critical breakthrough is pulling relevant production traces and logs down as files into the repo. Coding agents are highly effective with file formats, giving them the exact code path rather than guessing among millions of branches.
- Autonomous Fixing Loop: The goal is to build systems that autonomously fix themselves. The process involves an agent investigating first, gathering deep evidence (traces/logs), and proposing a fix before human intervention.
- Security and Deployment: To ensure compliance for large enterprises (e.g., Uber, Booking), agents must run within the customer's Virtual Private Cloud (VPC) using sandboxes, preventing production systems from connecting directly to external models.

## Technical details

- Agent Architecture & Skills: The system relies on 'skills' that interface with observability platforms. These skills gather necessary context (traces, logs, repo code) and combine them to provide a complete picture for the agent to generate a fix.
- Data Input Format: The most effective data input is pulling production traces/logs into the repository as files. This file format allows coding agents to process large amounts of context (e.g., 10MB files) and understand the precise execution path.
- Execution Environment: Agents must run in secure sandboxes, which can be deployed within a customer's VPC. This architecture allows for automated debugging without exposing sensitive production systems to external cloud models.
- Evaluation Layers (Evals): Evals act as an AI layer or 'judge' that runs periodically on production traces. They add pre-processed information to the raw data, allowing agents to assess specific failure patterns (e.g., prompt injection) at scale.

## Practical implications

- Build engineers should prioritize logging and tracing ten times more data than previously considered, as this deep telemetry is the fuel for autonomous agents.
- Design CI/CD pipelines to support continuous feedback loops where observability platforms feed directly into agent-driven fix proposals.
- When designing systems that interact with AI agents, ensure that necessary context (logs, traces) can be extracted and delivered in a file format suitable for coding harnesses.

## Topics

Observability, AI Agents, Distributed Tracing, Autonomous Systems, Software Reliability Engineering (SRE), Signal, Arize, Claude Code, Pyroscope

Source: https://www.youtube.com/watch?v=9HbzAWnKbo4