# NVIDIA, Docker & Hud on Agents in Production

## Executive summary

The discussion explores the operational challenges of deploying AI agents in a production environment (24/7 operation). Key insights emphasize that successful agent deployment requires shifting focus from root cause analysis to comprehensive context and observability. Speakers covered topics including using agents with combined data sources (Elastic logs + ServiceNow), redesigning automated fixes for human consumption, optimizing GPU utilization during tool calls, and leveraging AI-built tracing frameworks for debugging rare bugs.

## Key takeaways

- Context over Root Cause Analysis: When agents are running 24/7 in production, the most critical resource is context—understanding what changed yesterday and the relationships between services. This proactive data knowledge is more valuable than traditional root cause analysis.
- Automated Fixes Must Convince Humans: Simply automating investigations and opening pull requests (PRs) for high-impact fixes is insufficient, as developers often ignore them. The output must be rebuilt to convince the human developer of its value and priority.
- GPU Idle Time During Tool Calls: A counterintuitive finding is that when an agent makes a tool call, the GPU sits idle. Properly accounting for this CPU-intensive period allows users to serve roughly twice as many users compared to benchmark predictions that ignore tool calls.
- AI-Built Tracing Frameworks: For debugging rare bugs, the most useful investment is getting AI to build a tracing framework. Providing traces from an overnight run allows the agent to pinpoint the exact problem rather than guessing or failing to reproduce the issue.

## Technical details

- Agent Observability Stack: Agents are highly effective for diagnosing production issues when given combined data sources, specifically Elastic logs, ServiceNow incident reports, and the relevant code. This combination can solve issues in seconds that might take developers hours or days.
- High-Impact/Low-Risk Changes: When automating fixes, focus on identifying high-impact, low-risk changes (e.g., those not requiring migrations) and prioritizing them by business impact or risk rather than technical optimization.
- Concurrency Modeling: When modeling workload concurrency, ignoring the time spent on tool calls (which run on the CPU) leads to inaccurate predictions. The total capacity must account for this CPU work alongside GPU processing.

## Practical implications

- Build teams must treat AI agents as operational tools that require continuous context and observability data (e.g., logs, incident reports) to function effectively in production.
- Automating CI/CD processes using agents requires a human-centric output strategy; fixes must be presented with clear business impact and risk assessment to ensure adoption.
- Performance testing for AI agent workloads must model the CPU time consumed by tool calls, not just GPU utilization.

## Topics

AI Agents, DevOps, Production Reliability, Observability, Debugging Tools, LLMs, AI DevCon NYC 2026

Source: https://www.youtube.com/watch?v=P03T9DXBxYw