# 300 Agents, 60 People: How We Secured Our Own AI Agents - Aiden Sochowski, Runlayer

## Executive summary

Founding engineer Aiden Sochowski details Runlayer's internal process for securing and monitoring over 300 AI agents used by a 60-person company. The core strategy involves deploying a central gateway to log every session and tool call, identifying risks like prompt injection, data exfiltration, and over-permissioned bots. Beyond security, the talk highlights the need for monitoring poorly written MCP servers and controlling runaway costs caused by model switches.

## Key takeaways

- Centralized Visibility is Critical: Many companies lack visibility into their agents. Runlayer solved this by deploying a gateway to log every session and tool call, allowing them to map agents by topic and risk.
- Agent Risk Assessment: Agents can accelerate risks by mixing different classes of data. It is crucial to scope agents down and define policies for both inbound and outbound data flows.
- Advanced Security Measures: To prevent attacks (like data exfiltration or PR approval hijacking), implementing classifier models to scrutinize every tool call and ensure alignment with the agent's defined intent is necessary. This is complex and context-dependent.
- Cost and Reliability Monitoring: Monitoring is needed for poorly written MCP servers (which can lead to token cost blowups from formatting errors) and for tracking model switches (e.g., switching from GPT-4 to Claude 3.5, causing costs to balloon over 100x).

## Technical details

- Agent Architecture & Monitoring: The solution involves deploying a gateway to log all agent sessions and tool calls centrally. This allows for visualization of the agent's activity, including which MCP servers are touched and what actions are taken.
- Data Exfiltration Prevention: Security policies can be defined to block PII from leaving externally (e.g., using regex on customer names in emails) or to block access to specific data sources (like private calendars).
- Agent Scope and Intent Drift: To prevent a single agent from contaminating or harming others, the only reliable method found is to tightly scope how agents can write to systems, ideally limiting writes to specific databases within a SaaS app.
- Model Cost Control: Cost blowups can occur when models are switched (e.g., from GPT-4 to Claude 3.5) or when poorly written MCP servers repeatedly encounter formatting errors, leading to excessive token usage.

## Practical implications

- Implement a centralized gateway to monitor all AI agent activity (sessions, tool calls) for comprehensive visibility.
- Use classifier models to validate that an agent's actions remain aligned with its defined intent throughout a session.
- Apply granular policies to restrict data egress (PII) and limit write permissions to specific, non-overlapping data stores.
- Monitor and rate-limit agent interactions with MCP servers to prevent runaway token costs due to formatting errors.

## Topics

AI Security, Agent Orchestration, Data Governance, Cost Management, MCP Servers, Aiden Sochowski, AIE CODE summit

Source: https://www.youtube.com/watch?v=BgLdvF7Qkh8