# AAIF Reading Group: Measuring Agent Policy Compliance

## Executive summary

This reading group explores the challenge of measuring policy compliance in AI agents, addressing the critical issue that agents frequently fail to follow complex, non-gated instructions (SOPs) reliably. The discussion centers on the paper "Measuring Agent Policy Compliance" (arXiv:2607.25398), which utilizes a rigorous, code-evaluated benchmark. Key findings highlight that even advanced models struggle with compliance, often failing due to reasoning errors or environmental overrides. Solutions discussed include implementing external verification mechanisms like deterministic and model-based guards, and final state graders, moving policy enforcement outside of the LLM's internal reasoning.

## Key takeaways

- Low Baseline Compliance Rate: The benchmark showed that the highest level of success for an LLM following non-gated instructions was 36.2% (at the time of the paper's testing), indicating significant difficulty in achieving reliable policy adherence, even with extensive context.
- Four Recurring Failure Patterns: Agent failures were categorized into four patterns: 1) The check runs, but the result is ignored; 2) The environment overrides a policy rule; 3) The model skips a mandatory check; and 4) The agent self-reports success even when rules were violated.
- Need for External Verification: The authors conclude that the handbook does not behave as a persistent authority. To build reliable agents, the focus must shift to external, deterministic checks (guards) placed before critical actions, rather than relying on the agent's self-report.

## Technical details

- Benchmark Methodology: The benchmark uses domain-specific SOP documents (e.g., HR, Finance) that are mutated for every task to prevent memorization. Tasks are run in a sandbox environment with access to tools (e.g., Gmail, calendar) and are evaluated by Python code, ensuring deterministic grading.
- Guardrails for Compliance: Two families of guards are proposed: 1) **Deterministic Guards:** Code that runs before a tool call to check for simple, numeric policies (e.g., 'no more than $5,000'). 2) **Model-based Guards:** An LLM call that reads the conversation to check for complex, interpretive policies (e.g., 'does this email constitute an involuntary offboarding?').
- Evaluation Flow: The evaluation process involves loading the task, spinning up a container, running the agent, taking a state snapshot, and verifying the state change or output using code, ensuring 'no LLM judge anywhere.'

## Practical implications

- Building reliable enterprise agents requires implementing external, non-LLM-dependent verification layers (guards) to enforce policies, rather than trusting the agent's internal reasoning.
- The complexity of real-world workflows (e.g., HR vs. Finance) suggests that benchmarks must account for domain-specific rule variations and context size.
- For build engineers, this emphasizes the need to integrate compliance checks (like deterministic guards) into the CI/CD pipeline for agentic workflows to ensure policy adherence before deployment.

## Topics

AI Agents, Policy Compliance, LLM Evaluation, Guardrails, MLOps, System Reliability, Measuring Agent Policy Compliance, AAIF (Agentic AI Foundation)

Source: https://www.youtube.com/watch?v=HglV1_EF7tA