Middleware for Managed Deep Agents
Middleware is a mechanism for extending the lifecycle of Managed Deep Agents, allowing developers to implement custom behaviors such as policy enforcement, fault tolerance, and rate limiting. The demonstration covers two primary use cases: using prebuilt middleware (like `PIIMiddleware`) to redact sensitive data before it reaches the LLM, and building custom middleware from scratch using decorators like `wrap_tool_call` for logging and auditing tool usage.
Key takeaways
-
Middleware Functionality
Middleware extends the agent's lifecycle to manage behaviors like policy enforcement, fault tolerance, and rate limits when interacting with tools or the LLM.
-
PII Redaction Demo
0:01
Using prebuilt `PIIMiddleware` automatically detects and redacts sensitive information (e.g., customer emails) from the input, preventing the data from reaching the LLM or being stored in LangSmith.
-
Custom Middleware Development
0:03
Custom middleware can be built using decorators (e.g., `wrap_tool_call`) and hooks to intercept and log events, such as every tool call, at specific points in the agent's process.