# Fine-Grained Authorization: The Missing Piece in Agentic AI Security - Shivay Lamba - NDC Sydney

## Executive summary

As Generative AI and agentic systems become more integrated into enterprise workflows, securing data access is paramount due to increased attack vectors. This talk emphasizes that traditional Role-Based Access Control (RBAC) is insufficient for modern AI applications. The solution proposed is Fine-Grained Authorization (FGA), specifically utilizing Relationship-Based Access Control (ReBAC). OpenFGA is presented as a robust, scalable tool that allows developers to enforce document-level permissions at runtime, ensuring that LLMs and agents can only access data explicitly authorized by the user's role and context. This approach is critical for securing Retrieval-Augmented Generation (RAG) pipelines against sensitive information disclosure.

## Key takeaways

- AI Agents Require Guardrails: Agentic systems, while powerful, must be constrained by proper security guardrails. Analogies like the Harbor Bridge and designated swimming areas illustrate that actions must be limited to authorized boundaries.
- FGA is Necessary for GenAI Security: The most critical security risk in LLM applications, from an application developer's perspective, is Sensitive Information Disclosure. FGA ensures the AI respects user roles and permissions when accessing private data.
- ReBAC Scales Beyond RBAC: Authorization evolved from Access Control Lists (ACL) to Role-Based Access Control (RBAC), which is coarse-grained. Relationship-Based Access Control (ReBAC) is the natural successor, allowing complex access decisions by defining relationships between subjects (users/groups) and objects (documents/folders).
- Pre-filtering Secures RAG Pipelines: When implementing RAG, pre-filtering is the superior security approach. Instead of post-filtering results (where unauthorized documents might still be retrieved), a pre-filter check limits the vector search to only those documents viewable by the user's permissions.

## Technical details

- Authorization Model Evolution: The progression of access control models is: Access Control Lists (ACL) $\rightarrow$ Role-Based Access Control (RBAC) $\rightarrow$ Attribute-Based Access Control (ABAC) $\rightarrow$ Relationship-Based Access Control (ReBAC). ReBAC allows for complex queries like 'Is Kate in the product department AND does the product department have viewer permission to the roadmap document?'
- OpenFGA Implementation: The open-source project OpenFGA, inspired by Google's Zanzibar paper, implements ReBAC. It allows defining an authorization model (e.g., User $\rightarrow$ Folder $\rightarrow$ Document) and relationship triples to automatically infer complex access decisions at runtime.
- Agent Task-Based Authorization: For agentic use cases, authorization must be task-based. OpenFGA can restrict what an agent can do by defining specific permissions for tool calls (e.g., a user might have permission to *view* Slack channels but not *send messages*). This requires explicit checks before the tool is executed.
- RAG Security Pipelines: In RAG, security can be enforced via two methods: Post-filtering (OpenFGA filters results after vector search) or Pre-filtering (applying the permission check as a metadata filter *before* the vector search). Pre-filtering is recommended for better security.

## Practical implications

- When designing an application that uses AI agents or RAG, always prioritize implementing FGA using a ReBAC model to enforce document-level permissions.
- For data retrieval systems (RAG), implement pre-filtering checks in your vector database query pipeline rather than relying solely on post-filtering for maximum security.
- When building agentic workflows, define granular task-based authorization boundaries. An agent should only have the minimum necessary permissions to perform a specific action.

## Topics

Fine-Grained Authorization (FGA), Relationship-Based Access Control (ReBAC), Retrieval-Augmented Generation (RAG), Agentic AI Security, OpenFGA, Access Control Models (RBAC, ABAC), Google Zanzibar Paper

Source: https://www.youtube.com/watch?v=OK5xPUuco6o