The Video Signal

Topic

Retrieval-Augmented Generation (RAG)

All digests tagged Retrieval-Augmented Generation (RAG)

· 56:53

Fine-Grained Authorization: The Missing Piece in Agentic AI Security - Shivay Lamba - NDC Sydney

As Generative AI and agentic systems become more integrated into enterprise workflows, securing data access is paramount due to increased attack vectors. This talk emphasizes that traditional Role-Based Access Control (RBAC) is insufficient for modern AI applications. The solution proposed is Fine-Grained Authorization (FGA), specifically utilizing Relationship-Based Access Control (ReBAC). OpenFGA is presented as a robust, scalable tool that allows developers to enforce document-level permissions at runtime, ensuring that LLMs and agents can only access data explicitly authorized by the user's role and context. This approach is critical for securing Retrieval-Augmented Generation (RAG) pipelines against sensitive information disclosure.

Key takeaways

  1. AI Agents Require Guardrails 1:47

    Agentic systems, while powerful, must be constrained by proper security guardrails. Analogies like the Harbor Bridge and designated swimming areas illustrate that actions must be limited to authorized boundaries.

  2. FGA is Necessary for GenAI Security 20:40

    The most critical security risk in LLM applications, from an application developer's perspective, is Sensitive Information Disclosure. FGA ensures the AI respects user roles and permissions when accessing private data.

  3. ReBAC Scales Beyond RBAC 35:30

    Authorization evolved from Access Control Lists (ACL) to Role-Based Access Control (RBAC), which is coarse-grained. Relationship-Based Access Control (ReBAC) is the natural successor, allowing complex access decisions by defining relationships between subjects (users/groups) and objects (documents/folders).

  4. Pre-filtering Secures RAG Pipelines

    When implementing RAG, pre-filtering is the superior security approach. Instead of post-filtering results (where unauthorized documents might still be retrieved), a pre-filter check limits the vector search to only those documents viewable by the user's permissions.

Watch on YouTube Full article

· 49:44

The AI Memory Problem: Why Long Context Isn’t Enough — Dan Biderman, Engram Co-founder & CEO

The talk addresses the limitations of current AI architectures—specifically that simply increasing context window size or relying solely on Retrieval-Augmented Generation (RAG) is insufficient for building truly intelligent, long-horizon agents. Dan Biderman introduces a paradigm shift focusing on 'continual learning' and knowledge compression. Key solutions include using specialized knowledge representations called 'cartridges,' implementing 'test-time training' (or test-time compute), and achieving high token efficiency to enable models to handle the anticipated petabytes of proprietary enterprise data.

Key takeaways

  1. Beyond RAG: The Need for Internalized Knowledge 15:14

    Current methods like RAG are limited because they only provide external, textual context. True intelligence requires embedding knowledge into the model's parameters (weights) to achieve 'intuition,' allowing the model to generalize and extrapolate beyond explicit notes or recipes.

  2. The Problem of Scale: Context Rot and Token Limits 23:30

    As companies accumulate trillions of tokens of proprietary data, simple context management fails due to 'context rot' (the model becoming less accurate the more context it reads) and extreme token consumption. This necessitates methods that are both highly efficient and scalable.

  3. The Solution Stack: Cartridges, Training, and Memory 30:05

    Engram proposes a multi-faceted approach combining knowledge compression via 'cartridges' (compact capsules of knowledge), gradient-based updates during inference ('test-time training'), and advanced memory layers to achieve superior token efficiency and model accuracy.

  4. The Future: Autonomous, Personalized AI

    The ultimate goal is a system where the model autonomously determines what knowledge should be internalized (in weights) versus what should remain external (in text/RAG), creating personalized models that improve continuously with user interaction, similar to nurturing a Tamagotchi.

Watch on YouTube Full article