Retrieval Augmented Generation (RAG)

Sep 17, 2026 · 42:28

Seattle Spark Hack Winners Livestream Spotlight: LiveKit & Memo

This summary covers the NVIDIA DGX Spark Hack winners spotlight, featuring two local AI applications: LifeKit, an offline survival companion, and Memo, a local-first wearable assistant. Both projects demonstrate the capability of running complex, multi-modal AI workflows—including vision, speech, and retrieval—on edge devices like the Acer Veriton GN100. The core technical focus is on achieving reliable, low-latency AI functionality without constant internet connectivity, utilizing advanced NVIDIA models like Neotron and Cosmos.

Key takeaways

Offline AI Capability (LifeKit) 20:00
LifeKit addresses the critical need for stable information access when internet connectivity fails. It uses local AI and Retrieval Augmented Generation (RAG) against extensive, downloadable knowledge bases (e.g., Wikipedia) to provide guidance on survival topics (first aid, navigation) without relying on external servers.
Selective Memory and Privacy (Memo) 27:30
Memo is a wearable assistant designed for elderly care, focusing on privacy by only tracking objects explicitly chosen by the user (e.g., keys, wallet). It uses visual evidence and confirmation to save location data, ensuring that the system does not record or store general video feeds.
Concurrent Model Orchestration 22:30
Both projects rely on running multiple NVIDIA models concurrently (e.g., Neotron, Cosmos, Parakeet, VSSs, NIM, VLM) on the GN100. The primary architectural challenge is orchestrating these disparate tools—from speech transcription to reasoning—into a single, reliable, and deterministic application.

Watch on YouTube Full article

Sep 6, 2026 · 8:49

AI Simplified: 6 Concepts You Need to Know About Modern AI

The video simplifies modern AI by detailing six essential concepts: Large Language Models (LLMs), Model Training/Tuning, Retrieval Augmented Generation (RAG), AI Agents, Model Context Protocol (MCP), and System Prompts. The explanation uses human analogies to show how these components—from the core 'brain' (LLM) to the 'central nervous system' (MCP)—are integrated to create sophisticated systems capable of reasoning, accessing external knowledge, and taking autonomous actions.

Key takeaways

Artificial Intelligence (AI)
A subfield of computer science focused on matching or exceeding human intelligence in a machine. The core function is generating content using probabilities to predict output based on input, described as 'autocomplete on steroids.'
Large Language Models (LLMs)
The foundational component or 'brain' of the AI system where core intelligence and reasoning capabilities reside. It is responsible for generative AI outputs (words, images, sounds).
RAG (Retrieval Augmented Generation) 3:35
A method to extend the LLM's knowledge base by integrating trusted external sources (e.g., research papers or product documentation). This process helps reduce 'hallucinations'—confident errors made by the AI.
AI Agents 5:20
An advanced system where a model autonomously uses external tools (e.g., writing code, searching the web, reading/writing databases) to achieve specific goals, giving the AI 'hands and feet.'
Model Context Protocol (MCP) 6:20
The orchestration layer that acts as the central nervous system for an AI agent. MCP connects the model's reasoning to the external tools, coordinating actions.
System Prompts 7:30
A set of guiding principles or constraints given to the model that dictates its behavior and ethical boundaries. This prevents misuse (like prompt injections) without requiring constant, expensive retraining.

Watch on YouTube Full article

Sep 3, 2026 · 50:35

How To Build And Evaluate Search Agents

This talk details three critical areas for building and evaluating advanced search agents: evaluation benchmarks, synthetic data generation, and trajectory analysis. The speaker introduces ORBIT, a novel pipeline for creating complex, multi-hop questions without paid APIs or labeled data. For evaluation, the presentation highlights BrowseComp-Plus as a reproducible benchmark that converts QA tasks into open retrieval problems. Key findings emphasize that both the retriever model and the LLM contribute significantly to accuracy, but efficiency (fewer search calls/turns) is crucial for practical deployment. Finally, trajectory analysis tools like Hawkeye are presented as essential for diagnosing agent failures by visualizing query provenance, topic shifts, and repetition.

Key takeaways

Search Agents vs. RAG Architecture 5:45
Agentic search differs from naive Retrieval-Augmented Generation (RAG) because the LLM acts as the main driver, iteratively interacting with a retrieval tool to refine queries before generating a final answer. This iterative process is key.
ORBIT for Synthetic Data 17:25
The ORBIT pipeline generates complex, multi-hop questions by describing an entity's properties without naming it (inverting the question). It verifies every question by requiring the agent to confirm claims against source documents.
BrowseComp-Plus Reproducibility 22:20
To improve reproducibility, BrowseComp-Plus converts QA tasks into open retrieval problems. This involves providing a web corpus (including hard negatives) and human relevance judgments for each query/answer pair.
Efficiency vs. Accuracy in Search Agents 29:20
While high Exact Match (EM) accuracy is good, efficiency—measured by the number of search calls or turns—is equally important. A model with fewer search calls but comparable accuracy is often preferred.
Trajectory Analysis Importance 41:00
Relying solely on EM accuracy is insufficient. Analyzing agent trajectories helps identify issues like query repetition, topic shifts, and inefficient looping (e.g., a model taking many turns without reaching the answer).

Watch on YouTube Full article

Aug 17, 2026 · 1:03:26

Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI

This talk provides a deep dive into context engineering for large language model (LLM) agents, focusing on optimizing performance and cost in real-world applications like an AI tutor. The core finding is that compaction techniques (like summarization) are often detrimental because they invalidate the prompt cache, forcing the system to pay full price for every token. For optimal recall and cost efficiency, retaining the full history—especially when leveraging advanced caching mechanisms—is superior to aggressive context reduction.

Key takeaways

Compaction is a potential trap due to Prompt Caching 22:06
When using prompt caching (which can make cached tokens up to 50 times cheaper), any transformation or summarization of the context invalidates the cache, forcing full-price token usage. Compaction must shrink the context by more than 50 times just to pay for itself.
Full History Retention Wins on Recall 33:33
Experiments showed that leaving the full history untouched (the 'full history' preset) provided the best memory recall and overall performance, even though it was the most expensive option. Aggressive compaction techniques significantly degraded factual retrieval.
Hybrid Search is Superior for Retrieval 22:48
For knowledge base browsing, a hybrid search combining semantic similarity (embedding model) with keyword search (BM25) is necessary to achieve high recall. Pure dense retrieval failed when facts were buried at large context sizes (e.g., 400k tokens), while BM25 maintained 100% accuracy.

Watch on YouTube Full article

Aug 13, 2026 · 13:21

AI & Data Science Periodic Tables: How They Work Together

The video details the synergistic relationship between Data Science and Artificial Intelligence (AI), presenting both disciplines using 'Periodic Tables' as a conceptual framework. It emphasizes that modern AI applications are built upon robust data science foundations. A comprehensive example—Document Q&A—is used to illustrate a full pipeline, detailing how elements like Extract Transform Load (ET), Data Ingest (DI), and Data Cleansing (CD) prepare the data, which is then processed by AI components such as Embeddings (EM), Retrieval Augmented Generation (RAG), and Guardrails (GR). The process can be completed into a continuous loop using Drift Detection (DR) and Synthetic Data generation for continuous system improvement.

Key takeaways

AI relies on foundational data science work 0:25
The speaker notes that all advancements in AI sit atop the groundwork laid by data science, creating a feedback loop where models inform how data is prepared for future use. (0:15-0:30)
Data Science Pipeline Stages 1:38
The Data Science periodic table defines five groups across the top (Acquisition, Preparation, Modeling, Generation, Evaluation) and tracks data maturity through rows: Raw Data $\rightarrow$ Prepared Data $\rightarrow$ Model Data $\rightarrow$ Validated Insight. (1:30-2:25)
AI Pipeline Core Elements 2:40
The AI periodic table features groups like Retrieval and Orchestration, with core primitives including Prompt, Embed, and LLM. Key components include embeddings (encoding info into numbers) and RAG (coordinating retrieval). (2:35-3:40)
The Full Document Q&A Pipeline 3:30
Building a system requires combining elements from both tables. The process moves linearly through data preparation (ET $\rightarrow$ DI $\rightarrow$ CD $\rightarrow$ ST $\rightarrow$ EN $\rightarrow$ GO) and then AI processing (EM $\rightarrow$ Vx $\rightarrow$ RAG $\rightarrow$ PR $\rightarrow$ LG $\rightarrow$ GR). (4:30-6:20)
Closing the Loop for Continuous Improvement 6:20
To prevent the system from being a one-way street, the pipeline is closed using Data Drift (DR) detection and Synthetic Data generation. This allows the embedding model to fine-tune itself continuously based on failing patterns. (6:40-7:50)

Watch on YouTube Full article

Aug 12, 2026 · 13:04

Memory Harnesses for Long-Running Research Agents — Stefania Druga, Sakana.ai

The presentation details the design and efficacy of 'memory harnesses' for managing state in long-running research agents. The core finding is that while memory harnesses offer no benefit when task context fits within the model's window (adding only cost), they are critical for solving long-horizon tasks where relevant information sits far outside the current context. The speaker emphasizes treating memory as a write-manage-read control loop, not merely an attached database.

Key takeaways

Memory is a Control Loop, Not a Database 5:44
The memory harness must be viewed as a 'write-manage-read' control loop wrapped around the model, rather than simply a retrieval database. This architectural focus allows for precise state management.
Local Models Enable Sovereignty and Cost Control 2:36
Running evaluations on local hardware (e.g., M3 Ultra) enables better control over the entire pipeline, which is crucial for maintaining 'sovereign AI' capabilities and reducing operational costs.
Ranked Recall Outperforms Other Policies 7:16
Across the X-Bench benchmark (over 68 questions), the 'ranked recall' policy consistently achieved the best performance, outperforming even approaches that simply gate memory usage or use an ideal 'oracle' ground truth.

Watch on YouTube Full article

Aug 9, 2026 · 7:00

What Is Chunkless RAG? How Docling & AI Agents Navigate Documents

The video contrasts traditional Retrieval Augmented Generation (RAG), which relies on chunking documents and similarity search, with a novel approach called Chunkless RAG. Traditional methods discard crucial document structure (headings, tables) by flattening the content into small text chunks. Chunkless RAG proposes that AI agents navigate the inherent tree structure of a document—retaining context and allowing for complex reasoning across sections—rather than relying solely on vector similarity matching. This requires specialized tools like Docling to reconstruct the hierarchical structure from formats like PDFs.

Key takeaways

Limitations of Traditional RAG
Standard RAG chunks documents (e.g., every 500 words) and uses similarity search on these small text blobs. This process discards the original document structure, making it difficult for the model to understand relationships between separated sections or tables.
Concept of Chunkless RAG 2:00
Chunkless RAG utilizes AI agents that navigate the document's inherent tree structure (sections, subsections) rather than matching by similarity. This allows for answering questions that span multiple, disconnected parts of a long document.
Role of Docling 4:10
Since PDFs often bury the author's hierarchy, specialized tools like Docling are necessary to process a PDF and output a structured 'Docling document,' which preserves sections, headings, reading order, and table integrity.
Trade-offs of Structure-Aware Retrieval 5:50
While structure-aware retrieval provides superior precision on long, organized documents, it is more complex than chunking. It involves multiple passes and increased model back-and-forth (latency), making the choice dependent on whether fuzzy search or structural precision is needed.

Watch on YouTube Full article

Jul 30, 2026 · 1:05:19

From complex docs to decision ready context for agentic financial workflows

The webinar details how LlamaIndex enables the creation of sophisticated document agents for financial services by transforming complex, unstructured data (PDFs, images, etc.) into accurate, decision-ready context. The platform provides a comprehensive workflow—including parsing, extraction, and indexing—to automate historically manual processes in areas like Private Equity deal analysis, corporate lending, and mortgage servicing. Key features include advanced layout analysis for tables/charts, cross-document reconciliation, and robust human-in-the-loop validation.

Key takeaways

Context is the foundation of agentic workflows 3:55
Successful AI agents are highly dependent on accurate context. The platform's mission is to make previously untapped, unstructured data accessible by providing high accuracy and low cost in document processing.
Three core financial use cases demonstrated 10:30
The solution covers Private Equity (LBO modeling from deal room dumps), Corporate Lending (processing credit packets like ABL agreements and financials), and Mortgage Servicing (analyzing loan packages including deeds of trust, appraisals, and disclosures).
Workflow relies on multi-stage processing 11:45
The process moves from document arrival through intelligent classification/splitting $\rightarrow$ optimal parsing (LlamaParse) $\rightarrow$ targeted extraction (LlamaExtract) $\rightarrow$ cross-validation and grounding, culminating in a structured output for downstream modeling.

Watch on YouTube Full article

Jul 19, 2026 · 7:46

Why RAG Solutions Fail with Complex Documents & Vector Databases

Standard Retrieval Augmented Generation (RAG) solutions often fail when processing complex, ambiguous, or contradictory real-world documents (such as evolving laws or policies). The video outlines practical architectural improvements—including robust document management and clarification loops—to ensure that AI systems can accurately handle data ambiguity and avoid presenting single answers where multiple valid interpretations exist.

Key takeaways

RAG Failure Point: Data Contradiction 2:33
Because real-world document sets are compiled over time by multiple people, they frequently contain contradictions (e.g., a 2012 law contradicting a 1912 law). A standard RAG solution must be designed to handle the possibility of multiple correct answers rather than assuming singularity.
Solution 1: Preventing Unforced Errors 3:55
Implement strong document management processes to prevent 'unforced errors' in the vector database. This means ensuring that outdated or superseded policies are removed, preventing confusion when a newer policy replaces an older one.
Solution 2: Implementing Clarification Loops 4:45
A clarification loop is a mechanism built into the AI solution that prompts the user to rephrase or specify their question if it is too vague (e.g., asking 'Who won the championship in 2010?' without specifying the sport). This ensures the input question is specific enough for accurate retrieval.

Watch on YouTube Full article

Jul 6, 2026 · 49:59

Let's build an AI agent - Phil Nash - NDC Copenhagen 2026

This talk demystifies AI agents by building one from scratch, demonstrating how Large Language Models (LLMs), tools, and memory work together in a continuous loop to achieve goals. The core mechanism involves an agent runtime that orchestrates function calls—allowing the LLM to interact with external systems like file systems or calculators. Advanced concepts covered include the Model Context Protocol (MCP) for standardized tool interaction and 'Skills' for progressive disclosure of capabilities, enabling agents to perform complex tasks like self-refactoring.

Key takeaways

Agent Architecture 17:03
An agent fundamentally runs tools in a loop to achieve a goal. This process requires an LLM, external tools (functions), and an orchestration layer (the 'harness') that manages the interaction.
The Agent Loop 28:10
The core agent functionality is implemented in a loop: The model generates function calls $\rightarrow$ The harness executes those functions (awaiting results) $ ightarrow$ The results are fed back to the model for the next step, continuing until the goal is met.
Standardization via MCP 36:00
The Model Context Protocol (MCP) provides a standardized way for agents to interact with services. It separates concerns into Server components (tools, resources, prompts) and Client components.
Progressive Disclosure with Skills 40:05
Skills allow for progressive disclosure of capabilities. Instead of loading all tool declarations at once, the agent only loads a skill's header initially and can request more details (resources, scripts) as needed.

Watch on YouTube Full article

Jun 30, 2026 · 1:09:03

Building retrieval harness for enterprise agents

This technical walkthrough details the architecture of an enterprise agentic retrieval harness, moving beyond traditional Retrieval Augmented Generation (RAG) systems. The core argument is that effective data access requires giving agents a comprehensive set of tools—including local file system traversal (GP), semantic search, keyword search, and multimodal analysis—rather than relying on a single indexing method. Key focus areas include managing complex document parsing, ensuring multi-tenancy via custom metadata, and optimizing vector storage for scale and cost.

Key takeaways

Harness over RAG 2:00
The most capable approach is not to select a single retrieval method (e.g., pure semantic search or pure GP access) but to build an agentic harness that allows the agent to decide how to combine multiple tools for data exploration and verification.
The Role of File Primitives 4:20
For document-driven workflows, critical primitives include: (1) Retrieval (hybrid search), (2) Directory listing/hierarchy traversal, (3) Grep for exact text matching, (4) Chunk-by-chunk analysis, and (5) Multimodal processing via localized screenshots.
Enterprise Scaling Challenges 6:45
Scaling to multi-tenant or million-file situations requires specialized indexing pipelines that manage data freshness, security, and permissioning. Using custom metadata attached during the parsing process is crucial for filtering access at the vector storage layer.

Watch on YouTube Full article