RAG — The Video Signal

Sep 23, 2026 · 21:04

Building the Document Context Layer for AI Agents — Jerry Liu, LlamaIndex

The talk outlines the evolution of Retrieval-Augmented Generation (RAG) toward 2026, positing that RAG will decompose into an agent harness and a dedicated context layer. The primary challenge remains unlocking the vast, unstructured context locked in enterprise documents (PDFs, Word, etc.). The solution requires a three-layered platform: 1) Parsing into token-efficient markdown; 2) Semantic storage/document management; and 3) Repeatable, specialized workflows (e.g., KYC, invoice processing). The speaker emphasizes that document OCR is a complex problem due to how PDFs are rendered for display, not machine consumption, necessitating hybrid parsing approaches that combine deep binary understanding with advanced Vision-Language Models (VLMs).

Key takeaways

RAG in 2026: Agent Harness + Context Layer
Naive RAG (chunking, embedding, top-k retrieval) is outdated. Modern agents integrate retrieval complexity into the agent layer, allowing them to reason about the optimal search term rather than relying on fixed top-k retrieval methods. (0:00 - 1:33)
Context Moves Up the Stack 5:11
The focus is shifting from managing context window overflow to hooking up specialized Micro-Capabilities Platforms (MCP) and skills. Furthermore, defining programs and tasks is moving from code (Python, TypeScript) toward natural English language definitions. (3:11)
Document Parsing Requires Hybrid Approaches 8:35
Because PDFs are designed for printing (using glyphs and line segments) and not machine reading, document OCR is difficult. The optimal approach combines pipeline-based understanding of file binaries (e.g., Word's custom XML) with VLM-based visual analysis to achieve high accuracy and low cost. (5:15)
The Three-Layer Document Platform 8:35
A comprehensive document context layer requires three components: 1) Parsing into token-efficient markdown/metadata; 2) Semantic storage (document management for agents); and 3) Repeatable, specialized workflows (e.g., claims, invoices). (5:15)

Watch on YouTube Full article

Sep 9, 2026 · 14:09

Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked

AI agents require more than just access to information; they need a 'context engine' to achieve true understanding of a company's operations, business logic, and historical decisions. The talk details how agents fail when they lack deep, organizational context, leading to wasted tokens and time. The solution involves building a context engine that ingests data from disparate sources (e.g., ticketing systems, code repos) and provides structured, conflict-resolved context to the model in a token-optimized manner.

Key takeaways

The Context Gap
AI-generated code must feel like it was written by a long-term team member, which requires deep operational context that current agents lack. This gap compounds as agents scale from simple tab completion to background processes.
Failure Modes of Context 7:00
Common approaches like the 'curated context trap' (where documentation rots) and the 'MCP plateau' (where agents suffer from satisfaction of search bias) fail because access to information is not the same as understanding it.
The Role of the Context Engine 9:12
A proper context engine must perform conflict resolution (e.g., resolving conflicting architecture diagrams vs. Slack messages), respect permissions (OAuth/SSO), and deliver context in a token-optimized way.
Efficiency Gains 13:00
Implementing a context engine can lead to significant efficiency gains, demonstrated by a reduction from 21 million to 10.8 million tokens and a two-hour reduction in wall-clock time for the same task.

Watch on YouTube Full article

Aug 20, 2026 · 16:39

Your Fine-Tuned Model Is Tech Debt: A 50x ROI House of Cards — Dan Bjornn, Lease End

The video argues that relying heavily on fine-tuned LLMs for production applications can create 'calcification tax'—a form of technical debt characterized by rigid architecture and complex maintenance cycles. The speaker details migrating from a fine-tuning approach to an agentic framework built on skills, prompts, and context. This shift drastically reduced the time required to deploy fixes (from weeks/days to under an hour), improved accuracy, maintained model agnosticism, and ultimately lowered total operational costs.

Key takeaways

The Calcification Tax 10:04
Fine-tuning creates technical debt by locking systems into specific models and architectures. The complexity of retraining—which involves gathering examples, synthesizing data, manual validation, and iterative fixing—is costly and slow (up to a week per cycle).
Agentic Frameworks Over Fine-Tuning
The rebuild swapped the fine-tuned model for skills, prompts, and context on a model agnostic framework. This allowed fixes to ship by simply uploading files to an S3 bucket.
Cost vs. Accuracy Tradeoff
While the new agentic approach increased cost per message (due to using better models), overall total cost decreased significantly because the time spent on maintenance and retraining was drastically reduced.

Watch on YouTube Full article

Aug 2, 2026 · 10:51

Understanding AI Agent Hallucination in AI Systems

AI hallucination occurs when an AI system confidently provides information that is factually incorrect. As systems evolve from basic LLMs to autonomous agents (which plan, reason, and take action), the risk landscape changes significantly. While advanced agents can increase error opportunities, grounding them with tools (like search or APIs) dramatically reduces hallucination by allowing verification. Mitigation requires implementing robust design choices: connecting agents to verified 'sources of truth,' enforcing tool-based reasoning, strictly controlling operational scope boundaries, and maintaining a human in the loop for critical decisions.

Key takeaways

Hallucination Definition 0:13
Hallucination is when an AI system confidently provides information that is totally incorrect (1:25). This risk increases as systems move from simple chatbots to autonomous agents.
Agentic Risk Profile 0:02
Agents do hallucinate less when grounded with tools (e.g., search tools, data connectors, RAG) because they can verify information instead of guessing (2:08). However, they introduce more danger because a wrong action (like updating a field or scheduling a meeting) can be taken with complete confidence (3:15).
Mitigation Strategy: Grounding and Tools 0:07
The fastest way to reduce hallucination is to connect the agent to reliable 'sources of truth' (e.g., SharePoint, CRM systems) and enforce tool-based reasoning rather than pure text prediction (7:30).

Watch on YouTube Full article

Jul 21, 2026 · 10:52

Is Fine-Tuning Still Needed? LLMs, RAG, & LoRA

While early successes demonstrated that fine-tuning custom LLMs could outperform general models (e.g., legal AI in 2023), the landscape has shifted significantly. Modern frontier models are rapidly closing the gap due to massive context windows and improved reasoning capabilities. The current architectural best practice suggests prioritizing non-weight modification techniques like Retrieval Augmented Generation (RAG), Context Engineering, and Agent Skills before resorting to fine-tuning. Fine-tuning remains valuable for specific bottlenecks, such as achieving low latency or when using parameter-efficient methods like LoRA.

Key takeaways

Fine-Tuning vs. General Models
Historically, custom fine-tuned models outperformed off-the-shelf leaders (e.g., legal AI over GPT-4 in 2023). However, general models are catching up due to massive context windows and improved inference reasoning.
Modern Customization Stack 6:20
The preferred order for customization is: Base Model $\rightarrow$ Prompt/Context Engineering $\rightarrow$ RAG (for proprietary knowledge) $\rightarrow$ Agent Skills (for procedural know-how). Fine-tuning should be the last resort.
Cost and Complexity of Customization 7:45
Fine-tuning is costly, involving not only training runs but also data collection, evaluation, regression avoidance, and continuous maintenance as frontier models advance.

Watch on YouTube Full article

Jul 9, 2026 · 30:13

How to Go From Data Scientist to AI Engineer (I Did This)

This roadmap guides individuals transitioning from Data Science or Machine Learning into AI Engineering. The core message is that the role shifts focus from statistical modeling in notebooks to becoming a full-stack software engineer capable of building reliable, production-grade AI systems. Key areas covered include closing the software engineering gap (OOP, structured Python projects), mastering LLM backends (FastAPI, Pydantic, Docker), and implementing advanced techniques like Retrieval Augmented Generation (RAG), evaluation (Evals), and guardrails.

Key takeaways

The AI Engineer Shift 0:59
AI Engineering requires moving beyond Jupyter notebooks to structured Python projects using OOP principles, Git, testing, debugging, logging, and environment management. The focus shifts from pure research to building reliable systems around pre-trained models.
Data Science Advantage 2:00
Individuals with a DS/ML background have an advantage because they are trained in statistical thinking (distributions, error analysis) which is critical for making non-deterministic LLM outputs reliable in production.
The Importance of Production Backends 5:29
To build deployable systems, learn to use FastAPI and Pydantic for API creation. Containerization using Docker and persistent data storage with PostgreSQL are essential steps.
Advanced AI Techniques 7:30
Mastering RAG (Retrieval Augmented Generation) requires understanding vector databases (e.g., using the PGvector extension in PostgreSQL). Furthermore, implementing Evals and Guardrails is crucial for quantifying performance and preventing issues like prompt injection.

Watch on YouTube Full article