The Video Signal

Topic

AI Agents

All digests tagged AI Agents

Understanding AI Agent Hallucination in AI Systems thumbnail

· 10:51

Understanding AI Agent Hallucination in AI Systems

AI hallucination occurs when an AI system confidently provides information that is factually incorrect. As systems evolve from basic LLMs to autonomous agents (which plan, reason, and take action), the risk landscape changes significantly. While advanced agents can increase error opportunities, grounding them with tools (like search or APIs) dramatically reduces hallucination by allowing verification. Mitigation requires implementing robust design choices: connecting agents to verified 'sources of truth,' enforcing tool-based reasoning, strictly controlling operational scope boundaries, and maintaining a human in the loop for critical decisions.

Key takeaways

  1. Hallucination Definition 0:13

    Hallucination is when an AI system confidently provides information that is totally incorrect (1:25). This risk increases as systems move from simple chatbots to autonomous agents.

  2. Agentic Risk Profile 0:02

    Agents do hallucinate less when grounded with tools (e.g., search tools, data connectors, RAG) because they can verify information instead of guessing (2:08). However, they introduce more danger because a wrong action (like updating a field or scheduling a meeting) can be taken with complete confidence (3:15).

  3. Mitigation Strategy: Grounding and Tools 0:07

    The fastest way to reduce hallucination is to connect the agent to reliable 'sources of truth' (e.g., SharePoint, CRM systems) and enforce tool-based reasoning rather than pure text prediction (7:30).

Watch on YouTube Full article

I Stopped Installing Claude Skills. Here's What I Do Instead. thumbnail

· 16:57

I Stopped Installing Claude Skills. Here's What I Do Instead.

The video provides an advanced deep dive into AI agent skills (used by models like ChatGPT, Claude, and Codex), arguing that these 'superpowers' are often misunderstood. Skills are not traditional apps; they are sets of instructions that must be designed to be both readable by humans for auditing and highly functional for the AI agent during runtime. The speaker emphasizes moving beyond simply collecting skills and instead focusing on structured development, conflict resolution across multiple skills, and utilizing specialized tools like a 'Skill Builder' to ensure reliable, production-grade performance.

Key takeaways

  1. Skills are not applications (apps)

    A skill is simply a set of instructions for the AI agent. Unlike apps, skills do not load their full functionality upfront; only the name and description are loaded initially. The full instruction set is only invoked when the task matches the description, making loading order critical.

  2. The Core Reframing: Dual Audience Design 3:25

    Skills must be written for two audiences simultaneously: the AI agent (for utility) and the human developer (for readability and auditing). If humans cannot read them, developers cannot understand what is being given to the AI.

  3. Auditing and Conflict Resolution

    As agents accumulate many skills (e.g., 25+), conflicts can dull the results because the AI averages out competing instructions. Advanced builders must audit their setup to resolve these performance degradations.

  4. The Role of Structured Development

    To ensure reliability, developers should use tools (like the 'Skill Builder') that enforce best practices for skill files, ensuring clarity in the front matter and structure while maintaining human readability.

Watch on YouTube Full article

Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software thumbnail

· 21:15

Rethinking Environments for Long-Horizon Work — Rayan Garg, Theta Software

The talk addresses the challenge of defining and measuring 'long horizon work' for AI agents. The core argument is that progress depends less on headline benchmark numbers (like human-equivalent hours) and more on rigorous design of the environment and verifier mechanisms. For build engineering contexts, this means focusing on how tasks involve complex tool coordination (e.g., CI/CD logs, databases), managing state changes, and implementing robust 'judge models' that verify correctness from the final system state rather than just the agent's path.

Key takeaways

  1. Defining Long Horizon 0:37

    Long horizon is a scalar metric, but relying solely on human-equivalent time (e.g., 16 hours via Meter) or model metrics (tokens/steps) is insufficient. The most accurate measure requires considering all variables and the inherent complexity of the task.

  2. Measuring Model Capability 4:03

    Model capability should be measured by environment complexity, specifically tool coordination (how many tools are used) and state change complexity. Tasks that can be artificially stretched by chaining unrelated steps do not meaningfully measure model ability.

  3. The Importance of Verifiers 7:12

    For complex software domains, deterministic verifiers are often impractical or impossible. The solution is introducing a 'judge model' (or critic) that verifies correctness by examining both the final state of the environment and the entire execution trajectory.

  4. Addressing Ambiguity 10:38

    Since real-world tasks are ambiguous, standardized evaluation is difficult. Judges must be designed to handle open-ended solutions rather than requiring a single reference answer or sample trajectory.

Watch on YouTube Full article

Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute thumbnail

· 18:20

Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute

The presentation outlines a vision for advanced AI agents that can continuously learn and adapt in real-world enterprise environments post-deployment. The core methodology involves an orchestrated training loop: interactions are captured by an orchestrator, processed through inference engines, graded, and the resulting data (graded chats/traces) is fed into a training engine to compute weight updates. Key challenges addressed include environment fidelity, preventing reward hacking, and transitioning from controlled, replayable synthetic environments to uncontrolled, off-policy production data.

Key takeaways

  1. The Progression of Agent Training

    Agent training moves from simple single-turn Q&A tasks (controlled by a dedicated training stack) to complex, multi-turn, long-horizon tasks that require offloading environment state outside the training stack.

  2. The Core RL Training Loop

    Training relies on an orchestrator driving rollouts, which sends prompts to a model and then passes results to a grader. The resulting graded chats are used by a training engine to compute weight updates for the inference engines.

  3. Addressing Real-World Data Challenges 15:45

    Replicating production environments is difficult due to issues like non-replayability and off-policy data. The future requires methods like automated data pipelines and qualitative feedback ingestion to learn from real interactions.

  4. The Vision: Self-Improving Agents

    The ultimate goal is a single deployment model that can interact across many different settings, continuously self-evaluating and computing weight updates from every interaction it has.

Watch on YouTube Full article

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang thumbnail

· 16:33

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang

Emulated focuses on creating high-fidelity training data environments that simulate entire companies and complex infrastructure operations, moving beyond simple code diffs or single-node sandboxes. The core argument is that for AI agents to achieve true autonomy in mission-critical systems (like cloud providers), they must be trained on long-horizon tasks involving distributed cluster failures, resource provisioning across VPCs/subnets, managing cost constraints, and reasoning through real-world operational incidents.

Key takeaways

  1. The Data Gap in AI Agents 3:30

    Current benchmarks (e.g., SweBench Pro, Terminal Bench) limit agents to operating within a codebase, failing to capture the complexity of real-world tasks like PM communication, performance testing, or owning underlying infrastructure over years.

  2. Complexity Requires Full Simulation 6:10

    Real infrastructure work is not a simple code diff; it involves managing failing nodes, stale deprecated components, live traffic serving, and operational blast radius across distributed clusters.

  3. Limitations of Single-Node Sandboxes 10:40

    Standard post-training pipelines often use homogeneous single-node sandboxes. However, real cloud services require simulating resource provisioning (EC2, Cloud Run), VPCs, subnets, and security groups, which necessitates a multi-node sandbox with access to real infrastructure.

Watch on YouTube Full article

Building Deep Agents and Deploying in Production thumbnail

· 15:40

Building Deep Agents and Deploying in Production

Deep Agents are defined as a sophisticated 'harness' built around foundational LLMs, providing the necessary infrastructure—beyond just the model itself—to make agents reliable and useful in production. The system integrates core primitives like memory, tools, file systems (acting as scratchpads), and middleware hooks. For deployment, critical considerations include implementing durable execution via checkpointing, managing short and long-term memory stores, establishing robust Role-Based Access Control (RBAC) for tool access, and designing for human oversight (human in the loop).

Key takeaways

  1. Deep Agents are a 'Harness' 0:27

    An agent is conceptualized as an LLM plus a harness. The harness encompasses all infrastructure—including system prompts, memory management, tools, file systems, and middleware—that makes the model reliable for a given task. (0:27)

  2. Deep Agents Architecture 6:58

    Deep Agents represent the highest level of abstraction in the LangChain stack, built on top of LangGraph, which provides the core composable nodes and edges necessary for complex agent workflows. (4:18)

  3. Production Reliability Requirements

    For production deployment, agents must handle long-running tasks using durable execution (checkpointing) to recover from failures at any step, manage short/long-term memory across sessions, and incorporate human approval loops. (9:48)

Watch on YouTube Full article

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect thumbnail

· 19:27

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect

Will Brown discusses extending Reinforcement Learning (RL) into complex, real-world tasks that lack clean, verifiable rewards. The core thesis is that 'environments' must serve as the anchor for learning. Techniques like grounding in source material, using LLM judges to audit actions, and employing a reverse direction trick are necessary to generate reliable reward signals when ground truth is unavailable. The ultimate goal is enabling continual learning—allowing deployed agents to autonomously improve by observing and correcting mistakes in messy production settings.

Key takeaways

  1. The Shift from Verifiable Rewards 6:53

    Traditional RL thrives on verifiable rewards (e.g., math, code test cases). However, most real-world tasks (like writing reports or handling refunds) are fuzzy and lack clean best answers, requiring new methods to generate reliable signals.

  2. Environments as the Learning Anchor

    An 'environment' is defined by a task, a harness (e.g., Docker image, codebase), and a scoring rule/verifier. These objects can be used not only for RL but also for Supervised Fine-Tuning (SFT) or prompt optimization.

  3. Mitigating Reward Hacking

    Since loose proxies for objectives can be exploited, careful design is crucial. Techniques include inspecting traces, running small experiments, and using judges to audit rollouts in hindsight.

  4. Generating Signal via Reverse Direction 17:26

    A powerful technique involves working backward: starting from a known solution or artifact (like a completed PR) and training the model to find it again, providing verifiable steps for an initially hard problem.

Watch on YouTube Full article

The misaligned incentives behind AI coding agents thumbnail

· 50:16

The misaligned incentives behind AI coding agents

The conversation details how AI coding agents, exemplified by Devin, are fundamentally changing software engineering workflows. The industry is moving past simply training larger models and focusing intensely on optimizing cost-efficiency (token spend) and speed. Key technical advancements include the 'sidekick' agent architecture for achieving high price performance, developing advanced evaluation metrics like 'mergeability' via Frontier Code, and implementing proactive automation to shift human engineers into decision-making roles rather than routine coding tasks.

Key takeaways

  1. The Shift from Capability to Efficiency 8:36

    As agents mature, the bottleneck is shifting from model training size to running evaluations and managing costs. The focus has moved toward optimizing speed and cost rather than chasing the absolute best-performing frontier model for every task (5:56).

  2. The Role of Mergeability in Evaluation 14:01

    A critical gap in current evaluation benchmarks is 'mergeability'—determining if code, while technically correct, would improve the overall quality or maintainability of a codebase. Cognition developed Frontier Code to address this (8:41).

  3. Cost Optimization via Sidekick Architecture 35:46

    The 'sidekick' agent architecture allows for running both a high-quality, expensive model and a more price-performant model in parallel. This dual approach enables significant cost savings (up to 35% better price performance) without sacrificing quality (21:46).

  4. Proactive Automation and Productivity Guarantees

    Agents are moving from reactive task completion to proactive automation, handling tasks like triaging messages or suggesting fixes. This capability led Cognition to underwrite a $10 million productivity guarantee based on measuring 'productive engineering output' (46:51).

Watch on YouTube Full article

Your Finance Agent's Bottleneck Is You — Ramana Siddanth Emani, Auditoria AI thumbnail

· 13:42

Your Finance Agent's Bottleneck Is You — Ramana Siddanth Emani, Auditoria AI

The primary bottleneck in developing production AI agents, especially for regulated fields like finance, is not model capability or hardware, but the developer's manual workflow. The talk outlines how advanced agent harnesses can automate and multiply human output by implementing primitives such as sub-agents, skills, goals, and loops. By orchestrating these agents across parallel worktrees (e.g., handling multiple Jira tickets simultaneously), developers can drastically reduce time spent on repetitive tasks like testing, building, and deployment, shifting the human role from executor to final verifier.

Key takeaways

  1. The Developer Loop is the Bottleneck

    Shipping production agents fails due to developer velocity limitations. The solution lies in automating the internal developer loop rather than waiting for model or hardware upgrades (0:00, 2:37).

  2. Parallel Task Execution via Sub-Agents 7:02

    Sub-agents allow for parallel processing using isolated 'worktrees'—virtual folders where agents write and execute code independently. This enables handling multiple tasks (e.g., QA bug tickets) simultaneously, far exceeding human capacity (4:22).

  3. Minimal UX for Orchestration 8:38

    Effective agent use requires minimal User Experience (UX), consolidating all project dashboards, logs, Jira tickets, and GitHub PRs into a single pane of glass widget to reduce context switching and manual orchestration effort (5:18).

  4. Achieving Autonomy with Goals and Dreaming

    Future agent development involves combining 'goals' (setting a desired outcome) with automated loops. Furthermore, allowing agents to 'dream' by collecting customer usage patterns can drive self-improvement and feature upgrades autonomously (11:35).

Watch on YouTube Full article

Let's integrate AI Agents in Event-Sourced Systems — Divakar Kumar, FlyersSoft thumbnail

· 21:37

Let's integrate AI Agents in Event-Sourced Systems — Divakar Kumar, FlyersSoft

The presentation outlines an advanced architectural pattern for integrating AI agents into existing enterprise systems, specifically targeting real-time fraud detection. Instead of replacing established Rule-Based Engines or ML models, the approach layers agentic intelligence to resolve 'gray zone' transactions—cases where current deterministic systems lack sufficient context. The architecture leverages Event Sourcing and Domain-Driven Design (DDD) principles by building a centralized semantic layer that aggregates data from multiple bounded contexts (e.g., Transaction, Device, Account). Agents then consume this enriched context via an orchestration layer in a saga pattern to reach a final verdict.

Key takeaways

  1. AI Agents Augment, Not Replace, Existing Systems

    The primary value of AI agents is not replacing existing systems (like rule-based or ML engines) but rather adding an agentic layer to handle ambiguous cases—the 'gray zone' where current models lack necessary context. The goal is to enhance business workflows using the state and history already captured by event sourcing.

  2. Context Aggregation via Semantic Layer 17:37

    To provide sufficient context for agents, data from disparate bounded contexts (Transaction, Device, Account, Payment) must be aggregated into a semantic layer or materialized view. This is achieved by propagating domain events using Change Data Capture (CDC) mechanisms or message brokers.

  3. Saga Orchestration and Agentic Decision Making

    The system uses an orchestration layer to manage the process. This layer employs a fan-out pattern, sending events simultaneously to multiple specialized agents (e.g., Risk Analyzer Agent, Behavior Analyzer Agent). A final Verdict Agent then analyzes these responses to reach a consensus event.

Watch on YouTube Full article

SimulationMaxxing: How Nubank ships agents 20× faster with simulations — Shreya Rajpal, Snowglobe thumbnail

· 16:29

SimulationMaxxing: How Nubank ships agents 20× faster with simulations — Shreya Rajpal, Snowglobe

Nubank achieved a massive acceleration in deploying AI agents—reporting up to 20x faster shipping—by shifting agent evaluation from relying solely on slow, expensive production data or manual curation to using grounded simulations. The core methodology involves running synthetic, multi-turn conversations against the agent within a specialized framework (Snowglobe), allowing teams to rapidly test models and variants in a continuous 'ship, observe, simulate, repeat' loop.

Key takeaways

  1. Simulations Shortcircuit the Eval Bottleneck 10:44

    Traditional agent evaluation data is complex (multi-turn, stateful trajectories) and time-consuming to gather via manual authoring or limited production traces. Simulations allow teams to generate thousands of multi-turn conversations quickly, reducing release cycles from weeks to hours.

  2. The Simulation Loop: Ship, Observe, Simulate

    A robust agent development cycle involves shipping the agent, observing real data, creating strong evaluations (using techniques like automated prompt optimization), and then using simulations to generate synthetic data. This simulated data is piped into the evaluation pipeline for continuous improvement.

  3. Closing the Sim-to-Real Gap

    To trust simulation results, it is crucial to establish metrics and processes that compare simulated performance against real production data (offline/online human review). The correlation between sim quality via evals and real data was found to be high.

Watch on YouTube Full article

We Scored Oracle's Database Skill Live: 95% Isn't Enough thumbnail

· 14:35

We Scored Oracle's Database Skill Live: 95% Isn't Enough

The video demonstrates how Oracle's database team utilized AI 'skills'—encapsulated in a repository like `oracle/skills`—to guide AI agents in writing secure and best-practice compliant SQL. The process involved running an automated review (using Tessl) on a skill designed to improve database code generation, which initially scored 95%. Through iterative refinement and applying changes via a 'run, review, fix' cycle, the skill was successfully upgraded to a 100% success rate, showcasing a robust mechanism for encoding complex domain knowledge into AI workflows.

Key takeaways

  1. Purpose of Database Skills 5:51

    The primary function of the skill is to help agents write better SQL and database code by making them knowledgeable about the database's capabilities (e.g., property graphs, vector data). It helps discover what the database can do, rather than relying solely on general agent knowledge.

  2. Skill Review Process 13:44

    The skill was subjected to an automated review comparing it against best practices. The process involves running a 'run, review, fix' cycle, which applies changes and re-runs the validation in an agile style approach.

  3. Achieving 100% Compliance 14:35

    After making specific developer-suggested changes (e.g., adding a recommended sequence task), the skill's review score reached 100%, demonstrating continuous improvement and validation of domain knowledge.

Watch on YouTube Full article

Your Agent Didn't Fail. Your Harness Did. — Vinoth Govindarajan, OpenAI thumbnail

· 18:26

Your Agent Didn't Fail. Your Harness Did. — Vinoth Govindarajan, OpenAI

The talk argues that most production failures in AI agents are not due to model hallucination or poor reasoning, but rather 'harness failures'—flaws in the underlying system architecture. The core principle is establishing a clear boundary: while a model can only propose an action, the surrounding harness must own the state transition, enforce ordered commits, and provide verifiable proof (the receipt) that the action was successfully executed and observed by the user.

Key takeaways

  1. Model Proposes, Harness Commits, Receipt Proves 4:06

    The model generates capability; the harness provides control. The system must own state transitions, enforce ordered mutations, and use a receipt to prove that an action was executed and visible to the user.

  2. Own the State, Order the Mutation, Prove the Action 5:20

    These three principles are critical for reliable agent systems. A fact must have a single owner (system of record) and a clear replay path; mutable state requires one ordered commit path.

  3. The Run Receipt Audit

    To audit an agent system, ask five questions: What woke it up (trigger)? What state did it inherit? Which authority did it use? What executed? And what evidence survived?

Watch on YouTube Full article

How Forward Deployed Engineering is done at Factory — Eno Reyes thumbnail

· 21:21

How Forward Deployed Engineering is done at Factory — Eno Reyes

Forward Deployed Engineering (FDE) is evolving from traditional professional services consulting into a strategic function where engineers act as 'the tip of the spear' for product development. The core goal is to build an autonomous 'Software Factory' within a customer's environment. This factory automates the entire signal-to-deploy pipeline—from capturing external signals (e.g., bug reports, Slack conversations) through planning, coding, and rigorous validation (SAST, linters, type checkers)—with minimal human intervention. Success hinges on achieving 'agent readiness,' which measures the density of deterministic validation loops in a codebase, enabling massive migrations (e.g., 30-50 million lines of code) autonomously.

Key takeaways

  1. The Shift from Consulting to Product Strategy

    Factory argues that FDE should not be professional services work on behalf of a customer, but rather the stream of information gathered from critical customers. This data is fed back into the product to rapidly improve it, making the deployed engineer an integral part of the product's evolution.

  2. The Software Factory Pipeline 8:23

    This factory concept describes a continuous process where external signals flow in, are prioritized into plans, converted into code changes (the source of truth), pass through automated validation stages (linters, type checkers, SAST tools), and finally deploy. The goal is an uninterrupted signal-to-deploy flow without human intervention.

  3. Data Ownership and Autonomy 10:57

    To ensure scalability and security, the deployed engineer must utilize a model independent agent harness (like Droid) that guarantees the customer owns all traces and data. This allows for highly secure deployments, including air-gapping in sensitive environments like finance or healthcare.

  4. Agent Readiness as a Metric 20:05

    Autonomy is measured by 'agent readiness,' which quantifies the number of deterministic validation loops (e.g., passing linters, type checkers) present in a codebase. A high volume of these loops allows advanced agents to operate on complex tasks for extended periods without human intervention.

Watch on YouTube Full article

AI tools for Forward Deployed Engineering — Vasuman Moza, Varick Agents thumbnail

· 20:23

AI tools for Forward Deployed Engineering — Vasuman Moza, Varick Agents

The core bottleneck in enterprise AI adoption is not model intelligence or execution capability, but rather the ability to deeply understand and re-engineer complex, undocumented human business processes. Forward Deployed Engineers (FDEs) are critical for this process, mapping existing workflows—including edge cases and handoffs—and building autonomous agents on top of legacy systems (e.g., SAP, NetSuite). The technical solution involves using a Veric OS platform that builds agents on existing systems without requiring costly migrations, leveraging dependency graphs, and employing custom post-trained models to extract accurate context from messy enterprise data.

Key takeaways

  1. The AI Bottleneck is Context, Not Execution

    While modern LLMs can solve the execution of work (intelligence constraint), the primary bottleneck remains understanding the unique business processes within a specific company. Every department (e.g., healthcare sales vs. SaaS sales) operates differently, requiring deep context extraction.

  2. Forward Deployed Engineering (FDE) Role 7:24

    FDEs are responsible for mapping how humans currently perform work and then re-engineering the process around AI. This ensures that AI solutions are adoptable and deliver measurable, department-wide ROI rather than failing as isolated 'point solutions.'

  3. Non-Disruptive Deployment Strategy 17:02

    To overcome enterprise resistance to migration (e.g., spending $5 million over five years on NetSuite), agents must be built *on top* of existing systems of record (Salesforce, SAP, Dynamics) rather than requiring a full system overhaul.

  4. Agent Tooling for FDEs

    The Veric platform provides specialized tools: an Engagement Agent (assistant to synthesize notes/docs), and a Workflow Agent that ensures the constructed workflow correctly shadows real-world process edge cases. A future autonomous assistant will handle minor changes without human intervention.

Watch on YouTube Full article

How Forward Deployed Engineering is done at Decagon — Sunny Rekhi thumbnail

· 18:09

How Forward Deployed Engineering is done at Decagon — Sunny Rekhi

The talk details the function and evolution of Forward Deployed Engineering (FDE) at Decagon, a company specializing in AI customer service agents. FDE is described as being functionally identical to product engineering, requiring engineers to not only configure complex agent workflows for specific enterprises but also to proactively identify and build platform features that solve anticipated problems across multiple clients. The core philosophy emphasizes architectural restraint: ensuring all custom work is designed to be self-serve and compound into the overall platform, allowing the solution to scale from bespoke deployments to a generalized product.

Key takeaways

  1. FDE Blurs with Product Engineering

    The line between forward deployed service and internal product development is highly blurred. When an enterprise expresses a pain point, it should be treated as a potential product feature that needs prioritization for the entire platform (13:00).

  2. Focus on Architectural Restraint 13:40

    A critical skill in scaling FDE is exercising restraint—avoiding simple one-off patches and instead architecting solutions so they benefit future, unknown customers. This ensures the solution does not become a brittle 'black box' (8:20).

  3. Custom Work Must Compound

    The goal of every deployment is to ensure that custom integrations or workflows are systematically fed back into the platform, transforming bespoke solutions into self-service capabilities for all customers (14:18).

Watch on YouTube Full article

Introducing the SQL MCP Server - Jerry Nixon - NDC Toronto 2026 thumbnail

· 55:44

Introducing the SQL MCP Server - Jerry Nixon - NDC Toronto 2026

SQL MCP Server, an open-source component from Microsoft's data API builder, provides a secure and standardized abstraction layer for modern applications, especially those powered by AI agents. It allows external models to interact with enterprise databases (including SQL Server, PostgreSQL, Cosmos DB, and MySQL) without requiring direct T-SQL knowledge or exposing sensitive connection strings. The solution standardizes database access into three endpoints—REST, GraphQL, and MCP—simplifying the development process and significantly enhancing security by enforcing policies at the API layer.

Key takeaways

  1. Unified Data Access Layer 2:00

    The SQL MCP Server acts as a single point of entry for all data interactions, supporting multiple database types (SQL, NoSQL) and exposing three standardized APIs: REST, GraphQL, and the specialized Model Control Protocol (MCP). This abstraction layer allows developers to write code against the API surface rather than directly against the underlying schema.

  2. AI Agent Integration via MCP 8:00

    For AI agents, which cannot speak directly to a database, the MCP endpoint is critical. It collapses the potential hundreds of CRUD tools (one per table) into a standardized set of seven tools (Describe Entities, Create, Read, Update, Delete, etc.), preventing model overload and ensuring reliable agentic workflows.

  3. Simplified Development & CI/CD 3:00

    The solution aims to eliminate the need for boilerplate CRUD API code in a codebase. By using data API builder, developers can significantly reduce complexity and potential errors, streamlining the CI/CD pipeline while maintaining high functionality.

  4. Advanced Security & Policy Enforcement 10:50

    Security policies (like Row-Level Security) can be enforced at the API layer, even if structural changes are impossible in the underlying database. Furthermore, it supports advanced authentication methods like OpenID Connect and On Behalf Of (OBO) pass-through authentication.

Watch on YouTube Full article

Inside the Dark Factory: AI That Ships Code Solo thumbnail

· 58:39

Inside the Dark Factory: AI That Ships Code Solo

The video details Tessl's 'Dark Factory,' an autonomous system that handles a significant portion of their code shipping (65-70% of PRs). This factory uses AI agents and sophisticated verification layers to process tickets from inception (Linear) through to merge. The core architectural shift involves moving engineering focus from writing code to designing robust, automated workflows, emphasizing context management and layered verification to build trust in autonomous systems.

Key takeaways

  1. High Automation Rate 2:24

    Tessl routes 65-70% of its PRs through the Dark Factory. Historically, up to 95% of their codebase has never been reviewed by a human.

  2. Shift in Engineering Focus 17:12

    The role of the engineer shifts from implementing tickets to designing complex workflows, scoping tasks for agents, and encoding organizational 'taste' into verification layers (e.g., whiteboarding conversations).

  3. Trust is Earned, Not Enabled 31:00

    Building trust in the Dark Factory requires continuous effort, including running into failure modes and improving verification layers. Accountability remains with the person who filed the original ticket.

  4. Verification Layers are Key 22:22

    The most critical component is not the coding agent itself, but the layered verification system (Verifiers) that allow engineers to encode fuzzy principles (e.g., 'library should be the single source of truth') into deterministic checks.

Watch on YouTube Full article

Llama.cpp vs vLLM: Which Local LLM Engine Actually Scales? thumbnail

· 10:36

Llama.cpp vs vLLM: Which Local LLM Engine Actually Scales?

The video compares Llama.cpp and vLLM, two leading engines for running Large Language Models (LLMs) locally. Llama.cpp is optimized for accessibility on consumer hardware (CPU/GPU), utilizing techniques like quantization and the GGUF format to run models efficiently on personal devices or edge environments. Conversely, vLLM focuses on maximizing efficiency at production scale, supporting diverse accelerators (NVIDIA, TPU, etc.) and implementing advanced optimizations such as continuous batching and paged attention for high-throughput workloads in cloud or Kubernetes deployments.

Key takeaways

  1. Llama.cpp Use Case

    Ideal for running LLMs on consumer hardware (laptops, Raspberry Pi) or edge devices due to its focus on accessibility and CPU/GPU optimization. Key features include quantization (reducing precision from FP16 to INT8/INT4) and packaging models into a single .gguf file.

  2. vLLM Use Case 4:10

    Designed for high-throughput, production-scale workloads in environments like VMs or Kubernetes. It supports diverse hardware accelerators (NVIDIA GPUs, TPUs, etc.) and advanced features like continuous batching and paged attention to manage KV cache efficiently.

  3. Model Deployment Strategy 8:10

    The choice depends on the environment: use Llama.cpp for personal/offline use cases, and vLLM when deploying in a high-performance, multi-user production setting.

Watch on YouTube Full article

AI Agents for Performance: Ship Faster, Pay Less — Rajat Shah, Netflix thumbnail

· 33:39

AI Agents for Performance: Ship Faster, Pay Less — Rajat Shah, Netflix

This talk outlines how Netflix leveraged AI agents to automate performance engineering, addressing the bottleneck of manually identifying and fixing inefficient code patterns in production services. The process involves feeding profiling data (call stacks, CPU usage) into an LLM agent, which identifies suboptimal algorithms (e.g., quadratic-time patterns). The agent then proposes a fix, validates it against a canary deployment using real production traffic, and can even scale the fix across multiple services by identifying common anti-patterns in a centralized catalog. This shifts performance optimization from a reactive, manual effort to a proactive, automated part of the SDLC.

Key takeaways

  1. The Performance Bottleneck 4:00

    Traditional performance engineering is highly manual and time-consuming (taking minutes just to identify hot paths), making it difficult to scale across large codebases. This bottleneck increases as coding agents write code faster, potentially introducing inefficiencies.

  2. AI Agent Workflow for Optimization 12:17

    An LLM agent can automate the entire loop: reading profiling data (call stack/CPU time), identifying anti-patterns (like O(N²) loops), locating the code path in the Git repo, proposing a fix, and validating it via canary deployment.

  3. Scaling Fixes with Pattern Catalogs 21:20

    Instead of fixing one instance, agents can search across multiple services using a centralized 'Pattern/Anti-pattern Catalog' (stored in a Git repo) to find and propose fixes for the same bad pattern repeatedly implemented elsewhere.

  4. Shifting Left: Proactive Performance 26:40

    The goal is to move performance improvement left in the SDLC. The catalog can be used proactively—at code review or even during code authoring—to suggest optimal patterns before suboptimal code reaches production.

Watch on YouTube Full article