The Video Signal

Topic

AI Agents

All digests tagged AI Agents

Harness Engineering: Building an AI Software Factory thumbnail

· 53:34

Harness Engineering: Building an AI Software Factory

Harness engineering is a discipline focused on building automated loops of checks and maintenance agents that allow teams to delegate increasing amounts of codebase development to AI. The goal is not merely higher velocity, but achieving higher quality by shifting from manual code review (the primary bottleneck) to systematic process oversight. This involves tracking three key dimensions—autonomy, automation, and quality—and implementing layered validation systems: the Inner Loop (unit tests/linters), Outer Loop (agentic QA/UI testing), and Meta Loop (maintenance agents that analyze historical data for systemic improvements).

Key takeaways

  1. The Three Dimensions of Agent Adoption 10:39

    When adopting AI agents, track three metrics: Autonomy (how many human course corrections are needed); Automation (the level of oversight required, indicating trust); and Quality (ensuring the shipped product remains high quality). Progressing requires improving these dimensions sequentially.

  2. The Three Loops of Harness Engineering 19:04

    1. Inner Loop (Autonomy): Focuses on cheap, frequent checks like pedantic linting or unit tests to ensure agents get it right the first time. 2. Outer Loop (Automation): Involves slower, in-depth checks, such as agentic code review or running the product through a UI/CLI. 3. Meta Loop: Utilizes maintenance agents that analyze historical data (CI logs, PR comments) to propose systemic fixes and improvements to the entire process.

  3. The Primary Barrier is Organizational 28:20

    Harness engineering is fundamentally an organizational transformation, not just a technical one. Success requires changing workflows—for example, moving from monolithic PRs to smaller, low-risk chunks that can auto-merge, thereby shifting human behavior toward better practices.

Watch on YouTube Full article

Simon Obstbaum & Rob Willoughby - Why evals are hard and how we're solving it - AI Native DevCon Jun thumbnail

· 36:39

Simon Obstbaum & Rob Willoughby - Why evals are hard and how we're solving it - AI Native DevCon Jun

This session introduces advanced methods for evaluating AI agents, arguing that relying solely on 'output evals' (what came out) is insufficient. The focus must shift to 'trajectory evals,' which measure whether the agent followed the correct steps and utilized the right tools. By instrumenting agent behavior—specifically through structured skills and context—teams can significantly improve metrics like PR throughput, decrease cognitive complexity, and ensure adherence to unique organizational conventions (e.g., internal API choices or security policies).

Key takeaways

  1. Shift from Output Evals to Trajectory Evals 29:56

    Evaluating agents requires measuring not just the final output, but also whether the agent activated the correct skills and followed the intended workflow (trajectory) [0:35:46]. Separating activation, trajectory, and outcome is essential for optimizing performance.

  2. Structured Context Improves Code Quality 5:46

    The analysis shows that moving from unstructured (L1) to structured context (L3) significantly improves code quality metrics. Specifically, L2 and L3 teams show increased PR throughput, decreased revert rates, and lower cognitive complexity compared to L1 [0:58:46].

  3. Instruction Following is the Key Differentiator 6:32

    While task completion may remain high regardless of structure, 'instruction following' (grounded in skills) measures adherence to unique organizational rules. This metric shows the biggest lift and represents the value of encoding proprietary business IP into the agent's context [1:03:52].

  4. The System, Not Just the Model, Matters 10:52

    Performance is highly dependent on the entire system stack. Testing must account for the specific model harness (e.g., Opus 4 8 in Claude Code vs. OpenHands), as changing the harness can move scores by up to 100% [1:09:52].

Watch on YouTube Full article

Oleg Šelajev - You're absolutely right, it was your home directory! - AI Native DevCon June 2026 thumbnail

· 33:15

Oleg Šelajev - You're absolutely right, it was your home directory! - AI Native DevCon June 2026

The session addresses the security risks posed by autonomous AI agents running locally, noting that these agents can pose significant threats by accessing sensitive data and executing arbitrary code. To mitigate this risk, Docker introduces sandboxing using MicroVMs—a hardware-level isolation primitive superior to traditional containers. This approach allows developers to run powerful AI agents in a restricted environment with controlled access to the filesystem, network, and secrets, enabling high productivity without sacrificing security.

Key takeaways

  1. The Danger of Autonomous Agents 2:00

    AI agents are highly autonomous and persistent; if given sufficient access (data, external communication, internet), they can execute malicious actions like stealing SSH keys or cloud credentials, leading to massive security incidents. The stakes are asymmetric: the attacker only needs to succeed once.

  2. MicroVMs for Hard Isolation 6:00

    Traditional containers share a kernel and are insufficient for high-stakes AI agent sandboxing due to potential container escaping exploits. Docker's solution uses MicroVMs, providing hardware-level isolation that significantly raises the security boundary.

  3. The Role of Kits (YAML) 10:00

    To improve developer experience in isolated environments, 'Kits' are introduced. These declarative YAML files define reusable configurations for a sandbox, specifying required commands, tools (e.g., programming language toolchains), file placements, and network rules.

  4. Controlled Access Mechanisms 8:00

    Sandboxes implement critical controls including networking proxies (to block specific domains/services) and secrets injection mechanisms, allowing agents to function without direct access to the host machine's private data.

Watch on YouTube Full article

From Agent Traces to Agent Simulations — Rustem Feyzkhanov, Snorkel AI thumbnail

· 20:24

From Agent Traces to Agent Simulations — Rustem Feyzkhanov, Snorkel AI

The talk outlines a shift in AI agent evaluation from simple production 'traces' to controlled, repeatable 'simulations.' To reliably develop and deploy agents, companies must build private benchmarks that mimic their specific production environment, tools, and policies. These simulations allow engineers to test the entire agent stack—including cost, latency, and policy adherence—and integrate this process into a formal CI pipeline for continuous improvement (Agent Ops).

Key takeaways

  1. Private Benchmarks are Essential for Production Readiness 6:41

    Public benchmarks (e.g., WebArena) are useful for general orientation but fail to capture domain-specific use cases, internal tooling, or company policies. A private benchmark is necessary because it allows comparison on critical metrics like cost per task and latency, not just success rate.

  2. Simulation Enables Full Stack Evaluation 2:00

    Offline simulation turns production traces into repeatable experiments. This allows testing the full agent stack (model, prompt, skills, tools) while keeping the environment and evaluators constant between runs, enabling apples-to-apples comparisons.

  3. Benchmark Development Requires CI/CD Discipline

    The benchmark itself must be treated as software. It requires a dedicated CI pipeline to ensure dependencies are pinned, fixtures are present, and the core Oracle solution passes before it can be used for agent testing.

Watch on YouTube Full article

From Signal to PR: Anatomy of a Self-Improving Agent — Jason Lopatecki, Arize thumbnail

· 20:36

From Signal to PR: Anatomy of a Self-Improving Agent — Jason Lopatecki, Arize

The future of observability is shifting from human-driven dashboards to machine-readable telemetry that powers autonomous AI agents. Arize's Signal automates the debugging process by pulling deep production traces and logs (sometimes ten megabytes) directly into the repository as files. This allows coding harnesses, like Claude Code, to understand the exact code path taken during an error, enabling agents to propose fixes and creating a continuous loop where systems can autonomously improve themselves. The human role is evolving from responder to reviewer.

Key takeaways

  1. Observability Shift (2.0) 2:16

    Observability is moving beyond UI clicks and graphs; it's becoming a 'smoke'—telemetry data that agents can read to debug software, allowing for continuous automated fixing.

  2. The Key Unlock: Traces on Filesystem 6:08

    The critical breakthrough is pulling relevant production traces and logs down as files into the repo. Coding agents are highly effective with file formats, giving them the exact code path rather than guessing among millions of branches.

  3. Autonomous Fixing Loop 6:54

    The goal is to build systems that autonomously fix themselves. The process involves an agent investigating first, gathering deep evidence (traces/logs), and proposing a fix before human intervention.

  4. Security and Deployment

    To ensure compliance for large enterprises (e.g., Uber, Booking), agents must run within the customer's Virtual Private Cloud (VPC) using sandboxes, preventing production systems from connecting directly to external models.

Watch on YouTube Full article

The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI thumbnail

· 6:06

The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI

The complexity of modern AI agents—which now incorporate reasoning, tool calls, and long multi-step loops—has rendered traditional evaluation methods insufficient. The talk argues that the future of evaluating these systems lies in moving from static deterministic checks or fixed LLM rubrics to 'Agent as a Judge,' which performs adaptive dynamic analysis to uncover subtle failure modes.

Key takeaways

  1. Evals are critical for AI maturity

    Evals have become essential for serious AI teams, with the industry noting that they catch all failures and fuel continual learning loops. Arize reports running over 100 million evals monthly.

  2. Agent complexity breaks traditional evals 3:26

    As agents evolved from simple prompt answering (2023) to complex, multi-step loops with sub-agents and dynamic UI creation, the failure modes became fundamentally different, exceeding the scope of classical LLM as a Judge checks.

  3. The future requires adaptive evaluation 5:45

    While deterministic checks and LLM-as-a-Judge are valuable, the next step is 'Agent as a Judge,' which provides adaptive dynamic analysis to find failure modes that were previously undetectable.

Watch on YouTube Full article

Build Hour: Valuemaxxing with GPT-5.6 thumbnail

· 54:33

Build Hour: Valuemaxxing with GPT-5.6

The session provides a deep dive into 'Value Maxxing' with GPT-5.6, shifting focus from merely maximizing token usage to optimizing AI agents for concrete business outcomes and cost efficiency. Key strategies include selecting the optimal model (Sol, Terra, Luna) based on workload needs, implementing advanced API features like programmatic tool calling and prompt caching, and structuring agent workflows to minimize redundant context processing.

Key takeaways

  1. Shift from Token Maxxing to Value Maxxing 2:30

    Progress should be measured by the value generated (e.g., time saved, quality improved), not just the number of tokens consumed. This requires defining clear outcomes and measurable 'good' for AI agents (evals).

  2. Model Selection Strategy 4:09

    The GPT-3.5 family includes Sol (flagship/complex tasks), Terra (balanced intelligence/cost/latency), and Luna (high-volume, cost/latency sensitive) to optimize for specific workloads.

  3. Optimizing Agent Workflows with Codex 5:28

    For day-to-day coding, starting with `GPT-5.6 Soul` on medium reasoning is often sufficient. Developers can also trade tokens for time using Fast mode or utilize Chronicle to build task memory.

  4. Advanced API Techniques (Harness Optimization) 6:51

    Implement programmatic tool calling (using a JavaScript sandbox) and prompt caching to significantly reduce input token costs and processing time. Persistent reasoning and compaction also boost performance and cache efficiency.

Watch on YouTube Full article

Everything Is a Rollout — Alex Shaw + Ryan Marten, Terminal-Bench, Harbor, Laude Institute thumbnail

· 21:11

Everything Is a Rollout — Alex Shaw + Ryan Marten, Terminal-Bench, Harbor, Laude Institute

Alex Shaw introduces Harbor, a framework designed for evaluating and optimizing AI agents. The talk argues that agent development should be viewed through the lens of machine learning rather than traditional software engineering. This requires treating agent performance as a 'blackbox artifact' and managing it via empirical evaluation—a process formalized by 'rollouts.' Harbor provides the necessary infrastructure (sandboxes, standardized environments) to execute these complex evaluations in parallel.

Key takeaways

  1. Agent Development vs. Software Engineering 5:15

    Unlike traditional software engineering where behavior is predictable before execution, agentic coding and AI agents are best treated as blackbox artifacts whose performance requires empirical evaluation (e.g., 'Generated code is best treated as a blackbox artifact').

  2. The Role of Rollouts in Agent Evaluation 10:30

    Agent evaluation relies on 'rollouts' within sandboxed environments. This process involves passing the sandbox to the agent, collecting a trajectory, and then passing it to a verifier which produces rewards. Harbor standardizes this universal process.

  3. Harbor as an Interoperable Standard 12:20

    Harbor is presented as a common language and open-source framework for specifying agentic environments, allowing interoperability across different agents, models (e.g., GPT 5.5), and sandboxes to maximize data velocity.

  4. Diverse Evaluation Use Cases 17:30

    Evaluation can be highly specialized, including assessing how well agents build products (e.g., RampBench), how they use a product's headless mode, or automating internal processes.

Watch on YouTube Full article

Full Workshop: Setting Yourself Up for Success —Jason Liu, OpenAI Codex thumbnail

· 1:15:02

Full Workshop: Setting Yourself Up for Success —Jason Liu, OpenAI Codex

Jason Liu provides a comprehensive workshop on leveraging Codex and the Agents SDK for advanced knowledge work automation. The focus shifts from simple text generation to using AI agents (threads) as collaborative teammates capable of managing complex projects, interacting with external applications via 'Computer Use,' and maintaining long-running processes through automated 'heartbeats.' Key strategies include building custom skills/plugins, utilizing a personal memory vault, and treating the computer itself as an integrated component of the workflow.

Key takeaways

  1. Systematic Context Management via Appshots 20:05

    The 'Appshot' feature is highlighted as critical for providing deep context to AI models. Unlike simple screenshots, Appshots capture the entire accessibility tree of an application (e.g., Slack), allowing agents to perform complex actions with single function calls and greatly improving accuracy.

  2. Advanced Project Management with Threads 5:50

    The concept of 'compaction' allows threads (pinned conversations) to maintain long-term context, acting like dedicated project managers. These threads can delegate tasks to sub-agents and communicate with each other, forming a collaborative team structure.

  3. Automating Long-Running Workflows (Heartbeats) 11:30

    Instead of manual checks, 'heartbeat' automations can wake up threads over time to perform continuous maintenance tasks (e.g., checking PR status, monitoring support issues), allowing for sustained project oversight.

  4. Extending AI Capabilities with Skills and Plugins 23:25

    Skills are simple constructs (files/scripts) that solve specific problems (e.g., 'Review my code like Charlie'). Plugins connect the system to external services (Slack, Gmail), allowing agents to perform actions across different platforms.

Watch on YouTube Full article

Why AI Agents Need a Data Harness, Not Just a Lakehouse thumbnail

· 37:07

Why AI Agents Need a Data Harness, Not Just a Lakehouse

The talk outlines the necessity of a 'data harness'—a modern data foundation optimized for conversational-pace querying by AI agents. Traditional data pipelines built for minutes-to-hours turnaround fail under this rapid load, leading to errors and wasted tokens. The solution requires combining an open lakehouse architecture (using standards like Apache Iceberg and Apache Arrow) with a robust semantic layer, advanced caching (like Columnar Cloud Cache/C3), and federation capabilities to ensure data is accessible, understandable, and performant for AI use cases.

Key takeaways

  1. AI Agents Require Subsecond Performance 6:32

    Conversational analytics moves at a conversational pace. Legacy systems designed for minutes-to-hours turnaround cannot support this speed; subsecond response time is a critical requirement for agentic workflows.

  2. Three Pillars of AI-Ready Data Foundation 9:32

    Any modern data foundation must be accessible (seeing the whole estate), understandable (having clear, standardized definitions via a semantic layer), and performant (querying data where it lives without moving or cleaning it).

  3. Open Standards Prevent Vendor Lock-in 24:12

    The use of open standards like Apache Iceberg, Apache Arrow, and Apache Polaris ensures that the data platform remains vendor-neutral. This allows users to build performance without sacrificing portability.

Watch on YouTube Full article

Perception Agents — Antje Barth, Amazon AGI Lab thumbnail

· 21:45

Perception Agents — Antje Barth, Amazon AGI Lab

Current AI agents excel at discrete tasks like clicking or calling APIs but fail in complex, end-to-end knowledge work because they lack reliable perception and verification capabilities. The talk introduces 'Perception Agents,' which close the architectural gap by enabling agents to perceive rendered UIs (not just underlying code), maintain shared context, and verify their own output against design specs or user flows, mimicking human collaboration.

Key takeaways

  1. The Gap in Agent Capability

    Current agents struggle with end-to-end workflows because the 'real work' lives within the seams of multiple applications. While they can perform individual steps, they cannot manage the full process reliability required for critical tasks (e.g., deleting a database).

  2. The Need for Reliability and Verification 3:50

    Unlike code, which is verifiable through unit tests, most knowledge work is 'messy' and lacks easy verification methods. This lack of verifiability is the primary hurdle to building trust in agents.

  3. Perception Agents: Closing the Loop 7:40

    A perception agent must perceive the screen (rendered layout, state) like a human, not just scrape code. They must complete the loop by observing results to confirm if actions succeeded, rather than simply firing off commands.

  4. Shared Context and Multimodal Perception 10:40

    Perception is more than just visual input; it includes understanding context from sources like audio transcripts. The goal is to build agents that react in real-time, similar to human collaboration, without the back-and-forth of prompt/response cycles.

Watch on YouTube Full article

Coding Agents Are Secretly General Agents thumbnail

· 1:12:03

Coding Agents Are Secretly General Agents

The discussion posits that coding agents are inherently generalist, meaning proficiency in code translates into superior performance across all knowledge work tasks due to a concept called 'positive transfer.' The future of knowledge work is converging on single, integrated platforms (Systems of Record) that provide comprehensive context and surfaces for agent interaction. Key technical advancements include using verifiable code execution environments (like unit testing/linting) as the perfect training ground for agents, leading to autonomous workflows like ticket-to-pull request cycles.

Key takeaways

  1. Coding Agents are Generalist Agents 22:00

    The core thesis is that improving an agent's ability to write and execute code makes it better at everything else. This 'positive transfer' capability means agents with coding skills are effectively AGI-complete, as they can write their own tools and interact with various systems.

  2. Verifiability is Key for Agent Training 17:15

    Code provides an ideal training ground because its output (e.g., a function, schema) can be programmatically verified (linted or passed through unit tests). This verifiable feedback loop allows agents to learn and refine their performance iteratively, which is crucial for autonomous workflows.

  3. Convergence of Platforms Wins 23:40

    The most successful platforms will be those that achieve convergence—integrating context, surfaces, and unit economics into a single system (a 'System of Record'). Fragmentation (e.g., Slack's data walls) is identified as the primary enemy to agentic workflow adoption.

  4. The Future is Autonomous Knowledge Work 26:40

    The trend suggests that much of today's office work will be handled by agents. This shift means platforms must evolve from being communication hubs (like Slack) to becoming the central operational layer where all data and tasks reside.

Watch on YouTube Full article

Knowing When Not to Use AI: AI Agents vs Rules vs ML thumbnail

· 10:35

Knowing When Not to Use AI: AI Agents vs Rules vs ML

System design requires disciplined choices regarding intelligence types—Human judgment, Rules/Code, Machine Learning, or Generative AI—rather than defaulting to AI agents for every problem. The choice must balance trade-offs across accuracy, cost, complexity, and risk. Successful modern systems are typically hybrid, combining the determinism of code with the pattern recognition of ML and the flexibility of LLMs.

Key takeaways

  1. Human Judgment 2:00

    Best for high-stakes decisions, ambiguity, ethical considerations, or situations requiring accountability (e.g., medical diagnosis, legal interpretation). Trade-offs include being expensive, slow, and difficult to scale.

  2. Rules/Code-Based Solutions 3:00

    Ideal for tasks requiring clear, stable logic, consistent exact outputs, and zero error tolerance (e.g., payment processing, input validation, security access control). Code is fast, cheap, reliable, and highly interpretable.

  3. Machine Learning (ML) 4:10

    Excels at finding patterns in structured data and making probabilistic predictions when rules are too complex to define manually (e.g., fraud detection, customer churn prediction). Requires monitoring for model drift.

  4. Generative AI (LLMs/Agents) 5:40

    Best used when inputs are unstructured (text, documents) and tasks require reasoning or transformation. Flexibility is prioritized over precision, and some error is tolerated (e.g., summarization, intent understanding). Trade-offs include non-determinism and higher cost at scale.

Watch on YouTube Full article

Robert Overweg - One Brain, No Filtering - AI Native DevCon June 2026 thumbnail

· 31:25

Robert Overweg - One Brain, No Filtering - AI Native DevCon June 2026

The presentation details a 'One Brain' concept—a centralized, AI-native knowledge management layer designed to eliminate information silos and improve decision-making by weaving together research, client context, and operational data. The system uses an orchestrator (OpenClaw) and structured vaults to allow agents to access and synthesize organizational knowledge in real time, shifting focus from manual file retrieval to idea generation and proactive insights.

Key takeaways

  1. Shift from File Search to Idea Synthesis 5:40

    The core value lies in moving beyond searching for specific files; the system allows users to search for 'ideas' or 'contacts.' Agents can interpret natural language queries (e.g., asking about CI/CD steps) and provide contextually accurate answers based on stored knowledge.

  2. Structured Knowledge Flow 7:10

    Knowledge is categorized into 'company knowledge' (new developments, research wikis) and the 'creation pipeline.' Information must be promoted to a central vault from various sources (e.g., Obsidian notes, meeting transcripts) to gain grounding in reality before being shared widely.

  3. Scaling and Security Challenges 17:32

    While the system is powerful, scaling remains a challenge, particularly regarding data segregation (permissions) across different client or team buckets. The local setup on one person's laptop was initially used for testing, but enterprise rollout requires careful consideration of security boundaries.

Watch on YouTube Full article

Expose your site's actions to AI agents using WebMCP thumbnail

· 1:29

Expose your site's actions to AI agents using WebMCP

This talk introduces WebMCP (Web Manifest Capabilities Protocol), a method for web sites to expose their functional capabilities directly to AI agents. Instead of relying on agents to interpret the UI like a human, developers can register tools using plain JavaScript, defining explicit inputs and outputs via a schema. This allows agents to perform direct tool calls (e.g., 'Download report CSV') rather than attempting button interactions, significantly improving reliability for automated agent workflows.

Key takeaways

  1. Problem with Current AI Agent Interaction

    AI agents often fail or go off course because they misinterpret the visual interface (UI) of a website, forcing them to interact like a human user.

  2. WebMCP Solution: Exposing Capabilities

    WebMCP allows sites to explicitly expose what they can do. Developers register tools using plain JavaScript, providing a name, description, and function that performs the action.

  3. Reliable Agent Interaction via Schema

    By defining explicit inputs and outputs (the schema), agents can bypass guesswork. They discover capabilities and execute direct tool calls, such as calling 'Download report CSV' directly.

  4. Alternative Exposure Methods

    If JavaScript is not used, capabilities can be exposed by annotating supported HTML forms instead.

Watch on YouTube Full article

60% Faster Time-to-Interview: Transforming Hiring with AI Agents with LangChain thumbnail

· 18:05

60% Faster Time-to-Interview: Transforming Hiring with AI Agents with LangChain

LinkedIn details the architecture of a hiring agent built with LangChain and LangGraph that successfully cut time-to-interview by 60% for small businesses. The system evolved from static workflows to an advanced agentic control model utilizing a central planner within a plan-execute-replan loop. Key architectural components include specialized memory types (conversational and experiential), middleware hooks for PII detection, and rigorous 'harness engineering' techniques—such as state flag chaining and one-shot tool guards—to ensure the probabilistic nature of LLMs results in a dependable product.

Key takeaways

  1. Hiring is an Agent Problem

    The hiring process is inherently iterative (plan, act, observe, adapt), requiring continuous adaptation rather than being a one-shot task. This necessitates an agentic approach.

  2. Architectural Evolution to LangGraph 0:03

    The system progressed from hard-coded static workflows (if/then) to sequential LangChain chains, culminating in LangGraph for its true agentic control model featuring a central planner and plan-execute-replan loop.

  3. Choosing LangGraph 0:05

    LinkedIn selected LangGraph over 89 evaluated frameworks because it complements existing infrastructure, builds upon core LangChain primitives (runnables, tools), and allowed for zero rewrite adoption.

  4. Achieving Determinism via Harness Engineering 0:10

    To make the agent dependable, LinkedIn implemented advanced 'harness engineering' techniques, including context management (checkpoint trimming), output format determinism (template confirmation/fallbacks), and node-change determinism (state flag chaining and one-shot tool guards).

Watch on YouTube Full article

Better Agent Auth — Bereket Habtemeskel & Paola Estefania, Better Auth thumbnail

· 40:56

Better Agent Auth — Bereket Habtemeskel & Paola Estefania, Better Auth

This workshop details the Agent Auth protocol, a solution designed to secure autonomous AI agents operating services for users or organizations. The core principle shifts from granting broad user credentials (acting *as* the user) to giving specific, traceable authority (acting *for* the user). Key mechanisms include agent identity management using private keys, fine-grained capability discovery via directories, and granular authorization that limits actions to explicitly granted capabilities.

Key takeaways

  1. Shift from Credentials to Authority 2:00

    Instead of providing agents with full user credentials, the protocol treats the agent as a principal actor with its own distinct authority. This is analogous to hiring an assistant rather than giving them CEO access.

  2. Capability Discovery and Authorization 4:05

    The system requires a directory mechanism for agents to discover what services are available and what specific actions (capabilities) they can perform, moving beyond broad scopes like 'read.'

  3. Agent Identity and Traceability 5:10

    Every agent must possess a unique identity (private key). This allows the system to maintain comprehensive audit logs, track exactly which agent performed an action on behalf of which user, enabling immediate revocation if misuse occurs.

Watch on YouTube Full article

Every Harness Will Become A Claw — Sam Bhagwat, Mastra thumbnail

· 15:36

Every Harness Will Become A Claw — Sam Bhagwat, Mastra

The evolution of AI agents is moving from localized 'Harnesses'—tools used for coding and task execution—to persistent, always-on services called 'Claws.' This transition involves imbuing agents with initiative, external connectivity (like a heartbeat), and continual learning capabilities. The speaker proposes Steinberger's law: every harness will expand until it becomes a Claw, driven by the desire for powerful, integrated developer experiences.

Key takeaways

  1. Harnesses are evolving into Claws 1:42

    The next generation of agents moves beyond local execution to become always-on services that listen to external events (e.g., Slack, mobile apps) and maintain a persistent 'heartbeat.'

  2. Agentic Spectrum Advancement 0:49

    Agents are advancing through stages: Agent $ ightarrow$ Harness $ ightarrow$ Claw. Key technical advancements include durability, doggedness, planning mode, and parallel subagents.

  3. Cloud vs. Local Architecture 1:48

    The shift from local harnesses to cloud harnesses provides greater parallelism and resources but necessitates a different distributed system architecture.

Watch on YouTube Full article

Why This Company Won't Let AI Agents Touch Bash thumbnail

· 1:06:04

Why This Company Won't Let AI Agents Touch Bash

The presentation details the evolution of an internal AI agent platform at Cyera, transforming a personal assistant project into an enterprise-wide tool for data security and operational efficiency. The core focus is on building robust guardrails to prevent agents from becoming uncontrolled 'black boxes.' Key architectural innovations include whitelisting tools over blacklisting them, implementing a structured citation system for verifiable claims, and replacing traditional RAG with a Knowledge Graph (KG) that allows agents to navigate interconnected data like an LLM wiki. The platform emphasizes controlled deployment, allowing multiple developers to build and own specialized agents while maintaining centralized security and governance.

Key takeaways

  1. Controlled Agent Architecture

    The system prioritizes control by whitelisting specific tools rather than blacklisting forbidden actions. A critical guardrail is the use of structured, validated output, ensuring that an agent cannot execute arbitrary code (like unrestricted Bash) or leak sensitive data outside its designated context.

  2. Citation and Hallucination Mitigation 20:55

    To ensure reliability, every claim generated by the agent must be backed by a citation. This is achieved by forcing the model to output structured blocks containing both the claim and the source reference. A second model then performs clean-context verification against the raw data to drastically reduce hallucinations.

  3. Knowledge Graph over RAG 32:33

    The platform utilizes a Knowledge Graph (KG) instead of standard RAG for context retrieval. This allows agents to 'walk' connections between entities (e.g., an exception, a service, and the related pull request), providing more structured and reliable data exploration than simply dumping retrieved documents.

  4. Adoption through Platform Engineering 50:27

    To drive adoption across engineering teams, the platform was designed to be easily customizable. By allowing users to name and modify their own agents (e.g., 'It's my agent'), the barrier to entry is lowered, promoting organic growth with a strategy of 'carrots, not sticks.'

Watch on YouTube Full article

Your agent architecture has a half-life of 6 months — Dan Farrelly, CTO, Inngest thumbnail

· 19:20

Your agent architecture has a half-life of 6 months — Dan Farrelly, CTO, Inngest

The talk addresses the rapid obsolescence of AI agent architectures, noting that their 'half-life' can be as short as six months due to evolving models and frameworks. The core thesis is that engineers must build systems by focusing on three decoupled conceptual layers—Execution, Context, and Compute—to ensure stability. The Execution Layer is identified as the most stable component, responsible for managing flow, state durability, retries, and coordinating complex workflows regardless of changes in LLMs or prompts.

Key takeaways

  1. Agent Architecture Volatility

    AI agent architectures are highly volatile; components like prompts may last weeks, models months, and frameworks can quickly become outdated. This rapid change necessitates an architectural approach that decouples core primitives to prevent technical debt.

  2. The Three Conceptual Layers 3:30

    Effective agent design requires considering three discrete layers: the Execution Layer (the 'brain,' handling flow, state, and durability); the Context Layer (the 'knowledge,' including models, prompts, tools, and memory); and the Compute Layer (the 'hands,' involving sandboxes, runtimes, or browsers).

  3. Focus on the Execution Layer 5:20

    The execution layer must be designed to remain stable over years. It is responsible for managing the full life cycle of an agent—including planning, model calls, code running, and sub-agent invocation—independent of the context or compute used.

Watch on YouTube Full article