The Video Signal

Topic

CI/CD

All digests tagged CI/CD

AI Engineer Paris 2026 Opening Keynotes: Mistral, Langfuse & Sizzy | Day 1 thumbnail

· 1:14:46

AI Engineer Paris 2026 Opening Keynotes: Mistral, Langfuse & Sizzy | Day 1

The keynotes covered the profound economic impact of AI as a General Purpose Technology (GPT), detailing historical parallels with the Industrial Revolution. The technical discussion highlighted the evolution of AI development from simple prompting to complex, orchestrated 'software factories.' Speakers emphasized the shift toward robust, enterprise-grade solutions, focusing heavily on agentic safety, data sovereignty, and unified platforms that manage diverse tools and connectors.

Key takeaways

  1. AI as a General Purpose Technology (GPT)

    AI is positioned as a GPT, similar to electricity or the internet, capable of driving long-term economic growth. However, realizing this growth requires more than just invention; it demands diffusion, investment, and organizational change (the 'slow clock' of adoption).

  2. The Shift to Software Factories 20:00

    AI development is moving beyond simple 'vibe coding' and 'meat proxy' roles toward structured 'software factories.' This requires sophisticated orchestration to manage tasks, testing, and code generation in a controlled manner.

  3. Focus on Agentic Safety and Sovereignty 1:03:20

    Enterprise AI requires robust controls to manage risks. Key concepts include dynamic privilege access, runtime guardrails, and data sovereignty (knowing where data is stored and processed) to build trust and compliance.

  4. Advanced Orchestration Tools 41:40

    Modern tooling is moving toward unified, modular systems (e.g., using `Proxmox` or `Exeutor`) to manage multiple AI models and services from a single endpoint, ensuring portability and control.

Watch on YouTube Full article

Ask the Experts: Evaluating Agent Skills | Nemotron Labs thumbnail

· 55:19

Ask the Experts: Evaluating Agent Skills | Nemotron Labs

This session details the evaluation of autonomous AI agent skills using NVIDIA's open-source framework, SkillEvaluator. The system provides a comprehensive, multi-tier pipeline to ensure that skills are safe, reliable, and genuinely useful before deployment. The evaluation process covers static security scanning (detecting risks like prompt injection and data exfiltration), deduplication checks, and live agent testing in a sandbox environment. This framework is critical for establishing quality gates in AI-driven workflows.

Key takeaways

  1. Three-Tier Evaluation Framework 2:00

    Skill evaluation utilizes a three-tier framework: 1) Static/Security checks (using SkillSpector), 2) Deduplication checks against an internal registry, and 3) Live Agent Testing, which assesses if the skill improves task performance in a sandbox environment.

  2. Security Scanning for Agent Risks 4:00

    SkillSpector is a standalone security scanner that checks for agent-native risks, including prompt injection, data exfiltration, trigger abuse, and tool poisoning. It combines static analysis with optional LLM semantic analysis.

  3. Measuring Skill Utility (Skill Lift) 7:30

    The evaluation measures 'Skill Lift' across five dimensions: efficiency, correctness, effectiveness, discoverability, and security. This determines if the skill genuinely helps the agent perform the task better compared to a baseline.

  4. Live Agent Testing Methodology 9:00

    Live agent evaluation runs the agent in a secure sandbox, performing an A/B test by comparing the agent's performance when the skill is available versus when it is restricted. This confirms if the skill is functionally necessary for the task.

Watch on YouTube Full article

What is LangSmith? thumbnail

· 5:33

What is LangSmith?

LangSmith is a comprehensive platform designed for the Agent Development Lifecycle (ADLC), enabling build engineers to build, test, deploy, and monitor LLM applications and agents. It functions as a tracing backend, providing crucial observability into complex agent behavior—which can involve dozens of model and tool calls—by tracking every step, diagnosing bugs, and facilitating continuous quality assurance through structured testing and production monitoring.

Key takeaways

  1. Agent Observability is Critical

    Agents are inherently 'black boxes'; LangSmith solves this by providing visibility into the sequence of model calls and tool decisions, which are not visible in the final output.

  2. Tracing Components 0:01

    LangSmith defines three components: a 'Run' (a single unit of work, e.g., one model call or tool call), a 'Trace' (a full pass through the agent, composed of multiple runs), and a 'Thread' (a conversation grouping multiple traces from one customer interaction).

  3. Testing and Validation Loop 0:02

    The platform uses Datasets (sets of examples), Evaluators (which score examples, potentially using an LLM-as-a-judge), and Experiments (running agents over datasets) to verify fixes and compare performance changes (regression testing).

  4. Production Monitoring 0:03

    In production, LangSmith allows online evaluators to score live traffic, generating dashboards that track scores, volume, latency, errors, and cost, and can trigger alerts or route traces to annotation queues.

Watch on YouTube Full article

What Is MLflow? Tracing AI Agents & LLM Workflows thumbnail

· 9:38

What Is MLflow? Tracing AI Agents & LLM Workflows

This video details the necessity of advanced observability for multi-agent and LLM workflows, arguing that traditional monitoring (HTTP status, response time) is insufficient for diagnosing complex AI failures. MLflow is presented as an OpenTelemetry-compatible platform that captures detailed traces, allowing engineers to monitor not just request success, but the internal logic, tool usage, and quality of the AI's decision path. Key techniques include using LLM judges for non-deterministic evaluation and integrating quality gates into the CI pipeline.

Key takeaways

  1. Beyond Request-Level Monitoring

    Multi-agent systems require visibility into internal failures like silent tool failures, cascading latency, context overflow, and non-determinism, which standard monitoring dashboards cannot detect. (0:00)

  2. MLflow's Core Primitive: The Trace 0:47

    MLflow uses 'traces'—complete records of one request—composed of 'spans.' Each span details a single action (e.g., LLM call, DB query), capturing inputs, outputs, latency, and token counts. (0:47)

  3. Advanced Evaluation with LLM Judges 2:32

    For non-deterministic AI, MLflow allows using a second model (LLM Judge) to grade agent output against criteria like tool call correctness, relevance, and safety. (2:32)

  4. Production Deployment Best Practices 4:10

    For production, the tracking server must use a real database (e.g., Postgres/MySQL), asynchronous trace logging should be enabled, and sampling should be used to manage overhead. (4:10)

Watch on YouTube Full article

What Is Legacy Code? How AI Is Modernizing Legacy Systems thumbnail

· 9:11

What Is Legacy Code? How AI Is Modernizing Legacy Systems

Legacy code, while functional, poses significant risks due to outdated languages (e.g., Cobol, Mainframe), lack of documentation, and deep, misunderstood domain-specific logic. Modernization is challenging due to retiring expert developers and compounding technical debt. Generative AI is emerging as a force multiplier, accelerating the modernization lifecycle by summarizing code, translating languages (e.g., Cobol to Java), and autonomously generating plans, tests, and documentation, allowing teams to focus on innovation rather than maintenance.

Key takeaways

  1. The Risk of Legacy Systems

    Legacy code is mission-critical software that is difficult to maintain, often lacking automated tests and documentation, and handling core infrastructure like financial transactions and healthcare data.

  2. Modernization Challenges

    The industry faces a widening skills gap as experts in old systems retire, and technical debt compounds, creating security vulnerabilities because systems don't receive modern security patches.

  3. AI's Role in Modernization 2:00

    AI accelerates the modernization process by allowing Large Language Models (LLMs) to read entire codebases, summarize module functions, and convert code between languages (e.g., C to Python) much faster than manual effort.

  4. Best Practices for Modernization 4:30

    Successful efforts focus on three areas: breaking monolithic applications into smaller, independent services (architecture); replacing aging infrastructure with modern, cloud-integrated platforms (technology); and implementing automated testing and continuous deployment (process).

Watch on YouTube Full article

Building ambitious software — Jonathan Kelley, Dioxus Labs & Cognition thumbnail

· 19:14

Building ambitious software — Jonathan Kelley, Dioxus Labs & Cognition

Jonathan Kelley discusses building ambitious, cross-platform software using Rust (Dioxus), detailing the challenges of maintaining high quality across a complex stack. He highlights how AI coding agents have dramatically accelerated development—allowing the team to implement features like deeply integrated Kotlin and Swift plugins in weeks—but notes that this shift changes the focus from writing code to maintaining architectural quality. The core message is that while code is now cheap, architectural integrity and quality remain the primary concerns for ambitious projects.

Key takeaways

  1. The Shift in Software Engineering Focus 17:12

    With AI agents handling the low-level development burden (e.g., fighting the borrow checker), the job of a software engineer shifts from writing code to architecting elegant solutions and thinking about system evolution.

  2. Code Quality vs. Code Quantity 17:12

    The speaker asserts that 'code is cheap now, but quality is not.' Maintaining a high quality bar across the entire codebase is critical for developer experience and business stability.

  3. AI Agents as Knowledge Accelerators 13:20

    Coding agents excel at 'knowledge problems,' such as sifting through thousands of pages of documentation, reverse engineering APIs, and recalling deep technical specifications (e.g., CSS specs), accelerating tasks that previously took years of hand-written effort.

Watch on YouTube Full article

Manage your agents while you’re on the move with the Antigravity Remote Control thumbnail

· 0:48

Manage your agents while you’re on the move with the Antigravity Remote Control

Antigravity Remote Control allows users to manage and monitor long-running agents across multiple machines from a single, unified interface (browser or app). This capability ensures that local context is retained, eliminating the need to recreate or sync the build environment when working on different devices.

Key takeaways

  1. Centralized Agent Management

    Users can control and monitor all running agents from one place, providing a unified view for managing development sessions.

  2. Context Persistence

    All local context is retained, meaning the build environment does not need to be recreated or synced when switching devices.

  3. Workflow Integration

    The system supports push notifications for review changes (diffs) and allows users to answer questions and maintain progress from any location.

Watch on YouTube Full article

Catch Agent Regressions Before You Ship: Evals for Managed Deep Agents thumbnail

· 8:57

Catch Agent Regressions Before You Ship: Evals for Managed Deep Agents

This session details how to implement robust evaluation (evals) for Managed Deep Agents to prevent performance regressions as the agent's capabilities grow. The process involves using Harbor, which ensures each evaluation runs in a fresh container. Evals are structured into an environment (data/state), a job (instruction), and a check (verifier). The workflow is scaffolded using `mda evals init`, which can be automated by handing the task to a coding agent (e.g., Claude Code). Results and traces are managed and monitored in LangSmith, allowing for continuous evaluation and integration into nightly CI pipelines.

Key takeaways

  1. Purpose of Evals

    Evals serve two primary goals: catching regressions (ensuring changes don't break existing features) and 'hill climbing' (actively improving agent capabilities). The focus is on defining and catching regressions.

  2. Harbor's Role in Evaluation

    Harbor is bundled into Managed Deep Agents and is crucial because it builds an image around the agent, ensuring every eval runs in a fresh container. It also manages test execution to prevent environment pollution.

  3. Anatomy of an Eval 2:15

    An evaluation consists of three parts: the environment (the starting data/state), the job (the instruction, defined in a markdown file), and the check (the verifier that determines if the job was completed adequately).

  4. Automating Eval Scaffolding 2:36

    The command `mda evals init` scaffolds the necessary files (e.g., `task.md`, `Dockerfile`, tests). Using the `-i` flag allows the work to be handed off to a coding agent, which generates the initial eval suite.

  5. Production Monitoring

    Evals can be managed in production by running them nightly within a CI system. Results are logged into LangSmith, allowing developers to continuously track agent performance and stability.

Watch on YouTube Full article

Baz, Docker & Meta on Verifying Agent Code thumbnail

· 10:14

Baz, Docker & Meta on Verifying Agent Code

The session discusses the critical limitation of current AI coding agents: while they excel at generating code, they often fail at verifying that the code meets complex specifications. Speakers from Baz, Docker, Meta, and Christopher Batey presented advanced strategies—such as using real systems as test oracles, implementing anti-test slop initiatives, and elevating human review to Architectural Decision Records (ADRs)—to ensure reliability and maintain system integrity when integrating AI-generated code.

Key takeaways

  1. Verification is the Hard Part

    The primary challenge with coding agents is not writing code, but proving that the generated code accurately implements the required specification. Agents are optimized for feature generation, not full spec extraction or verification.

  2. Specifying Bugs, Not Features

    Baz demonstrated that providing explicit evidence of a bug (e.g., a screen recording of an overlapping button) is far more effective than general specs, as agents may fail to replicate specific, subtle behavioral constraints.

  3. Test Oracles Over 100% Coverage

    Attempting to achieve 100% test coverage is inefficient. Instead, pointing tests at a real, stable system (a 'test oracle,' like S3) and running a limited set of critical tests can effectively lock down behavior and validate system interactions.

  4. Anti Test Slop Initiative

    Meta implemented an anti test slop initiative, using a separate AI tool to judge the quality and necessity of AI-generated tests, thereby reducing CI overhead from useless or redundant tests.

  5. Elevating Human Review to ADRs

    When reviewing large pull requests (e.g., 7,000 lines), human review must move earlier into the process. Architectural Decision Records (ADRs) should capture system-level decisions, allowing agents to check implementations against a structured, human-reviewed architectural baseline.

Watch on YouTube Full article

Score Every Production Trace with an LLM Judge, from Your Terminal (LangSmith CLI) thumbnail

· 5:08

Score Every Production Trace with an LLM Judge, from Your Terminal (LangSmith CLI)

This walkthrough demonstrates how to implement automated, large-scale evaluation of AI agents using an LLM-as-a-judge within LangSmith. By setting up an online evaluation, developers can programmatically score incoming chat traces (e.g., checking for user frustration) without manually reviewing thousands of interactions. The process involves using a coding agent to install LangSmith skills and configure the evaluation, followed by managing the sampling rate to control costs.

Key takeaways

  1. Online LLM-as-a-Judge Setup

    Online evaluations allow an LLM to automatically score incoming agent traces based on a defined prompt and rubric (e.g., detecting user frustration). The output is a score and reasoning attached directly to the trace.

  2. Using LangSmith CLI for Automation 0:01

    A coding agent can be instructed to use the LangSmith CLI to create and manage the LLM judge evaluator, automating the setup process.

  3. Cost Management via Sampling Rate 0:04

    The sampling rate can be updated (e.g., from 100% to 50%) to reduce evaluation costs while maintaining continuous monitoring.

Watch on YouTube Full article

Tessl Code Review: Catching 74% of Bugs Pre-Merge thumbnail

· 25:56

Tessl Code Review: Catching 74% of Bugs Pre-Merge

Tessl Code Review is an AI-native code reviewer designed to address the critical bottleneck of trusting code generated by coding agents in a software factory. It functions as an automated GitHub app that runs specialized, customizable review rules (lenses) against Pull Requests (PRs). The system emphasizes self-improvement loops, allowing the review process to learn from the codebase and the team's specific best practices, thereby increasing defect detection rates (claiming up to 74% detection).

Key takeaways

  1. AI Code Review and the Software Factory 5:56

    Building a 'software factory' requires more than just plumbing; it requires self-improvement loops that continuously refine the context and skills used by coding agents. Code review is positioned as a critical loop that feeds back into the factory to improve code quality before merging.

  2. Customization via Lenses and Globs 22:07

    The system uses customizable 'lenses' (review rules) that can be tuned to specific domains (e.g., audio DSP) and restricted to specific file paths using globs. This allows for highly targeted reviews, even in large mono-repos.

  3. Local and CI Integration 23:50

    Tessl Code Review can be run directly via the CLI (`Tessl code review give it the profile`) against local changes, enabling developers to catch defects before opening a PR, complementing its automated GitHub app integration.

Watch on YouTube Full article

Meta, Stanford & Odevo on Agentic Coding at Scale thumbnail

· 10:10

Meta, Stanford & Odevo on Agentic Coding at Scale

The session explores scaling agentic coding adoption from a single team to hundreds of engineers. Key findings highlight that while AI tooling can drive massive organic community growth (e.g., Meta reaching 80%+ weekly usage), success is highly dependent on organizational maturity. Speakers warn that deploying agents into an organization with weak software delivery practices will worsen outcomes, emphasizing that foundational improvements—such as robust CI/CD pipelines, dedicated platforms, comprehensive testing, and established coding standards—must precede advanced AI adoption.

Key takeaways

  1. Meta's Adoption Strategy 1:19

    Meta grew an organic community from ad hoc usage to over 40 times its original size. Weekly tool usage increased from under half to above 80%, demonstrating that sustained adoption can be achieved without mandatory enforcement. (00:01:39)

  2. Performance Spread and the 10x Engineer 2:47

    Studies across 150,000 engineers show the widest performance spread ever measured. Contrary to initial hypotheses, top performance is now being achieved by individuals skilled in creating and utilizing agents. (00:02:47)

  3. Prerequisites for Agentic Coding 5:11

    The 2025 DORA report warns that pointing agents at an organization already struggling with software delivery will make things worse. Successful adoption requires fixing fundamentals first: CI/CD, a platform, tests, and coding standards. (00:05:1)

Watch on YouTube Full article

From coding to Knowledge work agents — Karan Vaidya, Composio thumbnail

· 20:42

From coding to Knowledge work agents — Karan Vaidya, Composio

The presentation argues that while autonomous AI agents have excelled in software engineering due to inherent infrastructure support (e.g., Git history, CI/CD), knowledge work agents are currently limited because they lack comparable foundational systems. The speaker identifies six critical primitives—Centralization, History, Context, Verification, Governance, and Reversibility—that must be built into the enterprise layer to enable reliable AI agents for fields like sales and support.

Key takeaways

  1. The Infrastructure Gap

    Coding agents benefit from infrastructure (repo, commit history, tests, CI/CD) that was designed for automation. Knowledge work lacks this surrounding system, causing agents to operate 'blind' when applied outside of code bases.

  2. Centralization is Key 3:55

    Knowledge work data is typically scattered across multiple platforms (e.g., Salesforce, Notion, Gmail, Slack). Agents require a single source of truth—a centralized layer—to pull all necessary threads and connections before they can operate effectively.

  3. The Six Missing Primitives

    To bridge the gap between coding agents and knowledge work agents, six primitives must be built: Centralization (single data source), History (record of past actions), Context (organizational map + style guide), Verification (pre-action checks), Governance (deterministic boundaries/walls), and Reversibility (undo capability).

  4. Failure is Permanent in Knowledge Work 20:00

    Unlike code, where changes can be reverted or walked back, many knowledge work actions (sent emails, wire transfers) are irreversible. This shifts the risk profile, requiring agents to check their work *before* executing any destructive action.

Watch on YouTube Full article

850 PRs a Week: How Tessl Runs a Software Factory thumbnail

· 51:18

850 PRs a Week: How Tessl Runs a Software Factory

The video outlines the concept of a 'Software Factory'—a modern development paradigm where most work is managed by self-improving AI agents. This factory model progresses from defining foundational standards (Skills) to creating automated, iterative workflows (Loops), culminating in full automation. Key insights emphasize that success requires prioritizing context engineering and standardization over raw speed, as the primary gains are increased code quality and functional fungibility across teams.

Key takeaways

  1. The Factory's True Gains 11:57

    Increased capacity does not solely lead to feature velocity; two major benefits observed are an increase in overall code quality (e.g., fixing consistency, copy, and brand voice) and improved fungibility, allowing non-engineering teams (like GTM) to make high-leverage changes without waiting on support.

  2. Context Precedes Automation 13:56

    The core principle is that before automating a process, one must first define 'what correct looks like.' This context definition serves as the manual for operating the company and is critical for building resilient, scalable agents.

  3. Skills $\rightarrow$ Loops $\rightarrow$ Factory 6:10

    The factory model is a continuum: Skills are defined units of work or policies; Loops are automated skills that improve with each run (compounding gains); and the Factory is the system where most development effort shifts to maintaining and improving these loops.

  4. Enforcing Standards: Verifiers and Evals 27:20

    To ensure quality at scale, systems must implement verifiers (generating LLM-judge tools from skills) and evals. These mechanisms enforce standards in CI/CD pipelines, moving code review checks 'left' into the development process.

Watch on YouTube Full article

How AI Is Changing Code Reviews & Software Development thumbnail

· 14:09

How AI Is Changing Code Reviews & Software Development

The evolution of software code reviews has progressed from highly structured methods like Fagan inspections and pair programming, through the version control mechanisms of pull requests (consensus reviews), to automated system checks via CI/CD pipelines. The current shift, driven by AI and Large Language Models (LLMs), is moving the focus away from reviewing implementation details (syntax) toward validating business outcomes, requirements fulfillment, and overall intent.

Key takeaways

  1. Historical Shift in Code Review Focus 2:20

    Reviews progressed from line-by-line inspections (Fagan Inspection) to on-the-fly peer review (paired programming), then to versioning/diff analysis via pull requests (consensus reviews).

  2. The Automation Era (CI/CD) 6:15

    This phase introduced system checks for code quality, compliance (internal/external regulations), and vulnerabilities, establishing 'system reviews' where automated systems validate adherence.

  3. AI-Driven Outcome Validation 7:40

    The AI era shifts the focus from implementation details to business outcomes and requirement fulfillment. AI assists by performing broad analysis (writing code, docs, architectures), allowing human engineers to set context, make judgments, and validate against desired business results.

Watch on YouTube Full article

From AI-Assisted to AI-Native: Building a Frontier Development Team — Clare Liguori, AWS thumbnail

· 20:57

From AI-Assisted to AI-Native: Building a Frontier Development Team — Clare Liguori, AWS

Frontier development represents a step-function improvement in productivity using AI agents, moving beyond simple coding assistance. The core finding is that success is not dependent on the tools themselves, but rather on intentionally changing engineering workflows and building new habits. Key practices include investing heavily in agent context, slowing down to refactor codebases (e.g., migrating from untyped languages like Python/JavaScript to TypeScript or Rust), implementing parallel agent execution, making intent explicit via documentation, and shifting testing left using local deterministic mocks.

Key takeaways

  1. Frontier Development Definition

    Productivity gains are characterized by engineers writing only 1-2% of the code; agents running for hours without interruption; and multiple agents operating in parallel. This represents a median productivity improvement of 4.5x to over 10x compared to previous AI phases (0:00, 7:04).

  2. The Importance of Process Over Tools 11:44

    A pilot study involving 50 teams on existing codebases found that the difference between <3x and >4.5x productivity gains was determined by how the team worked, not the AI tools used (9:21).

  3. New Bottleneck Identification 18:58

    As coding speed increases dramatically, the bottleneck shifts from writing code to decision-making speed and organizational review processes. Fast decisions, especially reversible ones, are critical for scaling (11:38).

Watch on YouTube Full article

AI Code Review That Understands Your PR's Intent thumbnail

· 5:30

AI Code Review That Understands Your PR's Intent

As coding agents write thousands of lines and open numerous PRs, the bottleneck shifts from writing code to reviewing and trusting it. Tessl Code Review addresses this by providing automated review capabilities that are aware of the Pull Request's (PR) intent, not just the diff. Key features include customizable 'review lenses' scoped via `globs` for specific standards (e.g., security or random design), ensuring accountability remains with the human reviewer while automating consistency across an organization's codebase.

Key takeaways

  1. The Shift in Bottleneck 0:35

    With agents writing code at scale, manual review and establishing trust in agent-written code is now the primary bottleneck. Reviewing becomes the critical 'final gate' (00:00:35).

  2. Intent-Aware Review 0:55

    Tessl Code Review differentiates itself by reading the PR summary and title to understand the intended goal of the change, allowing it to review according to context rather than just line changes (00:00:55).

  3. Customizable Review Lenses 2:03

    Review lenses are customizable skills that can be evaluated and distributed across a repository. They can be precisely scoped using `globs` to target specific sections of the codebase (e.g., security or random design) (00:02:03).

  4. Owning the Standard 3:14

    The review standard is defined by a versionable configuration file within the repository itself, ensuring that the team owns and controls the rules rather than relying on external web UI settings or black boxes (00:03:14).

Watch on YouTube Full article

Building Agents Is Trivial Now, Context Is the Next Frontier — Jeff Ng, Unblocked thumbnail

· 13:22

Building Agents Is Trivial Now, Context Is the Next Frontier — Jeff Ng, Unblocked

While cloud primitives and frameworks have made defining AI agents trivial—reducing complexity from requiring dedicated systems for checkpointing, sandboxing, and observability—the primary failure point remains missing organizational context. The speaker argues that simple access layers (like Multiple Connectors/MCPs) are insufficient because 'access is not understanding.' A Context Engine solves this by connecting disparate data sources (docs, code, tickets, conversations) to provide a synthesized, task-relevant understanding that agents can act upon, preventing critical errors and outages.

Key takeaways

  1. Agent Development Complexity Has Decreased

    Six months ago, building an agent required significant effort to solve infrastructure problems like state persistence (checkpointing), isolated sandboxes, and observability. Modern cloud primitives (e.g., Cloudflare, Vercel) have absorbed this 'plumbing,' simplifying agent definition to selecting a model, instructions, tools, and sandbox location.

  2. The Context Gap is the New Bottleneck 7:01

    Agents struggle with institutional knowledge—the decisions, failures, and postmortems stored across different systems (Slack threads, documentation). An agent lacking this full picture can make confidently wrong recommendations, potentially causing outages.

  3. Context Engines Provide Synthesized Understanding

    A Context Engine goes beyond simple data access by building a model of the organization. It reconciles conflicting results across multiple datasets (docs, code, tickets, conversations) and delivers a synthesized understanding that an agent can act on, rather than just raw documents.

Watch on YouTube Full article

LangSmith Preview Builds: Test agent changes before production thumbnail

· 8:01

LangSmith Preview Builds: Test agent changes before production

LangSmith Preview Builds extend standard software development workflows to AI agents. By automatically creating a temporary deployment for every GitHub Pull Request (PR), teams can test agent changes and validate functionality in a production-like environment without impacting the main branch. This system ensures that agents are thoroughly tested on preview branches before being merged, while also guaranteeing automatic teardown of the temporary deployment upon PR merge.

Key takeaways

  1. Preview Builds for Agents

    LangSmith automatically generates a temporary deployment for every pull request, allowing teams to test agent changes safely before they reach production. This brings standard software development workflows (PR testing) to AI agents.

  2. Local Testing and Validation

    Agents can be tested locally using `langgraph dev`, which automatically opens LangSmith Studio for validation. Changes made in the local project are reflected in the preview deployment.

  3. Automated Revisioning

    The system supports continuous iteration: every new commit added to a PR triggers a new revision of the preview deployment, allowing for granular testing before final merge.

  4. Automatic Cleanup (DevEx)

    Once the pull request is successfully merged into the main branch, LangSmith automatically tears down and removes the temporary preview deployment, ensuring clean resource management.

Watch on YouTube Full article

How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face thumbnail

· 20:37

How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face

Niels Rogge details how he automated his role at Hugging Face—the 'Google Drive to the hub' team—which focuses on improving the discoverability of machine learning artifacts. He built two systems: an initial deterministic workflow for outreach (using cron jobs and LLM APIs) and a subsequent fully autonomous agent loop for follow-up actions. The architecture leverages modern tooling like Modal, Bash CLI skills, and advanced models (e.g., GLM 5.2) to scale the process of identifying missing artifacts and prompting researchers to publish them on Hugging Face.

Key takeaways

  1. The Problem: Artifact Discoverability

    ML weights and datasets are often published on third-party services (Google Drive, Zenodo) rather than the centralized platform (Hugging Face), hindering discoverability. The goal is to automate outreach to authors.

  2. Initial Automation: Deterministic Workflow 11:43

    The first phase used a deterministic workflow, running as a nightly cron job on GitHub Actions. This approach utilized LLM APIs in predefined steps without an agent framework, offering high predictability and control.

  3. Advanced Automation: Autonomous Agent Loop 15:36

    The follow-up process was automated using a fully autonomous agent loop (e.g., leveraging the Claude agents SDK). This flexible approach allows the agent to use tools and skills, such as Bash and the Hugging Face CLI, to interact with GitHub issues.

Watch on YouTube Full article