The Video Signal

Topic

Agent Reliability

All digests tagged Agent Reliability

Tessl Code Review: Catching 74% of Bugs Pre-Merge thumbnail

· 25:56

Tessl Code Review: Catching 74% of Bugs Pre-Merge

Tessl Code Review is an AI-native code reviewer designed to address the critical bottleneck of trusting code generated by coding agents in a software factory. It functions as an automated GitHub app that runs specialized, customizable review rules (lenses) against Pull Requests (PRs). The system emphasizes self-improvement loops, allowing the review process to learn from the codebase and the team's specific best practices, thereby increasing defect detection rates (claiming up to 74% detection).

Key takeaways

  1. AI Code Review and the Software Factory 5:56

    Building a 'software factory' requires more than just plumbing; it requires self-improvement loops that continuously refine the context and skills used by coding agents. Code review is positioned as a critical loop that feeds back into the factory to improve code quality before merging.

  2. Customization via Lenses and Globs 22:07

    The system uses customizable 'lenses' (review rules) that can be tuned to specific domains (e.g., audio DSP) and restricted to specific file paths using globs. This allows for highly targeted reviews, even in large mono-repos.

  3. Local and CI Integration 23:50

    Tessl Code Review can be run directly via the CLI (`Tessl code review give it the profile`) against local changes, enabling developers to catch defects before opening a PR, complementing its automated GitHub app integration.

Watch on YouTube Full article

Accelerate agent improvement with LangSmith Engine thumbnail

· 3:25

Accelerate agent improvement with LangSmith Engine

LangSmith Engine automates agent improvement by analyzing large volumes of production traces (e.g., 20,000 traces). It identifies recurring issues—such as agents sending outreach to disqualified prospects without warning—and proactively proposes code changes and updated instructions. Furthermore, it automatically generates evaluation datasets from failing runs, allowing engineers to test proposed fixes against historical failures before deployment.

Key takeaways

  1. Automated Issue Identification

    Engine reviews production traces to surface recurring problems (e.g., agent outreach despite CRM flagging a prospect as disqualified), grouping related issues across multiple agent runs.

  2. Proposed Fixes and Instructions 2:05

    Beyond identifying the error, Engine drafts concrete changes: modifying tools (e.g., adding a 'confirmed disqualified' flag) and refining system prompts to include exceptions (e.g., requiring explicit confirmation before sending).

  3. Automated Evaluation Dataset Creation 2:40

    Engine handles the tedious process of growing evaluation datasets by formatting failing production runs into examples, including reference outputs detailing how the agent should have behaved.

Watch on YouTube Full article