# What is LangSmith?

## Executive summary

LangSmith is a comprehensive platform designed for the Agent Development Lifecycle (ADLC), enabling build engineers to build, test, deploy, and monitor LLM applications and agents. It functions as a tracing backend, providing crucial observability into complex agent behavior—which can involve dozens of model and tool calls—by tracking every step, diagnosing bugs, and facilitating continuous quality assurance through structured testing and production monitoring.

## Key takeaways

- Agent Observability is Critical: Agents are inherently 'black boxes'; LangSmith solves this by providing visibility into the sequence of model calls and tool decisions, which are not visible in the final output.
- Tracing Components: LangSmith defines three components: a 'Run' (a single unit of work, e.g., one model call or tool call), a 'Trace' (a full pass through the agent, composed of multiple runs), and a 'Thread' (a conversation grouping multiple traces from one customer interaction).
- Testing and Validation Loop: The platform uses Datasets (sets of examples), Evaluators (which score examples, potentially using an LLM-as-a-judge), and Experiments (running agents over datasets) to verify fixes and compare performance changes (regression testing).
- Production Monitoring: In production, LangSmith allows online evaluators to score live traffic, generating dashboards that track scores, volume, latency, errors, and cost, and can trigger alerts or route traces to annotation queues.

## Technical details

- LangSmith Architecture: LangSmith is a tracing backend, not a LangChain feature, meaning it can be used with any agent framework (LangChain, LangGraph, OpenAI/Claude SDKs, or custom loops).
- Tracing Implementation: Integration is achieved via setting environment variables (for LangChain/LangGraph) or using a one-line integration (for OpenAI/Claude SDKs). Custom loops require adding a decorator to the function.
- Debugging and Diagnosis: By reading a trace, engineers can pinpoint bugs (e.g., an agent adding a pizza ingredient despite a zero stock unit lookup) and view associated latency and token costs per step.
- Advanced Features: The platform supports Annotation Queues for human review and scoring, and Insights, which analyzes batches of traces to categorize user questions and identify areas where the agent may lack support.
- LangSmith Engine: This feature automates the ADLC by clustering traces into issues, opening pull requests against a GitHub repository, and streamlining the creation of evaluation assertions.

## Practical implications

- Build engineers can implement robust quality gates for LLM agents by integrating LangSmith's evaluation and experiment features into CI/CD pipelines.
- The platform provides detailed observability, allowing teams to move beyond simple output validation and diagnose root causes related to tool selection, function calling, and model interaction failures.
- Production monitoring capabilities enable proactive alerting on performance degradation (e.g., latency spikes or score drops) in live agent traffic.

## Topics

LLM Agents, Agent Observability, Tracing, MLOps, Build Engineering, LangChain, CI/CD, LangSmith Essentials course, Sign up for LangSmith, LangSmith Docs

Source: https://www.youtube.com/watch?v=KPcEhLoh-Wc