LangSmith Preview Builds: Test agent changes before production
Summary
LangSmith Preview Builds extend standard software development workflows to AI agents. By automatically creating a temporary deployment for every GitHub Pull Request (PR), teams can test agent changes and validate functionality in a production-like environment without impacting the main branch. This system ensures that agents are thoroughly tested on preview branches before being merged, while also guaranteeing automatic teardown of the temporary deployment upon PR merge.
Key takeaways
-
Preview Builds for Agents
LangSmith automatically generates a temporary deployment for every pull request, allowing teams to test agent changes safely before they reach production. This brings standard software development workflows (PR testing) to AI agents.
-
Local Testing and Validation
Agents can be tested locally using `langgraph dev`, which automatically opens LangSmith Studio for validation. Changes made in the local project are reflected in the preview deployment.
-
Automated Revisioning
The system supports continuous iteration: every new commit added to a PR triggers a new revision of the preview deployment, allowing for granular testing before final merge.
-
Automatic Cleanup (DevEx)
Once the pull request is successfully merged into the main branch, LangSmith automatically tears down and removes the temporary preview deployment, ensuring clean resource management.
Technical details
-
Enabling Preview Builds
340s
To enable this feature, users must navigate to the deployment settings and activate 'Preview builds.' Settings can be configured to trigger on every PR or based on specific GitHub labels. The system also allows setting an idle TTL (e.g., 72 hours) and max concurrent previews.
-
Continuous Deployment Setup
365s
The deployment must have continuous deployments enabled to automatically create a new revision when pushing to the main branch. Preview builds are then layered on top of this process.
-
Local Development Workflow
Changes (e.g., adding a new tool like 'check return eligibility') are made in the local project, tested using `langgraph dev`, and then committed to a feature branch before creating the PR.
-
Deployment Lifecycle
The process involves: (1) Creating a PR; (2) LangSmith automatically spins up the preview build; (3) Testing in LangSmith Studio; (4) Committing further changes to trigger new revisions; (5) Merging the PR, which triggers automatic teardown of the temporary deployment.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.