Topic

CI/CD for LLMs

All digests tagged CI/CD for LLMs

Inside Clay's Eval Stack: 300M Agent Runs, One LangSmith Pipeline thumbnail

· 13:01

Inside Clay's Eval Stack: 300M Agent Runs, One LangSmith Pipeline

Clay detailed its agentic stack, featuring Claygent (go-to-market research) and Sculptor (go-to-market engineering workflow orchestration). The discussion emphasized that scaling agents to millions of runs requires a robust evaluation philosophy. Key architectural shifts include adopting a data lake for unified first/third-party data access, implementing comprehensive multi-stage evaluations (from local dev to CI), and leveraging large context models (like Fable) to enable complex, self-iterating agent loops.

Key takeaways

  1. Agentic Systems are Becoming Interfaces

    Clay is evolving into an 'agent interface,' making all parts of the product available via CLI and public API. This creates a flywheel where internal and external agents share the same tools, allowing failures in tool invocation to provide user signal for continuous improvement.

  2. Evals are Non-Negotiable for Scale 3:23

    With billions of runs, comprehensive evaluation suites are critical. Clay's philosophy mandates different levels of testing: low-lift local dev checks, and staging/CI environments that mirror production harnesses.

  3. Data Lake Foundation for Agents

    To handle disparate data sources (LangChain traces, Snowflake, Postgres, ClickHouse), Clay is moving to a data lake architecture. This allows agents to run wild on unified first-party and third-party data while enabling safe shadow builds.

Watch on YouTube Full article