# 7 Most Common AI Eval Mistakes (From Teaching 5K+ Engineers & PMs)

## Executive summary

This guide outlines seven critical mistakes teams make when designing AI evaluations (Evals), emphasizing that effective Evals must be grounded in real-world data rather than generic assumptions. Key recommendations include shifting from generic, scored metrics (e.g., 1-5 Likert scales) to specific, binary (pass/fail) outcomes, ensuring domain experts perform data annotation, and rigorously validating LLM judges against human ground truth using proper data splitting techniques.

## Key takeaways

- Mistake #1: Creating Evals without data context: Do not brainstorm potential failures in isolation. Evals must be grounded in actual observed data to define 'what good looks like.' For example, an AI leasing assistant should be evaluated on specific failures, such as handling sales objections, rather than general conversational flow.
- Mistake #2: Using generic, off-the-shelf metrics: Generic metrics (e.g., helpfulness, conciseness score) are non-actionable. Instead, define specific, observable metrics (e.g., 'Did we handle sales objection when a qualifying opportunity was presented?') and aim for binary pass/fail outcomes.
- Mistake #3: Not validating LLM judges against human labels: To trust an LLM judge, segment your data into Train (20%), Dev (40%), and Test (40%). Tune the LLM judge on the Dev set, and measure its generalization capability on the held-out Test set. Focus on measuring Precision and Recall (TPR/TNR), not just accuracy.
- Mistake #4: Not designing the product for Evals success: Design AI products, especially data agents, for the expert user persona. The interface should show intermediate calculations, source citations, and allow step-by-step review, making the system inherently more testable and auditable.
- Mistake #5: Using Likert scales (1-5) with LLM judges: Avoid continuous scoring. Likert scales introduce massive complexity and require extensive human alignment. Focus instead on scoping Evals to a single, specific error and achieving a binary pass/fail outcome.
- Mistake #6: Having the wrong people review the data: Data annotation and review must be performed by domain experts (e.g., lawyers for legal assistants, doctors for medical assistants), not the developers, who lack domain knowledge.
- Mistake #7: Relying too much on automated evals: Automated agents (like coding agents) can miss context-specific errors (e.g., sales objections, markdown formatting, human handoff issues). Manual data review is essential to define the boundaries of 'good' performance.

## Technical details

- Data Splitting for LLM Judges: When building an LLM judge, segment data into Train (20%), Dev (40%), and Test (40%) sets. The LLM judge should be tuned on the Dev set and tested on the held-out Test set to ensure generalization and prevent overfitting.
- Evaluation Metrics: Prioritize binary outcomes (pass/fail) over continuous scores (e.g., 1-5 Likert scale). Metrics should be highly specific, focusing on single, actionable failures (e.g., 'scheduling failure' or 'incorrect email recipient').
- Annotation Tools: To ensure data quality, domain experts should use custom annotation apps (which can be built using tools like Cloud Code or Codex) to review and annotate data, rather than relying on developers.
- Automated Evaluation Tools: The speaker compared leading vendors (Observe.ai, LangSmith, BrainTrust) and found that while they catch obvious issues, they all fail in the same ways, particularly missing context-dependent errors like sales objections.

## Practical implications

- Build Evals that measure specific, high-impact failure modes (e.g., business logic errors) rather than general performance scores.
- Design product interfaces to facilitate evaluation by exposing intermediate steps and source data, making the system auditable.
- Establish a clear process where domain experts are the primary annotators and reviewers of evaluation data.

## Topics

AI Evaluation (Evals), LLM Judging, Data Annotation, Product Design for AI, Build Engineering, Machine Learning Operations (MLOps), AI evals FAQ, Automated eval vendor study

Source: https://www.youtube.com/watch?v=Ime2rOZTVAw