# How to Build an AI Code Review Agent - Daksh Gupta, Greptile

## Executive summary

Daksh Gupta, co-founder of Greptile, outlines the advanced steps required to build an effective AI code review agent, moving far beyond simple sandbox-plus-webhook setups. The presentation details six key improvements: optimizing prompts to focus on specific bug types and skip nitpicks; utilizing a dataset of historical PRs; implementing model inversion (using different LLMs for generation vs. review); integrating deep codebase context (including docs and tickets); performing autonomous, dependency-rich testing; and optimizing inference costs by routing subtasks to cheaper models. Implementing these fixes can significantly improve bug detection rates and reduce operational costs.

## Key takeaways

- Initial Setup Limitations (The Simple Version): A basic setup (sandbox + agent + webhook) is insufficient because it will miss bugs, lack codebase context, be expensive ($3–$25 per PR), and generate excessive nitpicky comments. Bugs are described as 'fractally infinite,' meaning simple reviews will always miss critical issues.
- Fix 1: Prompt Optimization: Prompts must be optimized to specifically look for known bug types (derived from internal data science on past outages) and explicitly instruct the agent to skip nitpicks. The prompt must guide the agent to focus on catastrophic problems, not minor stylistic complaints.
- Fix 2: Historical Dataset and Model Inversion: To improve confidence, the agent must be run against a real-time updating dataset of historical PRs that caused outages. Furthermore, 'model inversion' is critical: if the code was written by Model A (e.g., Claude), it should be reviewed by Model B (e.g., GPT) to catch different types of bugs. Using different models can increase the catch rate by 10% in terms of incidents.
- Fix 3: Codebase Context and Living Wiki: Relying solely on `grep` is imprecise and expensive. A superior method is building an internal knowledge base that indexes the entire codebase, including authentication flows and file relationships. The most effective approach is creating a 'living wiki' that disintermediates context from sources like Jira, Linear, Confluence, and Notion, providing a 'world model' of the company's operations.
- Fix 4: Autonomous Testing and Cost Efficiency: For comprehensive testing, agents must operate in dependency-rich sandboxes (configurable virtual machines) and use computer vision to perform end-to-end QA, catching issues like UI elements hidden on specific devices (e.g., iPhones). To manage costs, the process must use a mix of models: frontier models for critical tasks (e.g., bug analysis) and smaller, cheaper models for the long tail of subtasks (e.g., file reads).

## Technical details

- Codebase Indexing: The goal is to move beyond imprecise tools like `grep` by creating an internal knowledge base that catalogs and indexes the entire codebase, providing flow diagrams and file relationships to improve context awareness.
- Model Inversion: To prevent the review agent from failing in the same ways the generating model does, use a different LLM for review than was used for code generation (e.g., use GPT to review code written by Claude).
- Autonomous Testing Environment: Requires building dependency-rich sandboxes (configurable VMs) and documenting all application flows. The agent uses computer vision to test end-to-end, identifying issues like UI elements that are technically present but functionally hidden.
- Cost Optimization: The review process should be segmented into 20+ subtasks, each assigned a specific model. Frontier models are reserved for the most critical tasks (e.g., bug analysis), while smaller, cheaper models handle the bulk of the work (e.g., finding context or performing file reads).

## Practical implications

- The review process must be viewed as a multi-stage pipeline, not a single API call, to manage cost and effectiveness.
- Focusing on the *intent* (docs, tickets) and the *system* (codebase context) is more valuable than simply reviewing the code diff.
- The cost of running a review agent can be drastically reduced by intelligently routing subtasks to the most appropriate and cheapest model.
- The development of a 'living wiki' that synthesizes information from multiple enterprise tools is crucial for providing deep context to the AI agent.

## Topics

AI Engineering, Code Review, LLMs, Build Automation, Software Testing, Greptile, GitHub/GitLab, Firecracker, OpenCode or Pi, Jira, Linear, Confluence, Notion

Source: https://www.youtube.com/watch?v=4UtrRVtUN_g