# Benchmarking Coding Agents on New vs Legacy Codebases — Denys Linkov, Wisedocs

## Executive summary

The talk analyzes the necessity of undertaking a major codebase refactor—specifically collapsing over ten legacy repositories into a monorepo—despite rapid advancements in AI coding agents. The speaker argues that while modern LLMs (like Opus 4.8) significantly accelerate tasks compared to older models (o3), the business value gained from proactively addressing technical debt and establishing robust development patterns outweighs waiting for perfect AI tooling. Key findings include benchmarking model performance on refactoring tasks, noting that Sonnet 4.6 required one iteration while Opus 4.8 achieved near single-pass completion. The speaker also cautions against relying solely on LLMs to complete complex tasks without proper guardrails and validation.

## Key takeaways

- Refactoring is necessary despite AI progress: The refactor was worthwhile because it improved business metrics (commit velocity, time-to-market) by establishing clean patterns, even if models continue to improve.
- LLM performance benchmarks show rapid improvement: Refactoring tasks that took three hours of back and forth with o3 could be accomplished in roughly one-fifth the time using modern models like Sonnet 4.6 (one iteration) or Opus 4.8 (near single pass).
- Monorepos simplify development flow: Consolidating ten separate repositories into a monorepo allowed almost every developer to contribute, even outside their area of expertise, significantly boosting commit velocity and collaboration.

## Technical details

- AI Model Benchmarking (Refactoring): The speaker benchmarked refactor tasks using o3 vs. modern models: o3 took three hours and resulted in ten major mistakes. Sonnet 4.6 required one extra iteration, while Opus 4.8 achieved the task in essentially one pass. This represents a significant reduction in manual effort and time.
- LLM Limitations (GPT-5.5): When given a complex refactoring goal, GPT 5.5 extra high completed the task quickly (10 minutes 22 seconds) but only wrote scaffolding code (2,000 lines), admitting that it had not added necessary deployment or bootstrap commands.
- Technical Debt and Code Quality: The speaker advises viewing technical debt rigorously: introducing complexity can quickly outrun the potential Return on Investment (ROI). Maintaining code quality is critical, even when adopting AI-native development practices.
- Agentic Workflow Development: Modern tooling allows for building agentic workflows that involve deep research, matching requirements to problem statements, creating sub-agents for criteria evaluation, and finally building Proofs of Concept (PoCs), accelerating processes previously requiring manual Confluence documentation.

## Practical implications

- Prioritize addressing technical debt and refactoring core systems (e.g., moving from multiple repos to a monorepo) to improve long-term maintainability and developer velocity.
- When using LLMs for complex tasks, treat the output as scaffolding; manual validation of deployment commands, bootstrap processes, and missing models is mandatory.
- Adopt structured agentic workflows that include deep research and sub-agent verification steps rather than relying on single, massive prompts to prevent 'AI psychosis.'
- Measure model accuracy at 80% or higher for long-running tasks to ensure reliable compute investment.

## Topics

Monorepo Architecture, Technical Debt, LLM Agents, Code Refactoring, Build Engineering, WiseDocs, Sonnet 4.6, Opus 4.8, GPT 5.5 extra high

Source: https://www.youtube.com/watch?v=7vn4WpqNpck