# How to kill your SaaS: what we learned replacing a $43K/yr tool with AI agents

## Executive summary

The video details the 'Kill My SaaS' challenge, where the speakers attempted to replace a $43,000/year event management SaaS tool using open-source AI agents. The experiment involved 72 teams and demonstrated that while AI agents (using mixed models like Claude Code and Codex) can successfully clone complex, mid-tier business software, the true difficulty lies in ownership, maintenance, and ensuring seamless integration across multiple, interconnected portals. The speakers conclude that the trade-off shifts from renting to owning, provided the team has the necessary operational expertise.

## Key takeaways

- The Cost and Inflexibility of SaaS: The target software was quoted at $43,000 per year for four events. Traditional SaaS vendors often require 3 to 6 months, or never, to implement a change request, and charge for unused features.
- The Importance of Domain Expertise: The evaluation was conducted by the speakers' non-technical events team, emphasizing that real-world usability and domain knowledge are more critical than technical implementation or the specific stack used.
- Common Technical Flaws in Clones: Three common issues observed in submissions were: 1) Lack of flexible, multi-role portals (e.g., separate portals for CFP, sponsors, and invited speakers); 2) Poor implementation of complex UI elements like drag-and-drop scheduling; and 3) Failure to implement core flows (like adding a talk) without using the designated form.
- The Value of Preliminary Planning: The top finalists emphasized that the most valuable work was done *before* prompting, involving extensive research, detailed spec writing, data schema planning, and pseudo-coding, rather than just running a single prompt.
- The Ownership Trade-Off: While AI agents make building and customizing software more feasible, the core challenge remains that the owner is responsible for all bug fixes and maintenance, requiring a high level of accountability.

## Technical details

- AI Agent Workflow & Modeling: Builders utilized mixed model families, often combining Claude Code and Codex. Three successful patterns were identified: 1) A smart model decides the task, and a cheaper model executes the code; 2) One model builds, and a different model checks/tests (often the one better at driving a browser); and 3) A model approves its own work (on-policy vs. off-policy verification).
- Development Process: The process involved agents running for days, resulting in a medium public repo of 91,000 lines of code. The speakers recommend running an automated evaluation (e.g., Claude logging in and clicking through 20 scenarios) to help teams 'hill climb' and identify weak spots.
- Technology Stack Examples: Finalists used modern stacks like Convex (for real-time database) and X.js. The speakers noted that the prompt was often the final step, following deep research into APIs and documentation.

## Practical implications

- When building complex business software, prioritize deep domain research and detailed specification writing (pseudo-code, data schema) over simply running a single prompt.
- For agent-based development, consider implementing mixed model families (e.g., Claude Code and Codex) and defining clear roles (builder vs. checker) to improve reliability.
- The most critical phase of development is not coding, but defining the user experience (UX) and ensuring all components (portals, scheduling, sign-in) communicate seamlessly.
- When evaluating AI-built software, test the system's ability to handle real-world, non-linear user flows, rather than just following the initial spec.

## Topics

AI Agents, SaaS Development, Build Engineering, Event Technology, LLM Workflow, OpenRostrum, Conferencer, Trackstage, Valentin's agent guardrails template, Harness Engineering 101

Source: https://www.youtube.com/watch?v=zlD-CkLVKtU