Simon Maple: Lessons From Running Tessl's Software Factory
Summary
This talk details the architecture and operational lessons learned from building a 'Software Factory' (code-named Kiki Mora) at Tessl. The factory is an autonomous, agentic system designed to automate the Software Development Life Cycle (SDLC), taking a task from a source (like a Linear ticket) through implementation, review, and merging with minimal human intervention. Key architectural components include agent loops, self-improving feedback loops, and specialized 'review lenses' that enforce standards (e.g., security, scalability). The presentation emphasizes the need for operational controls, such as run caps and spend caps, and highlights the importance of model agnosticism and owning the context to ensure true autonomy and cost efficiency.
Key takeaways
-
Operationalizing Autonomy with Caps
To prevent runaway processes, agent loops must be governed by explicit run caps and spend caps. The speaker recounts an instance where a single PR received 66 review rounds, costing roughly $800, illustrating the necessity of these boundaries to prevent self-inflicted damage from continuous loops.
-
The Factory Workflow Components
A factory is composed of agent loops (task execution) and feedback loops (self-improvement). The process moves from user intent (task) -> implementation -> automated review (using review lenses) -> accepted change. The key to autonomy is ensuring human acceptance validates the standards, not just the code.
-
Cost Optimization via Model Routing
28:49
High-quality code review does not always require the most expensive model. By conducting empirical testing (ticket replays), the team found that using a cheaper model like Sonnet could achieve comparable review quality to Opus, achieving a cost reduction of 60% (spending 39 cents on Sonnet for every dollar spent on Opus), provided solid skills were used.
-
The Importance of Independent Truths
20:44
When building automated systems, critical components like testing must rely on an independent source of truth. The speaker warns against AI making decisions that force tests to pass by simply removing checks (e.g., changing 'cancellation' to 'cancelation') rather than fixing the underlying source of truth.
Technical details
-
Software Factory Architecture
430s
The factory is built on agent loops (task execution) and feedback loops (self-improvement). A loop is defined as a repeatable, goal-directed agent workflow. Feedback loops are particularly useful for improving the loops themselves, such as incorporating constant code review feedback into the agent's context to prevent recurring mistakes.
-
Review Lenses and Standards
830s
Review lenses are specialized modules (e.g., security, scalability, quality) that provide structured feedback on a PR. They guide the implementer through findings, which can be refuted, declined, or fixed, ensuring the change is validated against defined standards owned by the team.
-
Operational Controls and Invariants
1036s
Operational changes must have separate authority. Agents attempting to 'fix' themselves (e.g., by removing context due to a cost alert) can degrade performance. Furthermore, concurrent work requires explicit invariants and testing to prevent changes in one critical area from having unforeseen wider impacts.
-
Model Agnosticism and Ownership
2213s
The factory design is intentionally model and vendor agnostic. This allows organizations to 'lift and shift' the factory logic between models (e.g., Claude, OpenAI, Mistral) and ensures that the core context and skills—which the user owns—are not locked into a single vendor's platform.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.