Topic

HumanLayer

All digests tagged HumanLayer

Dexter Horthy: Why We Stopped Trusting AI to Write the Plan thumbnail

· 56:12

Dexter Horthy: Why We Stopped Trusting AI to Write the Plan

The discussion explores the shift in software development from writing code to managing 'software factories' powered by AI agents. The central thesis is that while AI agents can automate much of the implementation, the primary value shifts to defining and codifying *intent* (specs) and *preferences* (taste). The speaker argues that the process of continuous improvement—building the factory itself—is more critical than the act of reviewing individual code pull requests. Human review, therefore, evolves from checking syntax to verifying high-level architectural intent and system constraints.

Key takeaways

  1. The Spec is the New Code 3:44

    The industry trend is moving toward treating specifications (specs) as the primary, verifiable, and executable artifact. This approach aims to capture the full intent of a feature, which can then be compiled into code, rather than relying on the code itself as the source of truth.

  2. Context Engineering and the 'Dumb Zone' 10:03

    Context engineering is crucial for effective agentic development. Early models exhibited a 'dumb zone' where performance degraded significantly when the context window exceeded a certain token count (e.g., 100,000 tokens), emphasizing the need for intentional context management.

  3. The Value of the Software Factory 28:23

    A 'software factory' is a system that automates the entire development lifecycle (planning, building, reviewing, rolling out). The goal is to shift focus from fixing individual bugs to continuously improving the factory's processes and skills, thereby increasing overall velocity.

  4. The Persistence of Human Review 53:52

    While AI is powerful, the speaker asserts that there will always be 'alpha in reviewing something.' Human review will shift from checking code correctness to verifying high-level architectural decisions, business logic, and unique organizational 'taste' that models cannot inherently replicate.

Watch on YouTube Full article

Loop Engineering from First Principles — Kyle Mistele, HumanLayer thumbnail

· 17:57

Loop Engineering from First Principles — Kyle Mistele, HumanLayer

The talk argues that relying solely on large-scale prompts for AI code generation is insufficient for complex, mission-critical systems. The solution lies in adopting 'control loops'—a pattern borrowed from control theory (Sensor $\rightarrow$ Controller $\rightarrow$ Actuator)—to ensure that code changes are small, verifiable, and incremental. This approach minimizes risk by preventing the creation of massive, unreviewable pull requests (PRs) and allows for robust automation within existing CI/CD pipelines.

Key takeaways

  1. The Problem with Large PRs

    Current AI workflows often result in 'blind Ralph loops'—massive, unreadable pull requests (e.g., 40,000 lines) that are too risky for large teams and critical systems. The goal must be to design loops that make code easier to read and verify.

  2. The Control Loop Framework 7:23

    A robust system uses a Sensor (to measure the current state/error), a Controller (to determine the optimal incremental change), and an Actuator (to apply that change). This pattern is ideal for systems where changes must be measured, applied incrementally, and verified.

  3. Implementing Low-Friction Loops

    Loops should be integrated into CI/CD platforms (GitHub Actions, CircleCI) using deterministic workflows. To maintain human oversight without friction, a 'feedback file' tracked in version control and triggered by comments (e.g., `/iterate`) is recommended to guide the agent.

Watch on YouTube Full article

Harness Engineering is not Enough: Why Software Factories Fail — Dex Horthy, HumanLayer thumbnail

· 19:18

Harness Engineering is not Enough: Why Software Factories Fail — Dex Horthy, HumanLayer

The video argues that current efforts in 'harness engineering' and increasing tokens are insufficient for building reliable AI software factories because they fail to address fundamental model training shortcomings. The core problem is maintaining codebase quality over time (maintainability), which current reward functions do not penalize. To move forward safely, the speaker advocates returning to rigorous human-led upfront planning: Product Review $\rightarrow$ System Architecture $\rightarrow$ Program Design (down to types and call graphs) $\rightarrow$ Vertical Slices.

Key takeaways

  1. The Failure of 'Lights Off' Factories 12:10

    Attempting to run a software factory with no human code review ('lights off') leads to failures, even for advanced agents. The issue is not scale or prompting, but a fundamental model training limitation.

  2. Model Training Flaw (The Maintainability Gap) 17:12

    Current coding models are primarily trained on passing tests and solving one-off problems. Their reward signal does not penalize poor program design or the erosion of codebase maintainability, meaning they get better at passing tests but worse at keeping large systems stable.

  3. The Path Forward: Structured Planning

    To move faster safely, engineers must re-emphasize upfront planning steps: Product Review (desired behavior/mockups), System Architecture (component contracts/data models), Program Design (types and call graphs), and Vertical Slices (implementation order).

Watch on YouTube Full article