Topic

Software Engineering

All digests tagged Software Engineering

Agentic Engineering vs Software Engineering: Beyond Vibe Coding thumbnail

· 10:46

Agentic Engineering vs Software Engineering: Beyond Vibe Coding

Software engineering is undergoing a fundamental shift from writing explicit, deterministic instructions to defining high-level goals and orchestrating autonomous AI agents. Agentic Engineering treats AI systems as collaborators capable of multi-step workflows, requiring the human developer's role to evolve into that of an architect who supervises, constrains, and validates probabilistic outputs rather than manually executing every task.

Key takeaways

  1. The Shift in Effort

    Traditional software engineering requires writing explicit instructions (deterministic logic). Agentic Engineering allows developers to define goals, while AI agents handle the execution, changing where the core engineering effort is applied.

  2. Defining Agentic Engineering 3:42

    Agentic refers to an organization of agents that write code, while the human developer maintains a 'human in the loop' to oversee and validate the output as the multi-agent system iterates through subtasks.

  3. The Coding Spectrum 5:01

    Coding methods exist on a spectrum based on human agency: Traditional SE (full control) $ ightarrow$ AI-assisted coding (snippets/refactoring) $ ightarrow$ Vibe coding (natural language intent) $ ightarrow$ Agentic coding (autonomous planning/execution) $ ightarrow$ Agentic engineering (designing environments for autonomous systems).

  4. Increased Value of Oversight 8:44

    As agentic systems become more autonomous, the value of human oversight increases significantly. Engineers are now responsible not only for writing code but also for ensuring reliability across probabilistic workflows.

Watch on YouTube Full article

What's Next After RLHF? — Diogo Almeida, TypeSafe AI thumbnail

· 18:05

What's Next After RLHF? — Diogo Almeida, TypeSafe AI

Diogo Almeida argues that current Large Language Models (LLMs), particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are fundamentally optimized for 'assistance'—meaning they prioritize pleasing the human user. This optimization leads to overpromising and a lack of reliability in autonomous tasks. The next frontier, he asserts, is not simply better code generation or enhanced chat capabilities, but achieving true automation by optimizing models for verifiable rewards and calibrated decision-making, moving beyond the need for constant human oversight.

Key takeaways

  1. The Limitation of RLHF 12:11

    RLHF trains LLMs to optimize for human preference (engagement), which makes them excellent assistants but poor autonomous agents. The goal is to please the user, not necessarily to execute a task correctly in a background server environment [7:31].

  2. Assistance vs. Automation 5:14

    The core divide in modern AI is between 'assistance' (where the human remains in the loop) and 'automation' (where the system operates autonomously with real stakes). Current models are optimized for the former, making them unreliable for critical business decisions [3:14].

  3. The Path to True Automation

    Future AI must shift its optimization target from human preference to verifiable rewards and calibrated decision-making. This requires redesigning the entire AI stack for reliability, moving beyond current LLM post-training methods like RLHF or even RLVR [15:43].

  4. The Importance of Software Expressibility 17:15

    True automation requires smarter software that is more expressive than current SaaS models. The focus must shift from merely automating the writing of code to solving extremely rote, simple tasks that can be done repeatedly and reliably by a computer [10:35].

Watch on YouTube Full article

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang thumbnail

· 16:33

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang

Emulated focuses on creating high-fidelity training data environments that simulate entire companies and complex infrastructure operations, moving beyond simple code diffs or single-node sandboxes. The core argument is that for AI agents to achieve true autonomy in mission-critical systems (like cloud providers), they must be trained on long-horizon tasks involving distributed cluster failures, resource provisioning across VPCs/subnets, managing cost constraints, and reasoning through real-world operational incidents.

Key takeaways

  1. The Data Gap in AI Agents 3:30

    Current benchmarks (e.g., SweBench Pro, Terminal Bench) limit agents to operating within a codebase, failing to capture the complexity of real-world tasks like PM communication, performance testing, or owning underlying infrastructure over years.

  2. Complexity Requires Full Simulation 6:10

    Real infrastructure work is not a simple code diff; it involves managing failing nodes, stale deprecated components, live traffic serving, and operational blast radius across distributed clusters.

  3. Limitations of Single-Node Sandboxes 10:40

    Standard post-training pipelines often use homogeneous single-node sandboxes. However, real cloud services require simulating resource provisioning (EC2, Cloud Run), VPCs, subnets, and security groups, which necessitates a multi-node sandbox with access to real infrastructure.

Watch on YouTube Full article

The misaligned incentives behind AI coding agents thumbnail

· 50:16

The misaligned incentives behind AI coding agents

The conversation details how AI coding agents, exemplified by Devin, are fundamentally changing software engineering workflows. The industry is moving past simply training larger models and focusing intensely on optimizing cost-efficiency (token spend) and speed. Key technical advancements include the 'sidekick' agent architecture for achieving high price performance, developing advanced evaluation metrics like 'mergeability' via Frontier Code, and implementing proactive automation to shift human engineers into decision-making roles rather than routine coding tasks.

Key takeaways

  1. The Shift from Capability to Efficiency 8:36

    As agents mature, the bottleneck is shifting from model training size to running evaluations and managing costs. The focus has moved toward optimizing speed and cost rather than chasing the absolute best-performing frontier model for every task (5:56).

  2. The Role of Mergeability in Evaluation 14:01

    A critical gap in current evaluation benchmarks is 'mergeability'—determining if code, while technically correct, would improve the overall quality or maintainability of a codebase. Cognition developed Frontier Code to address this (8:41).

  3. Cost Optimization via Sidekick Architecture 35:46

    The 'sidekick' agent architecture allows for running both a high-quality, expensive model and a more price-performant model in parallel. This dual approach enables significant cost savings (up to 35% better price performance) without sacrificing quality (21:46).

  4. Proactive Automation and Productivity Guarantees

    Agents are moving from reactive task completion to proactive automation, handling tasks like triaging messages or suggesting fixes. This capability led Cognition to underwrite a $10 million productivity guarantee based on measuring 'productive engineering output' (46:51).

Watch on YouTube Full article

We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank thumbnail

· 16:24

We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank

Lucas Palma details how Nubank addressed AI skills—which function like supply chain dependencies—being a significant security risk in a regulated financial environment. To mitigate this, they built 'Skill Vector,' a comprehensive security review system that sits between the skill and the internal marketplace. This system vets every skill using a hybrid approach: deterministic checks (e.g., regex for unsafe instructions) followed by LLM analysis to assess context and behavior before granting permissions or allowing deployment.

Key takeaways

  1. Treat AI Skills as Dependencies

    AI skills must be treated like any other software dependency, requiring rigorous vetting and security gates, rather than just configuration files.

  2. Hybrid Security Scanning is Essential 12:17

    The most effective approach combines deterministic checks (for easy risks) with LLM review (for contextual behavior analysis), ensuring comprehensive coverage.

  3. Security Gates Must Be Comprehensive

    Vetting must cover the entire workflow, including local developer testing, CI/CD steps, PR feedback, and integration into a formal vulnerability management program.

Watch on YouTube Full article