Topic

Software Engineering

All digests tagged Software Engineering

AI Agents Aren't the Revolution. They're the Catalyst! thumbnail

· 10:14

AI Agents Aren't the Revolution. They're the Catalyst!

While AI agents are widely discussed, the speaker argues that they are not the revolution, but rather a powerful catalyst accelerating fundamental, long-lasting improvements across the technology ecosystem. The most significant shifts are occurring in data modernization, system interoperability, API standardization, and the democratization of technical expertise, which are foundational improvements that will persist regardless of the future of agents.

Key takeaways

  1. AI Agents as Catalysts

    The lasting impact of AI is not the agents themselves, but the systemic improvements they force, including data unification, system design modernization, and enhanced digital literacy.

  2. Data Layer Modernization

    AI adoption forces organizations to overhaul data foundations, making data more accessible, searchable, understandable, and reusable by breaking down silos.

  3. System Interoperability and APIs 5:09

    Agents expose system weaknesses, driving the need for APIs that are predictable, well-documented, and machine-readable. This accelerates the shift toward seamless, cross-system connectivity (e.g., MCP, A2A).

  4. Outcome-Based Thinking

    The rise of powerful agentic coding IDEs lowers the barrier to entry, shifting the focus from 'how' (implementation) to 'why' (defining goals and solving high-level problems).

Watch on YouTube Full article

Why We Made Jev — Diogo Almeida, TypeSafe Co-founder & CEO thumbnail

· 2:22:22

Why We Made Jev — Diogo Almeida, TypeSafe Co-founder & CEO

The video introduces Jev, a new class of AI model termed 'System One Models,' designed not for chat or general text completion, but for deep integration into software infrastructure. The core thesis is that AI's next wave of value lies in being a machine-native component optimized for 'intelligence per dollar.' The discussion emphasizes moving away from traditional methods like RLHF and public benchmarks, advocating instead for focusing on reliability, structured state, and decomposing complex tasks into small, measurable, and verifiable decisions.

Key takeaways

  1. System One Models (Jev) 18:40

    Jev is designed to be a large programmable model optimized for intelligence per dollar, intended for code consumption rather than text reply. This represents a shift from pre-trained LLMs (autocomplete) or RHF models (chatbot instruction following).

  2. Critique of Current AI Alignment Methods 26:40

    The speaker critiques RLHF (Reinforcement Learning from Human Feedback) and mode collapse, arguing that these methods are often misaligned with the needs of software engineering. He posits that reliability and robustness are more critical than simple determinism.

  3. The Importance of Decomposition 35:00

    To maximize reliability and power, developers should decompose AI workflows into many small, measurable decisions rather than relying on large, single system messages. This approach allows for programmatic verification and bug fixing.

  4. Anti-Benchmarking Stance 40:00

    The speaker argues that public benchmarks are 'extremely gameable' and antithetical to true intelligence. He stresses that long-term value must be measured by how the model performs within a specific, real-world workflow.

Watch on YouTube Full article

From coding to Knowledge work agents — Karan Vaidya, Composio thumbnail

· 20:42

From coding to Knowledge work agents — Karan Vaidya, Composio

The presentation argues that while autonomous AI agents have excelled in software engineering due to inherent infrastructure support (e.g., Git history, CI/CD), knowledge work agents are currently limited because they lack comparable foundational systems. The speaker identifies six critical primitives—Centralization, History, Context, Verification, Governance, and Reversibility—that must be built into the enterprise layer to enable reliable AI agents for fields like sales and support.

Key takeaways

  1. The Infrastructure Gap

    Coding agents benefit from infrastructure (repo, commit history, tests, CI/CD) that was designed for automation. Knowledge work lacks this surrounding system, causing agents to operate 'blind' when applied outside of code bases.

  2. Centralization is Key 3:55

    Knowledge work data is typically scattered across multiple platforms (e.g., Salesforce, Notion, Gmail, Slack). Agents require a single source of truth—a centralized layer—to pull all necessary threads and connections before they can operate effectively.

  3. The Six Missing Primitives

    To bridge the gap between coding agents and knowledge work agents, six primitives must be built: Centralization (single data source), History (record of past actions), Context (organizational map + style guide), Verification (pre-action checks), Governance (deterministic boundaries/walls), and Reversibility (undo capability).

  4. Failure is Permanent in Knowledge Work 20:00

    Unlike code, where changes can be reverted or walked back, many knowledge work actions (sent emails, wire transfers) are irreversible. This shifts the risk profile, requiring agents to check their work *before* executing any destructive action.

Watch on YouTube Full article

How AI Is Changing Code Reviews & Software Development thumbnail

· 14:09

How AI Is Changing Code Reviews & Software Development

The evolution of software code reviews has progressed from highly structured methods like Fagan inspections and pair programming, through the version control mechanisms of pull requests (consensus reviews), to automated system checks via CI/CD pipelines. The current shift, driven by AI and Large Language Models (LLMs), is moving the focus away from reviewing implementation details (syntax) toward validating business outcomes, requirements fulfillment, and overall intent.

Key takeaways

  1. Historical Shift in Code Review Focus 2:20

    Reviews progressed from line-by-line inspections (Fagan Inspection) to on-the-fly peer review (paired programming), then to versioning/diff analysis via pull requests (consensus reviews).

  2. The Automation Era (CI/CD) 6:15

    This phase introduced system checks for code quality, compliance (internal/external regulations), and vulnerabilities, establishing 'system reviews' where automated systems validate adherence.

  3. AI-Driven Outcome Validation 7:40

    The AI era shifts the focus from implementation details to business outcomes and requirement fulfillment. AI assists by performing broad analysis (writing code, docs, architectures), allowing human engineers to set context, make judgments, and validate against desired business results.

Watch on YouTube Full article

Tribal Dungeons of Global Shipping: AI Agents at Global Scale — Dmitry Buykin, Maersk thumbnail

· 12:02

Tribal Dungeons of Global Shipping: AI Agents at Global Scale — Dmitry Buykin, Maersk

This report details the engineering challenges of operationalizing complex, global business processes using AI agents. The core challenge is translating 'tribal knowledge'—which exists as unstructured Standard Operating Procedures (SOPs) or screenshots—into executable agent workflows. The speaker emphasizes that the system is not the agent loop itself, but rather the surrounding 'refining loop.' This loop involves building robust mechanisms for validation, error handling, and systematic improvement, which requires capturing a vast SOP corpus (estimated 20:1 ratio to runtime).

Key takeaways

  1. The Long Tail Problem

    While happy paths are often automated, the most expensive part of global operations is the 'long tail'—the exceptions and variations that require expert orchestration across multiple incomplete systems.

  2. Agent vs. Expert Knowledge

    SOPs explain what a person sees and clicks (screenshots), but an agent SOP requires complex setup: preconditions, decisions, identifiers, backend calls, validation, recovery, and evidence of successful execution. Experts own the 'what,' and agents own the 'how.'

  3. The Refining Loop is the System 3:34

    The true system is not the agent loop but the surrounding refining loop. This process requires making corrections executable changes, establishing guardrails to prevent unsafe paths, and ensuring behavior is observable.

  4. Scaling Improvement

    Accuracy was not designed upfront; it was earned through systematic effort. The team logged over 100,000 corrections over nine months, using heat maps to prioritize the most beneficial work for experts and engineers.

  5. The Goal of AI Native Operation

    AI native operation is not just about agents in a workflow; it's an adaptive architecture that learns from successful scenarios, aggregates them into composite tools, and allows repeatable rollouts across hundreds of countries.

Watch on YouTube Full article

Like Having an Intern? The Impact of LLMs on Software Engineering Careers -Tom Sellek & Netta Avnoon thumbnail

· 1:01:27

Like Having an Intern? The Impact of LLMs on Software Engineering Careers -Tom Sellek & Netta Avnoon

The talk analyzes the profound impact of LLMs on software engineering expertise, moving beyond simple productivity metrics. While studies show high adoption rates and immediate gains (e.g., up to 5x increase in lines of code), the discussion highlights significant risks: a potential degradation of core skills, reduced critical thinking, and an over-reliance that impairs long-term learning ability. Experts are cautioned that while LLMs feel like 'interns'—super eager but requiring constant review—this dependency may create a dangerous gap in the junior engineer's ability to independently evaluate or debug complex code.

Key takeaways

  1. High Adoption, Low Trust 23:22

    Despite LLMs showing positive sentiment and high perceived quality (some respondents believe generated code is better than average), a significant portion of developers report low trust in the tool's output, leading to cautious practices like only 8% merging code without human review. This discrepancy suggests a disconnect between perceived capability and actual confidence.

  2. Skill Degradation is Quantifiable 35:05

    Research indicates that LLM use can impair fundamental learning abilities. Studies found that using LLMs for tasks led to a quantifiable impairment in understanding and debugging unfamiliar codebases, suggesting the tool doesn't just set a bad example but actively hinders skill acquisition.

  3. The 'Intern' Analogy 26:45

    LLMs are often compared to an inexperienced intern: highly productive, available 24/7, but requiring constant human oversight. The core risk is that junior engineers may not develop the necessary critical judgment skills required to effectively review and correct LLM output.

  4. Cognitive Surrender 38:25

    The process of over-relying on AI can lead to 'cognitive surrender,' where users are willing to follow the machine's incorrect path, even when it is statistically far less likely to be correct. This goes beyond typical automation bias.

Watch on YouTube Full article

How Anthropic uses Claude Code: Agentic Software Engineering at Scale - Daisy Hollman thumbnail

· 1:00:25

How Anthropic uses Claude Code: Agentic Software Engineering at Scale - Daisy Hollman

This talk details the engineering challenges and advanced primitives required for achieving agentic software engineering at scale, particularly within large monorepos. The core thesis is that scaling agents requires sophisticated context management—moving beyond simply increasing model size to implementing abstractions like Skills, Sub-agents, and Hooks. Key focus areas include managing limited context windows (currently around 1 million tokens) by ensuring tools and knowledge are only injected when relevant, thereby enabling complex, long-horizon tasks across massive codebases.

Key takeaways

  1. Context Engineering is the New Discipline 21:45

    As agents become better at writing software, teaching them how to do it—managing context—is becoming the primary discipline of software engineering. This involves carefully selecting and injecting relevant information into the model's limited context window (the 'box').

  2. Scaling Requires Abstraction Primitives 26:45

    To handle massive codebases, simple tool definitions are insufficient. Advanced primitives like Skills (lazy system prompts), Sub-agents (out-of-context processing), and Hooks (event-driven execution) are necessary to ensure that only relevant information is consumed by the model.

  3. The Importance of Feedback Loops 17:15

    The fastest way to improve an agent's performance on a codebase is not necessarily a smarter model, but implementing tighter feedback loops (e.g., type checking, linting) that provide immediate, granular error detection during the tool-use process.

  4. Scaling Workflows Beyond Single Sessions 46:45

    For enterprise use, scaling requires managing multiple concurrent agent sessions. Techniques like using dedicated worktrees and visual cues (e.g., color coding) are essential to manage cognitive load and maintain persistent agent identities across different tasks.

Watch on YouTube Full article

Agentic Engineering vs Software Engineering: Beyond Vibe Coding thumbnail

· 10:46

Agentic Engineering vs Software Engineering: Beyond Vibe Coding

Software engineering is undergoing a fundamental shift from writing explicit, deterministic instructions to defining high-level goals and orchestrating autonomous AI agents. Agentic Engineering treats AI systems as collaborators capable of multi-step workflows, requiring the human developer's role to evolve into that of an architect who supervises, constrains, and validates probabilistic outputs rather than manually executing every task.

Key takeaways

  1. The Shift in Effort

    Traditional software engineering requires writing explicit instructions (deterministic logic). Agentic Engineering allows developers to define goals, while AI agents handle the execution, changing where the core engineering effort is applied.

  2. Defining Agentic Engineering 3:42

    Agentic refers to an organization of agents that write code, while the human developer maintains a 'human in the loop' to oversee and validate the output as the multi-agent system iterates through subtasks.

  3. The Coding Spectrum 5:01

    Coding methods exist on a spectrum based on human agency: Traditional SE (full control) $ ightarrow$ AI-assisted coding (snippets/refactoring) $ ightarrow$ Vibe coding (natural language intent) $ ightarrow$ Agentic coding (autonomous planning/execution) $ ightarrow$ Agentic engineering (designing environments for autonomous systems).

  4. Increased Value of Oversight 8:44

    As agentic systems become more autonomous, the value of human oversight increases significantly. Engineers are now responsible not only for writing code but also for ensuring reliability across probabilistic workflows.

Watch on YouTube Full article

What's Next After RLHF? — Diogo Almeida, TypeSafe AI thumbnail

· 18:05

What's Next After RLHF? — Diogo Almeida, TypeSafe AI

Diogo Almeida argues that current Large Language Models (LLMs), particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are fundamentally optimized for 'assistance'—meaning they prioritize pleasing the human user. This optimization leads to overpromising and a lack of reliability in autonomous tasks. The next frontier, he asserts, is not simply better code generation or enhanced chat capabilities, but achieving true automation by optimizing models for verifiable rewards and calibrated decision-making, moving beyond the need for constant human oversight.

Key takeaways

  1. The Limitation of RLHF 12:11

    RLHF trains LLMs to optimize for human preference (engagement), which makes them excellent assistants but poor autonomous agents. The goal is to please the user, not necessarily to execute a task correctly in a background server environment [7:31].

  2. Assistance vs. Automation 5:14

    The core divide in modern AI is between 'assistance' (where the human remains in the loop) and 'automation' (where the system operates autonomously with real stakes). Current models are optimized for the former, making them unreliable for critical business decisions [3:14].

  3. The Path to True Automation

    Future AI must shift its optimization target from human preference to verifiable rewards and calibrated decision-making. This requires redesigning the entire AI stack for reliability, moving beyond current LLM post-training methods like RLHF or even RLVR [15:43].

  4. The Importance of Software Expressibility 17:15

    True automation requires smarter software that is more expressive than current SaaS models. The focus must shift from merely automating the writing of code to solving extremely rote, simple tasks that can be done repeatedly and reliably by a computer [10:35].

Watch on YouTube Full article

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang thumbnail

· 16:33

Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang

Emulated focuses on creating high-fidelity training data environments that simulate entire companies and complex infrastructure operations, moving beyond simple code diffs or single-node sandboxes. The core argument is that for AI agents to achieve true autonomy in mission-critical systems (like cloud providers), they must be trained on long-horizon tasks involving distributed cluster failures, resource provisioning across VPCs/subnets, managing cost constraints, and reasoning through real-world operational incidents.

Key takeaways

  1. The Data Gap in AI Agents 3:30

    Current benchmarks (e.g., SweBench Pro, Terminal Bench) limit agents to operating within a codebase, failing to capture the complexity of real-world tasks like PM communication, performance testing, or owning underlying infrastructure over years.

  2. Complexity Requires Full Simulation 6:10

    Real infrastructure work is not a simple code diff; it involves managing failing nodes, stale deprecated components, live traffic serving, and operational blast radius across distributed clusters.

  3. Limitations of Single-Node Sandboxes 10:40

    Standard post-training pipelines often use homogeneous single-node sandboxes. However, real cloud services require simulating resource provisioning (EC2, Cloud Run), VPCs, subnets, and security groups, which necessitates a multi-node sandbox with access to real infrastructure.

Watch on YouTube Full article

The misaligned incentives behind AI coding agents thumbnail

· 50:16

The misaligned incentives behind AI coding agents

The conversation details how AI coding agents, exemplified by Devin, are fundamentally changing software engineering workflows. The industry is moving past simply training larger models and focusing intensely on optimizing cost-efficiency (token spend) and speed. Key technical advancements include the 'sidekick' agent architecture for achieving high price performance, developing advanced evaluation metrics like 'mergeability' via Frontier Code, and implementing proactive automation to shift human engineers into decision-making roles rather than routine coding tasks.

Key takeaways

  1. The Shift from Capability to Efficiency 8:36

    As agents mature, the bottleneck is shifting from model training size to running evaluations and managing costs. The focus has moved toward optimizing speed and cost rather than chasing the absolute best-performing frontier model for every task (5:56).

  2. The Role of Mergeability in Evaluation 14:01

    A critical gap in current evaluation benchmarks is 'mergeability'—determining if code, while technically correct, would improve the overall quality or maintainability of a codebase. Cognition developed Frontier Code to address this (8:41).

  3. Cost Optimization via Sidekick Architecture 35:46

    The 'sidekick' agent architecture allows for running both a high-quality, expensive model and a more price-performant model in parallel. This dual approach enables significant cost savings (up to 35% better price performance) without sacrificing quality (21:46).

  4. Proactive Automation and Productivity Guarantees

    Agents are moving from reactive task completion to proactive automation, handling tasks like triaging messages or suggesting fixes. This capability led Cognition to underwrite a $10 million productivity guarantee based on measuring 'productive engineering output' (46:51).

Watch on YouTube Full article

We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank thumbnail

· 16:24

We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank

Lucas Palma details how Nubank addressed AI skills—which function like supply chain dependencies—being a significant security risk in a regulated financial environment. To mitigate this, they built 'Skill Vector,' a comprehensive security review system that sits between the skill and the internal marketplace. This system vets every skill using a hybrid approach: deterministic checks (e.g., regex for unsafe instructions) followed by LLM analysis to assess context and behavior before granting permissions or allowing deployment.

Key takeaways

  1. Treat AI Skills as Dependencies

    AI skills must be treated like any other software dependency, requiring rigorous vetting and security gates, rather than just configuration files.

  2. Hybrid Security Scanning is Essential 12:17

    The most effective approach combines deterministic checks (for easy risks) with LLM review (for contextual behavior analysis), ensuring comprehensive coverage.

  3. Security Gates Must Be Comprehensive

    Vetting must cover the entire workflow, including local developer testing, CI/CD steps, PR feedback, and integration into a formal vulnerability management program.

Watch on YouTube Full article