Topic

DevOps Automation

All digests tagged DevOps Automation

Give the Agent a Budget, Not a Token — Sachin Malhotra, Anthropic thumbnail

· 19:53

Give the Agent a Budget, Not a Token — Sachin Malhotra, Anthropic

The presentation argues that granting AI agents 'unbounded power' via simple tokens is dangerous. Instead of narrowing the token scope (a boolean fix), engineers must implement a comprehensive 'budget' system with four dimensions: how much, how fast, what can be undone, and who notices. Key solutions include using asymmetric verbs (prioritizing loud failures), enforcing rate limits on writes, implementing trip wires over static allow lists, and utilizing an 'undo test' to size the overall safety policy.

Key takeaways

  1. Budget vs. Token 7:03

    A token is a boolean (yes/no) scope; a budget is multi-dimensional, considering volume, velocity, reversibility, and observability. The failure was giving the agent unbounded power, not the model itself.

  2. Asymmetric Verbs 10:05

    Prioritize granting agents access to operations that fail loudly (e.g., unskipping a test, which causes CI to go red) and keep critical failure verbs (like skipping a test) reserved for human intervention with an audit trail.

  3. Rate Limits & Trip Wires 13:54

    Implement rate limits on every write operation, ensuring the ceiling refills automatically. Use trip wires (monitoring aggregate behavior) instead of static allow lists, as trip wires adapt to real-world data.

  4. The Undo Test

    This test asks if the agent can autonomously roll back its own changes and what the blast radius would be if it failed. If not, a second key (human involvement) and an audit record are required.

Watch on YouTube Full article

Unlock Agent Autonomy: The Runtime for AI-Native Systems — Tushar Jain, Docker thumbnail

· 22:50

Unlock Agent Autonomy: The Runtime for AI-Native Systems — Tushar Jain, Docker

The talk addresses the critical shift from making AI agents more intelligent to making them safer and more autonomous. The core challenge is that as agents investigate complex issues (like latency spikes), their required access expands at runtime, significantly widening the 'blast radius.' The speaker proposes a new runtime layer designed to manage this complexity by enforcing three pillars: **Containment** (running the agent in an untrusted boundary while controls remain outside), **Scoped Capabilities** (providing only the minimum necessary access for a specific task), and **Intent-Based Access** (determining if the requested action aligns with the original user intent). This runtime must be portable across all environments (local, cloud, VPC) and models.

Key takeaways

  1. The Shift from Intelligence to Safety

    The next major challenge in agent development is not intelligence, but safety. Traditional software had fixed permissions; autonomous agents change their required access at runtime, necessitating a fundamental shift in security architecture.

  2. The Danger of Expanding Scope 5:12

    When an agent investigates a problem (e.g., latency spike), it sequentially requests access to logs, GitHub history, and Slack. Each step expands the trust boundary, leading to a single process with excessive, accumulated permissions.

  3. The Three Pillars of Safe Autonomy 10:24

    A proposed runtime layer must implement: 1) **Containment** (controls outside the agent's boundary); 2) **Scoped Capabilities** (providing granular access per task, not accumulating them); and 3) **Intent-Based Access** (validating if a sudden request—like email access during an incident investigation—is correct or should be escalated).

  4. Portability and Orchestration 22:38

    The runtime must be omnipresent, working across different models (Anthropic, Claude, Open Code), multiple harnesses, and environments (local machine, cloud VPC). The speaker demonstrated that the same secure sandbox can run locally or in the cloud, and these sandboxes can be composed for parallel execution and orchestration.

Watch on YouTube Full article

Prototyping as Leadership: How a CTO Ships with AI Agents — Hursh Agrawal, The Browser Company thumbnail

· 18:18

Prototyping as Leadership: How a CTO Ships with AI Agents — Hursh Agrawal, The Browser Company

The speaker argues that autonomous AI agents have fundamentally changed the role of a leader, transforming 'building' from an extracurricular activity into a core job function. By leveraging overnight development loops, leaders can now prototype features, optimize LLM calls, and train custom models with minimal hands-on time. Success hinges on establishing robust organizational scaffolding, including trustworthy CI, feature flags, and rigorous code hygiene.

Key takeaways

  1. Building is Now Part of the Job

    Due to autonomous coding agents, the manager's schedule can now be used for building. This shift allows leaders to stay current with rapidly changing frontier models and demonstrate capabilities via working prototypes rather than just theoretical discussions.

  2. The Overnight Development Loop 10:56

    A core workflow involves a 'co-worker agent' gathering context (from Slack, Jira, Notion) into a comprehensive prompt. This prompt is then handed to a coding agent overnight (4–8 hours), resulting in a report and a functional package ready for review the next morning.

  3. Judgment Remains Human 7:05

    While modern models excel at execution, they are not yet reliable at judgment. Leaders must provide high-level context and strategic direction to guide the agents effectively.

Watch on YouTube Full article

Realtime multiplayer, automation, and you! — Idan Gazit, GitHub thumbnail

· 21:41

Realtime multiplayer, automation, and you! — Idan Gazit, GitHub

The talk introduces two prototypes for future software development: Agentic Workflows and ACE. Agentic Workflows automates complex tasks like dependency upgrades (e.g., Astro 5 to Astro 7) by interpreting plain English instructions into a structured playbook, which is then executed as an action workflow. Crucially, it emphasizes that robust guardrails are defined deterministically in YAML front matter, not merely through prompting. ACE explores real-time multiplayer development in cloud microVMs, treating the shared surface (like Slack) as the primary interface for surfacing non-code facts and collaborative planning.

Key takeaways

  1. Automation via Plain English Playbooks 5:08

    Agentic workflows translate simple natural language instructions (e.g., a message to a junior developer) into comprehensive playbooks that handle tasks like checking for new releases, reviewing changelogs, applying code changes, and creating pull requests.

  2. Guardrails Must Be Deterministic 6:46

    Effective security requires defining guardrails (permissions, allowed tools, network destinations, safe outputs) deterministically in front matter (YAML), rather than relying on prompt instructions, which are susceptible to injection.

  3. Shifting Development Interface 12:40

    The future of development involves iterating on direction and planning within a shared surface (like Slack/ACE), making the document itself—the 'truth'—a primary artifact, rather than solely relying on code.

  4. AI Augmentation is Not Typing 20:40

    A longitudinal study found that hands-on keyboard typing accounts for only about 5% of a developer's time; AI must therefore help scale up the remaining 95% of work (e.g., system design, planning, and collaboration).

Watch on YouTube Full article

Your Finance Agent's Bottleneck Is You — Ramana Siddanth Emani, Auditoria AI thumbnail

· 13:42

Your Finance Agent's Bottleneck Is You — Ramana Siddanth Emani, Auditoria AI

The primary bottleneck in developing production AI agents, especially for regulated fields like finance, is not model capability or hardware, but the developer's manual workflow. The talk outlines how advanced agent harnesses can automate and multiply human output by implementing primitives such as sub-agents, skills, goals, and loops. By orchestrating these agents across parallel worktrees (e.g., handling multiple Jira tickets simultaneously), developers can drastically reduce time spent on repetitive tasks like testing, building, and deployment, shifting the human role from executor to final verifier.

Key takeaways

  1. The Developer Loop is the Bottleneck

    Shipping production agents fails due to developer velocity limitations. The solution lies in automating the internal developer loop rather than waiting for model or hardware upgrades (0:00, 2:37).

  2. Parallel Task Execution via Sub-Agents 7:02

    Sub-agents allow for parallel processing using isolated 'worktrees'—virtual folders where agents write and execute code independently. This enables handling multiple tasks (e.g., QA bug tickets) simultaneously, far exceeding human capacity (4:22).

  3. Minimal UX for Orchestration 8:38

    Effective agent use requires minimal User Experience (UX), consolidating all project dashboards, logs, Jira tickets, and GitHub PRs into a single pane of glass widget to reduce context switching and manual orchestration effort (5:18).

  4. Achieving Autonomy with Goals and Dreaming

    Future agent development involves combining 'goals' (setting a desired outcome) with automated loops. Furthermore, allowing agents to 'dream' by collecting customer usage patterns can drive self-improvement and feature upgrades autonomously (11:35).

Watch on YouTube Full article

How Forward Deployed Engineering is done at Factory — Eno Reyes thumbnail

· 21:21

How Forward Deployed Engineering is done at Factory — Eno Reyes

Forward Deployed Engineering (FDE) is evolving from traditional professional services consulting into a strategic function where engineers act as 'the tip of the spear' for product development. The core goal is to build an autonomous 'Software Factory' within a customer's environment. This factory automates the entire signal-to-deploy pipeline—from capturing external signals (e.g., bug reports, Slack conversations) through planning, coding, and rigorous validation (SAST, linters, type checkers)—with minimal human intervention. Success hinges on achieving 'agent readiness,' which measures the density of deterministic validation loops in a codebase, enabling massive migrations (e.g., 30-50 million lines of code) autonomously.

Key takeaways

  1. The Shift from Consulting to Product Strategy

    Factory argues that FDE should not be professional services work on behalf of a customer, but rather the stream of information gathered from critical customers. This data is fed back into the product to rapidly improve it, making the deployed engineer an integral part of the product's evolution.

  2. The Software Factory Pipeline 8:23

    This factory concept describes a continuous process where external signals flow in, are prioritized into plans, converted into code changes (the source of truth), pass through automated validation stages (linters, type checkers, SAST tools), and finally deploy. The goal is an uninterrupted signal-to-deploy flow without human intervention.

  3. Data Ownership and Autonomy 10:57

    To ensure scalability and security, the deployed engineer must utilize a model independent agent harness (like Droid) that guarantees the customer owns all traces and data. This allows for highly secure deployments, including air-gapping in sensitive environments like finance or healthcare.

  4. Agent Readiness as a Metric 20:05

    Autonomy is measured by 'agent readiness,' which quantifies the number of deterministic validation loops (e.g., passing linters, type checkers) present in a codebase. A high volume of these loops allows advanced agents to operate on complex tasks for extended periods without human intervention.

Watch on YouTube Full article

Inside the Dark Factory: AI That Ships Code Solo thumbnail

· 58:39

Inside the Dark Factory: AI That Ships Code Solo

The video details Tessl's 'Dark Factory,' an autonomous system that handles a significant portion of their code shipping (65-70% of PRs). This factory uses AI agents and sophisticated verification layers to process tickets from inception (Linear) through to merge. The core architectural shift involves moving engineering focus from writing code to designing robust, automated workflows, emphasizing context management and layered verification to build trust in autonomous systems.

Key takeaways

  1. High Automation Rate 2:24

    Tessl routes 65-70% of its PRs through the Dark Factory. Historically, up to 95% of their codebase has never been reviewed by a human.

  2. Shift in Engineering Focus 17:12

    The role of the engineer shifts from implementing tickets to designing complex workflows, scoping tasks for agents, and encoding organizational 'taste' into verification layers (e.g., whiteboarding conversations).

  3. Trust is Earned, Not Enabled 31:00

    Building trust in the Dark Factory requires continuous effort, including running into failure modes and improving verification layers. Accountability remains with the person who filed the original ticket.

  4. Verification Layers are Key 22:22

    The most critical component is not the coding agent itself, but the layered verification system (Verifiers) that allow engineers to encode fuzzy principles (e.g., 'library should be the single source of truth') into deterministic checks.

Watch on YouTube Full article

Robert Overweg - One Brain, No Filtering - AI Native DevCon June 2026 thumbnail

· 31:25

Robert Overweg - One Brain, No Filtering - AI Native DevCon June 2026

The presentation details a 'One Brain' concept—a centralized, AI-native knowledge management layer designed to eliminate information silos and improve decision-making by weaving together research, client context, and operational data. The system uses an orchestrator (OpenClaw) and structured vaults to allow agents to access and synthesize organizational knowledge in real time, shifting focus from manual file retrieval to idea generation and proactive insights.

Key takeaways

  1. Shift from File Search to Idea Synthesis 5:40

    The core value lies in moving beyond searching for specific files; the system allows users to search for 'ideas' or 'contacts.' Agents can interpret natural language queries (e.g., asking about CI/CD steps) and provide contextually accurate answers based on stored knowledge.

  2. Structured Knowledge Flow 7:10

    Knowledge is categorized into 'company knowledge' (new developments, research wikis) and the 'creation pipeline.' Information must be promoted to a central vault from various sources (e.g., Obsidian notes, meeting transcripts) to gain grounding in reality before being shared widely.

  3. Scaling and Security Challenges 17:32

    While the system is powerful, scaling remains a challenge, particularly regarding data segregation (permissions) across different client or team buckets. The local setup on one person's laptop was initially used for testing, but enterprise rollout requires careful consideration of security boundaries.

Watch on YouTube Full article

May Walter - From Blind Spots to Merged PRs: Runtime Intelligence for Continuous Agentic Performance thumbnail

· 31:16

May Walter - From Blind Spots to Merged PRs: Runtime Intelligence for Continuous Agentic Performance

The talk details implementing a runtime intelligence layer for coding agents to enable continuous performance optimization in mature codebases. The approach moves beyond reactive bug fixing by running sensors in production to capture deep forensic context (e.g., function execution frequency, failure modes). This allows agents to proactively surface high-ROI fixes—such as N+1 queries or missing database indexes—scored by complexity and impact, enabling tech leads to prioritize improvements before sprint planning.

Key takeaways

  1. Automating the Investigation Phase 17:52

    Instead of waiting for performance degradation to become a crisis (a 'leaky bucket' problem), the goal is to automate the investigation phase. This allows teams to identify potential optimizations and their estimated impact/cost without dedicating unpredictable engineering time.

  2. Prioritizing Impact over Task Completion 25:30

    The focus shifts from merely generating pull requests (PRs) to identifying the highest impact, lowest risk changes. The system must provide human-readable context that justifies attention, preventing 'PR fatigue' from agents.

  3. Defining Business Context 22:00

    Effective agentic workflows require more than just code analysis; they need business context. The system must map production activity (endpoints, event consumers) to function-level context to understand the true impact of a potential fix.

Watch on YouTube Full article