Topic

CI/CD Pipelines

All digests tagged CI/CD Pipelines

Agents, codebases, and teams — Aditya Khandelwal, Amazon AGI Lab thumbnail

· 16:57

Agents, codebases, and teams — Aditya Khandelwal, Amazon AGI Lab

The adoption of AI agents in software development is presented as primarily a leadership and organizational challenge, not merely an individual contributor (IC) problem. Successful implementation requires systemic changes—specifically 'harness engineering'—to manage codebase complexity and ensure reliable agent performance across teams. Key strategies include implementing progressive disclosure, establishing high-value skills like 'ship it,' and creating self-healing CI/CD pipelines to mitigate inevitable AI 'slop.'

Key takeaways

  1. Agent Adoption is a Leadership Problem 9:52

    The speaker argues that making engineers work well with agents requires organizational buy-in (leadership action) rather than individual effort. Relying on ICs to restructure the codebase alone will fail, leading to uneven productivity and increased review burden for those who are not early adopters.

  2. Symptoms of a Poor Setup 7:18

    Warning signs that an agent setup is failing include: engineers 'babysitting' agents, the system silently burning context (e.g., blowing through 500k context units), or requiring constant manual intervention for simple tasks.

  3. Harness Engineering Principles 11:49

    Effective agent integration requires three principles: Smart prompt injection (treating the codebase as a single unit to inject context), closing the loop (creating self-healing pipelines to detect and remove 'slop'), and continuous iteration, treating the process like an ongoing organizational improvement effort.

  4. Progressive Disclosure Best Practices 16:57

    To manage context overload, implement strict boundaries. For example, a hard limit of approximately 100 lines is suggested for files like `skill.md` to ensure the agent receives only necessary context in its initial prompt.

Watch on YouTube Full article

Vibes Not Vulns: Securing the Era of AI-Written Software - Mackenzie Jackson thumbnail

· 23:21

Vibes Not Vulns: Securing the Era of AI-Written Software - Mackenzie Jackson

The integration of AI tools into software development introduces novel and complex security failure modes that traditional AppSec pipelines are unprepared for. The talk details how 'vibe coded' applications can ship insecure patterns, focusing heavily on prompt injection vulnerabilities within CI/CD workflows and the evolving risks in open-source supply chains (e.g., dependency hallucination). To mitigate these risks, guardrails must shift from simple code scanning to context-aware validation and strict access control.

Key takeaways

  1. AI Code Vulnerabilities 3:50

    AI systems are not perfect; they introduce vulnerabilities because they make assumptions about business logic. While models improve (especially with 'make sure it's secure' prompts), fundamental flaws like business logic errors remain, meaning AI code cannot be fully trusted yet.

  2. Prompt Injection in CI/CD 11:45

    A critical new vulnerability class is prompt injection, which allows an attacker to bypass system and application guardrails. This was demonstrated by exploiting the Gemini CLI tool within a CI/CD pipeline to achieve Remote Code Execution (RCE) and leak secrets from GitHub repositories.

  3. Supply Chain Risks 20:30

    Traditional vulnerability tracking using CVE numbers is fundamentally broken for modern malware attacks, which can spread rapidly. Furthermore, AI hallucination means package managers may suggest non-existent packages or outdated dependencies.

Watch on YouTube Full article

Agents Write 95% of Our Code. Here's the Catch thumbnail

· 29:43

Agents Write 95% of Our Code. Here's the Catch

As AI agents assume control over an estimated 95% of code production in advanced software factories, traditional code review processes are insufficient. The talk introduces the role of the 'harness engineer,' a new skill set focused on system-level controls: defining invariants, performing deep analytics on agent logs and PR data, and implementing fine-grained risk/operations policies (like auto-merge ladders). This shift requires engineers to move from writing code features to building robust guardrails that ensure consistency and quality across agent-driven pipelines.

Key takeaways

  1. The Paradox of AI Adoption 25:24

    While AI coding tool adoption is high, benchmarks are becoming saturated. Concurrently, the number of reported bugs and incidents is rising, indicating that agents may generate code that lacks maintainability or systemic health (00:15:24).

  2. The Rise of the Harness Engineer 9:34

    Engineering focus must shift from pure feature building to defining and enforcing system invariants. The three critical new skill sets are Systems Thinking, Analytics, and Risk/Operations (00:09:34).

  3. Instruction Following Gap in Skills 8:23

    Tessl's internal skills benchmark revealed that while agents achieved high task completion rates, they only followed approximately 70% of the total instructions defined within a skill (00:08:22).

  4. Systemic Control through Invariants and CI Gates 12:56

    Engineers must identify general principles (invariants)—such as design system rules or desired code structure—and encode them into deterministic checks, verifiers, or CI gates to ensure consistency across the codebase (00:12:56).

Watch on YouTube Full article