AI Development — The Video Signal

Sep 16, 2026 · 56:50

AWS's Marc Brooker: Specs, Not Code, Are the Hard Part

The core argument is that the hardest part of modern software development is no longer writing code, but defining the specification—a complete, detailed description of what the system should do. Marc Brooker, VP and Distinguished Engineer at AWS, emphasizes that advanced automation, particularly agentic AI, is making the building and maintenance of reliable implementations increasingly automated. This shifts the engineering focus to specification, rigorous testing, and managing complex system failures, such as metastable failures. Key technical advancements include specialized policy languages (like Dogwood) and advanced memory systems that allow agents to learn from historical postmortems and organizational context.

Key takeaways

The Shift to Specification 33:44
The most challenging aspect of software development is creating a complete, detailed specification of what is desired. Once this specification is established, the building and maintenance of reliable implementations are expected to become increasingly automated (00:32:14).
The Importance of Postmortems 11:11
Analyzing thousands of postmortems teaches engineers a 'deep sense of humility' regarding system failures and highlights the need to understand complex failure modes like metastable failures (00:11:23).
Agentic Policy vs. Classic Authorization 20:14
Classic authorization models are insufficient for autonomous agents. Agentic policy requires a specialized language (like Dogwood) to deterministically define what an agent is allowed to do, and in what order, providing necessary boundaries for trust (00:20:14).
Learning from Failure 11:14
Agents can be trained to learn from postmortems by using tools to semantically search historical incidents and build procedural memory, allowing them to anticipate and circumvent past failures (01:14:32).

Watch on YouTube Full article

Sep 14, 2026 · 11:28

How Developers Secure AI-Generated Code: 5 Security Best Practices

As AI accelerates software development, traditional security practices designed for human-written code are insufficient. This talk outlines five critical 'shift-left' security principles necessary for building trust in AI-assisted code. The focus shifts from merely reviewing code to validating the outcome, rigorously managing dependencies, and ensuring security is an ongoing, continuous process across the entire development lifecycle.

Key takeaways

Trust the Outcome, Not Just the Generation 2:20
AI-generated code can compile and pass tests while still harboring unseen security risks (e.g., unauthorized data leaks, failure to fail safe). Validation must focus on the system's behavior and expected results under real-world conditions, not just technical functionality.
Security Must Start During Development 3:25
Integrating security early (shifting left) is crucial. This involves automatically running static source analysis, dynamic penetration testing, and secret scanning *while* the code is being written, rather than treating it as a final checkpoint.
Validate Generated Dependencies 5:30
AI introduces new dependencies (packages, libraries, services) that carry inherent risk. Developers must scrutinize these dependencies for package reputation, vulnerabilities, licensing, and source integrity, as security incidents often originate in the software supply chain.
Consider Intent Over Code Quality 6:50
The solution must address the business intent, not just the technical requirements. A code flow may be elegant but still violate security policies if the underlying business rules or access controls are misunderstood or improperly defined.
Security is an Ongoing Practice 8:00
Security validation must be continuous, extending far beyond initial deployment. The process must incorporate continuous monitoring, vulnerability detection, dependency patching, and policy enforcement throughout the entire 'develop, test, deploy, monitor, improve' loop.

Watch on YouTube Full article

Sep 11, 2026 · 58:39

Trying the new Claude Eval tool

The video provides a technical deep dive into the new Claude Evals tool, designed to evaluate the performance of AI plugins and skills. While the tool automates test case generation and scoring (comparing runs with and without the plugin), the speakers highlight significant limitations. The process is currently highly dependent on manual, pre-collected data (e.g., defining failure modes, providing positive/negative examples) and lacks an integrated, 'in-the-loop' annotation interface, making its real-world application challenging for robust build pipelines.

Key takeaways

Claude Plugin Eval Functionality 2:05
The `Claude plugin eval` tool allows users to evaluate plugins by generating test cases, running the plugin against them, scoring the results, and comparing the scores with and without the plugin's intervention. This process generates a terminal output and an HTML report.
Evaluation Input Requirements 10:10
The tool requires users to define quality standards by providing examples of 'good' and 'bad' outputs, identifying specific failure modes (e.g., overediting, abrupt transitions, jargon), and supplying real prompts or past outputs.
Process Limitations 31:40
The current chat-based interface is criticized for being 'out of the loop.' Effective evaluation requires continuous, in-situ feedback and annotation, which the current tool does not adequately support.

Watch on YouTube Full article

Sep 10, 2026 · 10:14

Baz, Docker & Meta on Verifying Agent Code

The session discusses the critical limitation of current AI coding agents: while they excel at generating code, they often fail at verifying that the code meets complex specifications. Speakers from Baz, Docker, Meta, and Christopher Batey presented advanced strategies—such as using real systems as test oracles, implementing anti-test slop initiatives, and elevating human review to Architectural Decision Records (ADRs)—to ensure reliability and maintain system integrity when integrating AI-generated code.

Key takeaways

Verification is the Hard Part
The primary challenge with coding agents is not writing code, but proving that the generated code accurately implements the required specification. Agents are optimized for feature generation, not full spec extraction or verification.
Specifying Bugs, Not Features
Baz demonstrated that providing explicit evidence of a bug (e.g., a screen recording of an overlapping button) is far more effective than general specs, as agents may fail to replicate specific, subtle behavioral constraints.
Test Oracles Over 100% Coverage
Attempting to achieve 100% test coverage is inefficient. Instead, pointing tests at a real, stable system (a 'test oracle,' like S3) and running a limited set of critical tests can effectively lock down behavior and validate system interactions.
Anti Test Slop Initiative
Meta implemented an anti test slop initiative, using a separate AI tool to judge the quality and necessity of AI-generated tests, thereby reducing CI overhead from useless or redundant tests.
Elevating Human Review to ADRs
When reviewing large pull requests (e.g., 7,000 lines), human review must move earlier into the process. Architectural Decision Records (ADRs) should capture system-level decisions, allowing agents to check implementations against a structured, human-reviewed architectural baseline.

Watch on YouTube Full article

Aug 27, 2026 · 26:11

How Anthropic Builds: Lessons from Labs — Mike Krieger, Anthropic

Mike Krieger discusses the evolution of building with AI models, emphasizing that ambition requires being 'unreasonable' in tool usage and process design. He details advanced development workflows—such as porting large codebases (e.g., Python to TypeScript) over a weekend using LLMs—and outlines modern organizational structures for research labs (like Anthropic's), which prioritize rapid iteration through 'persevere or pivot' cycles rather than rigid project management. Key technical shifts include moving from task delegation to expressing end-state goals, and utilizing advanced features like tagging for multi-agent, proactive code development.

Key takeaways

Be Unreasonable in AI Usage 4:15
The first generation of AI products often constrained access to tools and degrees of freedom. To maximize potential, users must be 'unreasonable' in their prompts, moving beyond simple task delegation to defining the desired end state.
The Importance of Intent Over Code Lines 17:17
In large codebases (e.g., 2,000 line PRs), the bottleneck is no longer review time but human comprehension. The trend is shifting toward sharing 'Claude code artifacts' that explicitly lay out intent, trade-offs, and rationale alongside the code.
Labs Structure for Rapid Iteration 19:13
Anthropic Labs operates on a two-week 'persevere or pivot' cycle. Teams are assembled around 'bets,' and the structure deliberately avoids tying people to specific projects, allowing for quick disbandment and reassembly.
Scaling Requires Pre-Measurement 23:50
Drawing from Instagram's scaling experience, it is crucial to pre-measure everything that might be needed (metrics) and implement thoughtful knobs and feature flags for dynamic runtime configuration.

Watch on YouTube Full article

Aug 21, 2026 · 14:10

The Missing Layer: Design Taste in AI Agents — Hassan El Mghari, Together AI

The talk addresses the critical role of design and User Experience (UX) in making AI applications successful, arguing that polished aesthetics are a major competitive advantage. The speaker details methods to avoid generic 'AI slop' by codifying anti-patterns using tools like Hallmark, leveraging specific themes, and adopting structured development practices such as providing visual references, writing detailed prompts, and iterating with smaller open-source models.

Key takeaways

Use Visual References (Inspiration Vault)
Always provide AI models with a large collection of screenshots or inspiration images to guide the design output. This significantly improves the final product quality.
Structure Prompts and Features
Instead of sending one massive prompt for all features, break down development into smaller steps (one or two features per prompt) and use longer, more specific prompts (e.g., voice notes) to guide the agent.
Iterate with Smaller Models 10:53
Start base builds using larger models (like Codex/Claude Code), but use smaller, faster open-source models (such as GLM 5.2) for iterative refinement and polishing to achieve better results.

Watch on YouTube Full article

Aug 17, 2026 · 16:26

How to Kill the Code Review — Ankit Jain, Aviator

The traditional code review process is failing due to increasing code churn and reliance on AI agents, leading reviewers to only skim results. The speaker argues that code review must shift its focus from verifying line-by-line correctness (semantic accuracy) to preserving crucial non-technical aspects like knowledge sharing, mentorship, and architectural alignment. The proposed solution involves capturing the entire development session's decisions (the 'intent'), converting these into formal acceptance criteria, building an AI Slop Registry of recurring feedback, and generating a verifiable test plan that serves as the new review surface.

Key takeaways

The Code Review Crisis 2:07
Code churn is high (861%), leading to increased bottlenecks at the review stage. Currently, over 30% of changes merge without any review, and even when AI reviews are used, human involvement often reduces to skimming rather than deep reading.
Review Must Preserve Alignment 5:05
Code review's most critical function is not just catching bugs (semantic accuracy), but facilitating knowledge sharing, mentorship, and architectural alignment. This 'alignment' aspect must survive the shift to automated development.
The New Review Surface: Intent vs. Diff 12:38
Instead of reviewing code diffs, the review surface should become a verification of 'intent' and 'evidence.' This is achieved by capturing user decisions from development sessions, converting them into acceptance criteria, and generating a test plan that runs against a live preview.

Watch on YouTube Full article

Aug 9, 2026 · 22:31

Guide, Verify, Solve — Anirban Chatterjee, Sonar

As AI-driven development moves from experimentation to engineering, the primary challenge is 'verification debt'—the gap between the quality provided by LLMs and the high reliability required for critical production systems. The speaker argues that relying solely on human review (which can be compromised, as shown by a Wharton study) or the generating model itself is insufficient. A robust solution requires implementing a zero-trust, multi-layered verification regime within agentic development loops to ensure code quality across syntax, data flow, architecture, and control flow.

Key takeaways

Implement Zero Trust Verification 15:32
Assume all code (human or AI) could come from anywhere. Verification must use a different methodology than the one that wrote the code to prevent inheriting blind spots.
Adopt Multi-Layered Review 15:57
No single technique is sufficient; verification must combine computational review, LLM-driven reasoning, and other approaches to catch all types of issues (quality, security, compliance).
Use ACDC for Agentic Loops 17:12
The proposed framework is the Agent-Centric Development Cycle (ACDC), which includes three phases: Guidance (providing constraints/guardrails), Verification (running independent checks in real time), and Solve (allowing the agent to remediate issues).
Standardize Verification Across Projects 20:46
Organizations should establish a single, consistent verification platform that applies uniformly across all teams, projects, and AI coding tools to eliminate blind spots.

Watch on YouTube Full article

Aug 6, 2026 · 1:26

3 things to know about the new CopilotKit integration for Angular

The new Copilot Genkit integration for Angular significantly upgrades the capability of building in-app AI features. The update allows developers to build agent-powered web applications and deeply integrated, context-aware AI functionalities directly into existing Angular architectures.

Key takeaways

Copilot Genkit Integration for Angular
Copilot Genkit has officially landed for Angular, enabling the creation of agent-powered web applications. Service AI maintains the Angular wrapper, ensuring production-ready reliability and full support.
Context-Aware AI Features
Developers can build deeply integrated features, including native SmartText areas, custom AI chatbots, and autonomous agents that seamlessly integrate with the Angular application state.
Generative UI for Bespoke Interfaces
The Generative UI feature allows an agent to utilize any component catalog within a design system, selecting and presenting the optimal user interface (UI) or requesting necessary inputs dynamically.

Watch on YouTube Full article

Aug 5, 2026 · 15:19

We Scored a Real Snyk Skill Against Anthropic's Rules

This video details a live review process where a Snyk skill (`SKILL.md`) was evaluated using Tessl's `tessl review run` against Anthropic's best practices. The initial score of 87% was successfully improved to 90% by applying fixes, demonstrating how automated tools can enhance skill quality and security. Key focus areas include implementing progressive disclosure to prevent context bloat, improving skill conciseness, and using Snyk's Agent Scan tool to detect vulnerabilities like prompt injection in both first-party and third-party skills.

Key takeaways

Skill Quality Improvement via Automated Review
The review process successfully increased the skill score from 87% to 90% by applying fixes, demonstrating that automated tools can significantly improve adherence to best practices (e.g., Anthropic's guidelines).
Importance of Progressive Disclosure 10:08
Skills should not be overly dense or verbose. Implementing progressive disclosure—breaking large skills into smaller, referenced sub-files—prevents context bloat and ensures the agent only loads necessary information.
Security Scanning with Agent Scan
Snyk's dedicated tool, Agent Scan (available on GitHub), can scan skills for security vulnerabilities, including prompt injection, which is crucial when integrating third-party or user-written skills.

Watch on YouTube Full article

Aug 3, 2026 · 29:43

Agents Write 95% of Our Code. Here's the Catch

As AI agents assume control over an estimated 95% of code production in advanced software factories, traditional code review processes are insufficient. The talk introduces the role of the 'harness engineer,' a new skill set focused on system-level controls: defining invariants, performing deep analytics on agent logs and PR data, and implementing fine-grained risk/operations policies (like auto-merge ladders). This shift requires engineers to move from writing code features to building robust guardrails that ensure consistency and quality across agent-driven pipelines.

Key takeaways

The Paradox of AI Adoption 25:24
While AI coding tool adoption is high, benchmarks are becoming saturated. Concurrently, the number of reported bugs and incidents is rising, indicating that agents may generate code that lacks maintainability or systemic health (00:15:24).
The Rise of the Harness Engineer 9:34
Engineering focus must shift from pure feature building to defining and enforcing system invariants. The three critical new skill sets are Systems Thinking, Analytics, and Risk/Operations (00:09:34).
Instruction Following Gap in Skills 8:23
Tessl's internal skills benchmark revealed that while agents achieved high task completion rates, they only followed approximately 70% of the total instructions defined within a skill (00:08:22).
Systemic Control through Invariants and CI Gates 12:56
Engineers must identify general principles (invariants)—such as design system rules or desired code structure—and encode them into deterministic checks, verifiers, or CI gates to ensure consistency across the codebase (00:12:56).

Watch on YouTube Full article

Jul 17, 2026 · 8:53

Antigravity Arcade: From prompt to game in minutes

The video details 'Antigravity Arcade,' a system that leverages AI skills and workflows to allow users to generate functional web games from simple ideas without writing code. The solution covers both the front-end game generation process—which enforces visual consistency using specialized AI skills (e.g., ensuring retro aesthetics, handling physical input)—and a robust back-end deployment pipeline for hosting these user-generated games on physical arcade cabinets.

Key takeaways

AI-Powered Game Generation 3:30
Antigravity uses modular 'skills' to guide the AI agent, preventing context window bloat and ensuring adherence to organizational best practices (e.g., consistent retro aesthetic, specific key mappings) when generating game code.
Consistent Development Workflow 4:13
The project enforced consistency by using a starting template built with Phaser JS's game engine, which included skills for visual guidelines (black background, CRT shader) and input handling to ensure cross-platform compatibility.
Secure Deployment Pipeline 6:45
A custom deployer CLI tool handles the deployment of user code to physical arcade cabinets. This process uses a Unix setuid model API to securely read sensitive API tokens without exposing them to the end-user or agent, followed by Cloud Run validation and GCS storage.
Automated Game Moderation 8:02
Upon successful upload, an Eventarc trigger spins up a Cloud Run function that validates the game. If approved, Firestore increments a counter, issuing a sequential ID for moderation via a web app.

Watch on YouTube Full article

Jul 11, 2026 · 32:04

Justin Cormack - When Tests Lie: Using Observability to Keep AI Honest - AI Native DevCon June 2026

The talk explores the challenges of using AI to build large-scale, complex distributed systems, exemplified by building an AWS S3 compatible object storage system in Rust. While testing is crucial, relying solely on achieving 100% test coverage is insufficient for complex systems. The speaker emphasizes that observability, robust test articles (like external services), and a 'human-in-the-loop' approach are necessary to enforce correctness, discover edge cases, and manage issues like race conditions and flaky tests in AI-assisted development.

Key takeaways

Observability is Critical for Large Systems 17:47
For complex distributed systems, the public API often doesn't cover all background behaviors. Observability techniques (like tracing) are necessary to infer or observe invisible behaviors that standard APIs cannot expose.
Test Articles Provide Grounding 21:00
Using an existing system, such as AWS S3, as a 'test article' provides a crucial behavioral baseline. This is more valuable than relying on documentation, which may be inaccurate.
Beyond 100% Test Coverage 13:44
Achieving 100% test coverage can lead to writing trivial or unhelpful tests. The focus should instead be on expanding the scope of testing and thinking like a QA professional to find edge cases.
Flaky Tests Must Be Fixed 22:00
The speaker asserts that flaky tests must be fixed immediately, as AI models may incorrectly suggest ignoring them based on training data. Running repeated test suites helps identify these issues.

Watch on YouTube Full article