The Video Signal

Topic

Agentic Workflows

All digests tagged Agentic Workflows

· 14:05

How Forward Deployed Engineering is done at Ramp — Leo Mehr

The talk outlines two core principles for Forward Deployed Engineering (FDE): always scoping and scaling with tokens. FDE teams must resist the urge to simply say 'yes' to every request; instead, they must rigorously scope requirements by asking context-gathering questions to ensure the right product is built. To scale this function, the entire FDE pipeline—from initial intake to spec writing—must be automated using AI agents (scaling with tokens). This requires building robust agent harnesses and implementing validation loops using evals, rubrics, and human feedback.

Key takeaways

  1. Always Scope First 5:16

    FDEs must pause before accepting urgent requests (e.g., SAP S/4HANA integrations) to gather context and determine what is truly necessary for the customer's success, rather than simply fulfilling the immediate ask.

  2. Scale with Tokens 8:24

    The future of FDE requires automating every stage of the request lifecycle—from gathering context to writing a spec—by replacing manual work with AI agents and model capabilities.

  3. Hybrid Approach is Key

    Effective modern FDE needs both human judgment (for taste and final validation) and agent volume/automation to handle the sheer scale of incoming requests.

Watch on YouTube Full article

· 26:42

Tammuz Dubnov - When Our PM Started Writing Code: What Merge Rate Taught Us About AI Adoption - AI N

The shift toward 'AI native' organizations means that the primary bottleneck is no longer code generation speed but organizational alignment and handoff efficiency. The speaker argues that empowering non-technical decision-makers (PMs, Designers) with agentic tools allows them to execute work directly, collapsing gaps that previously required multiple sprints or lengthy coordination. Success in this transition is measured by the 'merge rate'—the percentage of Pull Requests (PRs) opened by non-technical users that successfully land in production.

Key takeaways

  1. Redefining AI Native 2:00

    AI native means that the person who cares about a feature has the authority and ability to execute the work, collapsing the traditional handover gap between PMs/Designers and Developers. The bottleneck shifts from coding speed to decision-making capacity.

  2. The Importance of Merge Rate 6:00

    Merge rate (the percentage of PRs that land in production) is the key metric for measuring if an organization is successfully adapting to AI-driven workflows. A high merge rate indicates trust and quality between non-technical contributions and the dev team.

  3. The Role of Guards and Architecture 11:20

    As organizations move fast, it is critical not to abandon engineering principles or 'guards' (like testing, architectural standards). The agent must be designed to learn the codebase deeply and maintain these constraints.

  4. Measuring Non-Technical Contributions 14:20

    To measure impact, track: 1) Count of PRs opened by non-technical individuals. 2) Merge rate (e.g., an average merge rate of 74% was cited). 3) Percentage of merged PRs that require zero developer intervention.

Watch on YouTube Full article

· 17:57

Loop Engineering from First Principles — Kyle Mistele, HumanLayer

The talk argues that relying solely on large-scale prompts for AI code generation is insufficient for complex, mission-critical systems. The solution lies in adopting 'control loops'—a pattern borrowed from control theory (Sensor $\rightarrow$ Controller $\rightarrow$ Actuator)—to ensure that code changes are small, verifiable, and incremental. This approach minimizes risk by preventing the creation of massive, unreviewable pull requests (PRs) and allows for robust automation within existing CI/CD pipelines.

Key takeaways

  1. The Problem with Large PRs

    Current AI workflows often result in 'blind Ralph loops'—massive, unreadable pull requests (e.g., 40,000 lines) that are too risky for large teams and critical systems. The goal must be to design loops that make code easier to read and verify.

  2. The Control Loop Framework 7:23

    A robust system uses a Sensor (to measure the current state/error), a Controller (to determine the optimal incremental change), and an Actuator (to apply that change). This pattern is ideal for systems where changes must be measured, applied incrementally, and verified.

  3. Implementing Low-Friction Loops

    Loops should be integrated into CI/CD platforms (GitHub Actions, CircleCI) using deterministic workflows. To maintain human oversight without friction, a 'feedback file' tracked in version control and triggered by comments (e.g., `/iterate`) is recommended to guide the agent.

Watch on YouTube Full article

· 32:16

BONUS EPISODE: 76 Malicious AI Skills Were Hiding in Plain Sight

The security landscape for AI coding agents is rapidly evolving, shifting focus from traditional code vulnerabilities to risks introduced by 'Skills' and context management. The presentation details findings from Snyk’s ToxicSkills research, which uncovered 76 malicious skills containing malware or prompt injection vectors in open agent repositories. To mitigate these threats, organizations must adopt secure-by-default practices, implement governance through private skill registries (like Tessl), and utilize advanced scanning tools that analyze natural language context alongside code.

Key takeaways

  1. The Shift to Agent Security 6:38

    AI agents are changing developer workflows from reviewing lines of code to managing a 'team of agents.' This necessitates securing the inputs (context) and artifacts produced by these agents, making agent security paramount.

  2. Malicious Skills Threat 17:55

    Research found that third-party skills can contain purely malicious code or prompt injection attacks. These threats are difficult for traditional scanners to detect because they operate by influencing the agent's context rather than just being visible malware.

  3. Governance and Scanning Solutions 23:58

    Implementing secure skill registries (e.g., Tessl) that integrate security scanning (Snyk) is crucial. This allows organizations to scan skills for potential prompt injection, malware, and versioning issues before deployment.

  4. Secure-by-Default Stack 27:50

    Best practices include scanning third-party skills *before* use and running security scanners on custom-authored skills to identify potential vulnerabilities, such as mishandling or passing secrets in plaintext.

Watch on YouTube Full article

· 19:47

2026 State of AI Engineering — Barr Yaron, Amplify Partners

The state of AI engineering is characterized by rapid maturity and increased complexity. Survey data from 1,048 respondents indicates that while open-weight models augment closed systems, the primary drivers for model choice are quality, agentic capabilities (like tool calling), and cost. Cost has become a 'first-class engineering constraint,' forcing teams to manage usage carefully. Furthermore, agents are rapidly evolving from summarization tools to systems with write access, necessitating robust control layers and sophisticated evaluation (eval) processes.

Key takeaways

  1. AI Experience is Democratizing 0:03

    The AI engineering workforce is maturing quickly; the median new engineer has nearly as much AI experience as a 10-year software veteran, indicating that AI skills are becoming foundational to modern development.

  2. Cost is a Primary Constraint 0:08

    Three out of four respondents report adjusting their AI usage based on cost, establishing 'cost' as a first-class engineering constraint alongside quality and capability.

  3. Agents are Taking Action 0:11

    Agentic workflows have shifted significantly: they are no longer limited to reading or summarizing, but are increasingly taking actions inside systems. Write access for agents has increased dramatically (from 52% to 89%).

  4. Evaluation Remains the Biggest Challenge 0:12

    Across all layers of the stack, 'eval' (evaluation) remains the number one biggest challenge reported by engineers.

Watch on YouTube Full article

· 35:18

Joseph Katsioloudes - Code Security Reinvented: Navigating the era of AI - AI Native DevCon June 26

AI is revolutionizing code security by helping minimize the gap between developers and dedicated security specialists. While AI tools like GitHub Copilot can significantly improve detection and, more critically, the speed of fixing vulnerabilities (the 'fixing problem'), they are not a replacement for human oversight. Responsible adoption requires utilizing structured protocols like Model Context Protocols (MCPs) and implementing agentic workflows within CI/CD pipelines to ensure deterministic, auditable, and context-aware security checks.

Key takeaways

  1. AI's Role in Security Gap Reduction 18:04

    The primary opportunity is using AI to scale scarce security expertise. While traditional static analysis tools (like CodeQL or Semgrep) are excellent for pattern matching, AI excels at modeling complex behavioral and contextual issues that pass standard checks.

  2. AI Limitations: Hallucinations & Non-Determinism 4:00

    The speaker notes that while AI is powerful, it should not be the sole safety net; good security hygiene remains paramount.

  3. Leveraging MCPs and Skills 22:20

    Model Context Protocols (MCPs) allow AI models to access server-side information from trusted sources (e.g., GitHub Security Lab findings), extending the context beyond narrow training data. Combining MCPs with structured 'skills' ensures that automated fixes are auditable, maintainable, and integrated into development processes.

  4. Agentic Workflows for CI/CD 27:00

    Agentic workflows allow security agents to run on a schedule or trigger automatically. They can perform sophisticated tasks, such as running manual security reviews automated by AI (using quantified knowledge from research) and proposing fixes directly within the Pull Request (PR), accelerating the fix rate.

Watch on YouTube Full article

· 2:53:57

📅 ThursdAI - LIVE from AI Engineer Worlds Fair - OpenAI, DeepMind, EXO, Sakana & more friends

This live panel discussion from the AI Engineer World's Fair focuses on the critical shift toward local and open-source AI models. Speakers debated the current state of frontier models (like OpenAI's GPT-5.6) versus decentralized, sovereign AI solutions running on consumer hardware. Key technical topics included model routing (Fugu), agentic workflows using tools like Weights & Biases' Coreweave Ara, and the necessity of local inference to ensure data sovereignty and prevent vendor lock-in.

Key takeaways

  1. The resurgence of Fable 22:40

    Fable is back, marking a significant moment for open models. The discussion highlighted that this trend emphasizes the need for decentralized AI solutions over reliance on single cloud providers.

  2. Local AI and Sovereignty 35:50

    Running large language models (LLMs) locally is presented as crucial for guaranteeing data sovereignty, preventing vendor lock-in, and ensuring continuous operation regardless of cloud provider restrictions.

  3. Model Routing and Orchestration 45:00

    The concept of model routers (like Fugu) was presented as a superior method for achieving high performance, allowing users to dynamically select the best model for specific tasks rather than relying on a single monolithic LLM.

  4. The Agentic Era and Tooling 1:03:20

    Tools like Weights & Biases' Coreweave Ara are emerging to automate the entire AI research loop (auto-research), moving beyond simple chatbots into full agentic co-pilots for ML engineers.

Watch on YouTube Full article

· 1:09:03

Building retrieval harness for enterprise agents

This technical walkthrough details the architecture of an enterprise agentic retrieval harness, moving beyond traditional Retrieval Augmented Generation (RAG) systems. The core argument is that effective data access requires giving agents a comprehensive set of tools—including local file system traversal (GP), semantic search, keyword search, and multimodal analysis—rather than relying on a single indexing method. Key focus areas include managing complex document parsing, ensuring multi-tenancy via custom metadata, and optimizing vector storage for scale and cost.

Key takeaways

  1. Harness over RAG 2:00

    The most capable approach is not to select a single retrieval method (e.g., pure semantic search or pure GP access) but to build an agentic harness that allows the agent to decide how to combine multiple tools for data exploration and verification.

  2. The Role of File Primitives 4:20

    For document-driven workflows, critical primitives include: (1) Retrieval (hybrid search), (2) Directory listing/hierarchy traversal, (3) Grep for exact text matching, (4) Chunk-by-chunk analysis, and (5) Multimodal processing via localized screenshots.

  3. Enterprise Scaling Challenges 6:45

    Scaling to multi-tenant or million-file situations requires specialized indexing pipelines that manage data freshness, security, and permissioning. Using custom metadata attached during the parsing process is crucial for filtering access at the vector storage layer.

Watch on YouTube Full article