The Video Signal

Topic

Cloud Architecture

All digests tagged Cloud Architecture

How Unify cut its AI agent costs 95% in two weeks thumbnail

· 1:08:24

How Unify cut its AI agent costs 95% in two weeks

Unify's agent platform automates go-to-market tasks by giving sales representatives 'an engineer in their back pocket.' The discussion details how Unify achieved a 90-95% cost optimization within two weeks of launch. Key architectural insights include optimizing for prompt caching hit rates, designing subagents as simple function calls, and implementing robust data handling over tabular data using virtualized Pandas-like interfaces. Furthermore, the talk emphasizes advanced evaluation techniques (DQA sets) and stateful, durable cloud execution environments like Monty to ensure scalability and cost control.

Key takeaways

  1. Cost Optimization via Prompt Caching 30:25

    Achieving a high prompt cache hit rate is critical for cost-effectiveness. The limit is approximately 15 requests per second (RPS) within OpenAI's cache, and developers must implement strategies to maximize hits, as providers won't solve this distribution problem for you.

  2. Subagents are Function Calls 25:50

    Architecturally, Unify treats a subagent not as a complex process, but simply as an arbitrary function call. This simplifies the system and allows for efficient execution patterns like mapping over rows.

  3. The Importance of Planning Steps 38:20

    Adding an initial planning step (e.g., using GPT-4) to the agent workflow significantly improves quality and efficiency by forcing the model to scout multiple potential trajectories before executing.

  4. Durable Cloud Execution

    To run agents in a cloud environment without full VMs, using specialized tools like Monty (a Python REPL that suspends) is crucial for maintaining statefulness and strong tenancy.

  5. Adversarial LLM Judging

    When using an LLM as a judge, it must be from a different model family than the original agent model to avoid 'mode collapse' or groupthink, ensuring an adversarial evaluation.

Watch on YouTube Full article

The Background Check You Can't Run on an AI Agent thumbnail

· 47:43

The Background Check You Can't Run on an AI Agent

The increasing autonomy of AI agents introduces a fundamental security challenge: non-determinism. Traditional cloud-era identity systems (relying on shared secrets or basic authentication) are insufficient because they cannot verify the agent's intent or ensure its actions align with human goals. The solution requires shifting focus from simple 'who is acting' to complex 'mission identity,' which defines *what* the agent was assigned to do and provides granular, task-based authorization boundaries (hard boundaries/sandboxing) to prevent catastrophic unintended actions.

Key takeaways

  1. Non-Determinism: The Feature and the Bug 17:03

    The core capability of agents—reasoning and guessing over large data sets, leading to non-deterministic behavior—is simultaneously their greatest feature and their biggest security vulnerability. This necessitates new architectural controls.

  2. Shift from Authentication to Mission Identity 36:56

    The identity problem is evolving beyond verifying a user's existence (authentication) or even their general permissions (authorization). The new requirement is 'Mission Identity,' which defines the agent's purpose, scope, and intended actions over time, enabling accountability for delegated authority.

  3. The Need for Hard Boundaries 22:16

    To manage risk, systems must implement hard boundaries (sandboxing) that prevent catastrophic failures. This is critical because agents lack human judgment and cannot inherently distinguish between 'I know' and 'I don't know.'

  4. The Evolution of Protocols 46:47

    New protocols are emerging to solve this, including Cross App Access (an evolution of OAuth) and a net-new standard called Agent Auth. These aim to move away from long-lived shared secrets.

Watch on YouTube Full article

Build a Document Review App in 3 Hours (Python, React, Azure) thumbnail

· 2:37:48

Build a Document Review App in 3 Hours (Python, React, Azure)

This video provides an end-to-end guide for building a sophisticated document review application designed for invoice and receipt processing. The solution utilizes a modern monorepo architecture, integrating Python (FastAPI) for the backend logic, React for the frontend UI, and Microsoft Azure services (Document Intelligence and Azure OpenAI) for core AI capabilities. The process demonstrates creating a robust, multi-stage pipeline that handles document classification, data extraction, business rule validation, and general ledger account suggestion.

Key takeaways

  1. Hybrid Document Processing Pipeline

    The application uses a hybrid approach by combining specialized pre-trained models (Azure Document Intelligence) for high accuracy/low cost extraction with general LLMs (Azure OpenAI) for complex tasks like classification and suggesting General Ledger accounts. This ensures robustness against edge cases.

  2. Structured Data Modeling

    Pydantic is used extensively to enforce strict schemas on extracted data, transforming raw JSON output into predictable Python objects for reliable business logic application.

  3. Modular Pipeline Design Pattern

    The backend implements a chainable pipeline pattern (Classification -> Extraction -> Validation -> GL Suggestion). This modular design allows individual steps to be easily added, swapped, or removed without breaking the overall workflow.

  4. Enterprise Cloud Setup

    The entire system is designed for enterprise readiness by utilizing Azure services and demonstrating deployment via Azure Container Apps (ACA), emphasizing best practices for production environments.

Watch on YouTube Full article