The Video Signal

Topic

Data Architecture

All digests tagged Data Architecture

How to Design a Data Agent People Can Verify thumbnail

· 6:47

How to Design a Data Agent People Can Verify

The video argues that current internal 'data agents' (AI chatbots that query databases to answer business questions) represent a significant design flaw if they only provide a final answer without showing their work. To build trustworthy AI products, the design must prioritize verifiability, allowing domain experts to trace the data's provenance, review the underlying logic (e.g., SQL queries, definitions, intermediate calculations), and confirm the result's accuracy. This shift requires adopting design patterns similar to literate programming and notebooks.

Key takeaways

  1. The Flaw in Current Data Agents 2:05

    Simply receiving an answer (e.g., 'net revenue is X') from a data agent is insufficient because the user cannot verify its correctness. The lack of transparency makes the output untrustworthy.

  2. The Need for Provenance and Traceability 2:30

    A proper data agent must show the 'working'—the path taken to reach the number. This includes the underlying SQL query, the definitions used (e.g., what constitutes 'net revenue'), the filters applied, and the specific calendar period used.

  3. Designing for Evaluation (Evals) 3:50

    The product design should mimic a data scientist's thought process, utilizing notebooks or semantic layers to display intermediate calculations and the source of data. This makes the product inherently easier to evaluate and debug.

  4. Benefits of Verifiable Design 4:40

    Designing for verifiability not only builds user confidence but also provides rich data signals, making it easier to automatically classify errors and collect better feedback for model improvement.

Watch on YouTube Full article

AI Agents Aren't the Revolution. They're the Catalyst! thumbnail

· 10:14

AI Agents Aren't the Revolution. They're the Catalyst!

While AI agents are widely discussed, the speaker argues that they are not the revolution, but rather a powerful catalyst accelerating fundamental, long-lasting improvements across the technology ecosystem. The most significant shifts are occurring in data modernization, system interoperability, API standardization, and the democratization of technical expertise, which are foundational improvements that will persist regardless of the future of agents.

Key takeaways

  1. AI Agents as Catalysts

    The lasting impact of AI is not the agents themselves, but the systemic improvements they force, including data unification, system design modernization, and enhanced digital literacy.

  2. Data Layer Modernization

    AI adoption forces organizations to overhaul data foundations, making data more accessible, searchable, understandable, and reusable by breaking down silos.

  3. System Interoperability and APIs 5:09

    Agents expose system weaknesses, driving the need for APIs that are predictable, well-documented, and machine-readable. This accelerates the shift toward seamless, cross-system connectivity (e.g., MCP, A2A).

  4. Outcome-Based Thinking

    The rise of powerful agentic coding IDEs lowers the barrier to entry, shifting the focus from 'how' (implementation) to 'why' (defining goals and solving high-level problems).

Watch on YouTube Full article

Why Does AI Need Access to the Web? thumbnail

· 19:03

Why Does AI Need Access to the Web?

Large Language Models (LLMs) are inherently limited because they operate as pre-trained snapshots, meaning their knowledge freezes after training and cannot account for real-time changes in the world. To build reliable AI agents that function in dynamic environments, the system must incorporate a robust Knowledge Layer fed by a specialized Web Data Infrastructure Layer. This infrastructure is necessary to provide fresh, structured, and trustworthy web data at inference time, thereby eliminating 'Garbage In, Garbage Out' (GIGO) errors.

Key takeaways

  1. LLMs are Static Snapshots 0:25

    Pre-trained LLMs only contain knowledge up to their training cutoff date. When released, they cannot account for new products, laws, or daily news events, leading to potential hallucinations [0:25].

  2. AI Agents Require Real-Time Context 1:30

    Unlike humans who can recognize and ignore incorrect information, AI agents act on all received data. If they receive outdated or fabricated information, the consequences can be costly and happen at scale [1:30].

  3. The Solution: Knowledge Layer & Web Data Infrastructure 2:20

    The most impactful leap in AI functionality is connecting models to a live context via a robust knowledge layer. This layer must receive fresh, reliable data from the web through a specialized web data infrastructure [3:25].

Watch on YouTube Full article

Inside Clay's Eval Stack: 300M Agent Runs, One LangSmith Pipeline thumbnail

· 13:01

Inside Clay's Eval Stack: 300M Agent Runs, One LangSmith Pipeline

Clay detailed its agentic stack, featuring Claygent (go-to-market research) and Sculptor (go-to-market engineering workflow orchestration). The discussion emphasized that scaling agents to millions of runs requires a robust evaluation philosophy. Key architectural shifts include adopting a data lake for unified first/third-party data access, implementing comprehensive multi-stage evaluations (from local dev to CI), and leveraging large context models (like Fable) to enable complex, self-iterating agent loops.

Key takeaways

  1. Agentic Systems are Becoming Interfaces

    Clay is evolving into an 'agent interface,' making all parts of the product available via CLI and public API. This creates a flywheel where internal and external agents share the same tools, allowing failures in tool invocation to provide user signal for continuous improvement.

  2. Evals are Non-Negotiable for Scale 3:23

    With billions of runs, comprehensive evaluation suites are critical. Clay's philosophy mandates different levels of testing: low-lift local dev checks, and staging/CI environments that mirror production harnesses.

  3. Data Lake Foundation for Agents

    To handle disparate data sources (LangChain traces, Snowflake, Postgres, ClickHouse), Clay is moving to a data lake architecture. This allows agents to run wild on unified first-party and third-party data while enabling safe shadow builds.

Watch on YouTube Full article

The Rise of CaaS: Context-as-a-Service for Agentic AI — Omer Primor, Bright Data thumbnail

· 22:20

The Rise of CaaS: Context-as-a-Service for Agentic AI — Omer Primor, Bright Data

The video analyzes the shift from viewing web data as a simple source of information to treating it as dynamic 'context' for agentic AI. The speaker argues that Context-as-a-Service (CaaS) vendors are emerging to provide structured knowledge graphs, acting as vertical search engines. Critically, he emphasizes that at scale, the cost killer is not initial volume but the *frequency* of repeated queries. For persistent knowledge work, owning and building a custom data pipeline—even if time-consuming—can eventually become more cost-effective than continually renting context from third-party vendors.

Key takeaways

  1. Context Decay: Data is never a snapshot 0:02

    Web data decays quickly (e.g., social content < 1 day; news/finance ~30 days). Therefore, extracting context must be treated as an ongoing process, not a one-time effort [2:43].

  2. The Rise of CaaS for Agents 0:06

    AI agents require structured knowledge beyond what general search provides. CaaS vendors address this by developing and indexing specialized knowledge graphs (vertical search) across multiple data sources, enabling deep reasoning [6:32].

  3. Frequency is the Cost Killer at Scale 0:12

    When performing repeated due diligence or market research, every query costs money, even if nothing has changed. This recurring cost (frequency) eventually surpasses the initial setup cost of building an owned pipeline [12:32].

  4. The Tipping Point for Ownership 0:15

    There is a tipping point where the cumulative cost of repeated context queries makes it economically viable to build and own the data retrieval pipeline in-house, potentially bypassing middleman costs [15:22].

Watch on YouTube Full article

Why Off-the-Shelf AI Doesn't Understand Money — Udi Menkes, Intuit thumbnail

· 19:50

Why Off-the-Shelf AI Doesn't Understand Money — Udi Menkes, Intuit

Off-the-shelf Large Language Models (LLMs) often provide fluent but inaccurate financial advice—a phenomenon termed the 'fluent bluff'—because they lack real-world experience. The core argument is that mere context is not sufficient; true intelligence requires grounding in verifiable business outcomes. To solve this, Intuit builds systems that transform raw business data (e.g., General Ledger, P&L) into millions of State-Action-Outcome records. These structured datasets are used to train models that predict the actual impact of actions, leading to 'outcome-driven finance' and outperforming frontier LLMs in real-world decision-making.

Key takeaways

  1. The Fluent Bluff: Why Off-the-Shelf Models Fail at Money 4:08

    LLMs trained on general internet data (blogs, books) can give confident advice that is fundamentally wrong for specific business constraints. Examples include recommending a negative cash flow business acquire another property or raising prices when it risks losing all revenue.

  2. The Solution: Grounding in State-Action-Outcome Data

    Instead of relying on general knowledge, the system must be grounded by creating structured records detailing what a company *did* (Action) given its current financial status (State), and what the verifiable result was (Outcome). This moves advice from theoretical possibility to proven reality.

  3. Measuring Impact: Beyond Simple Difference 15:45

    Determining the true impact of an action requires advanced statistical methods like Conditional Average Treatment Error (CATE) to account for confounding variables, such as a company's natural propensity for success. Failing to do this leads to inaccurate conclusions about causality.

Watch on YouTube Full article

Video Has No Memory. Here's How We Built One. — James Le, TwelveLabs thumbnail

· 20:27

Video Has No Memory. Here's How We Built One. — James Le, TwelveLabs

Video AI systems currently lack 'memory,' meaning they process each query from scratch rather than maintaining a durable understanding of an entire video corpus. The solution proposed is building a dedicated memory layer by treating video not as a bag of frames, but as a spatial temporal volume. This architecture utilizes a Context Graph—a navigable structure connecting entities, appearances, relationships, and metadata across time—to enable deep reasoning, advanced tracking, and multi-source analysis (e.g., analyzing an entire World Cup corpus).

Key takeaways

  1. Video is a Spatial Temporal Volume 0:50

    Instead of treating video as a stack of images or text tokens, it must be modeled as a spatial temporal volume that preserves continuity across space, time, visual information, speech, sound, and motion. (0:50)

  2. The Need for a Memory Layer 2:06

    Existing systems suffer from wrong context, weak memory (lacking durable continuity), and insufficient reasoning over complex video data. A dedicated memory layer is required to connect moments across different files or seasons. (2:06)

  3. Context Graph Structure 7:48

    The optimal mental model for video intelligence is a Context Graph, which provides a durable, queryable representation linking time-bounded moments (evidence units), entity appearances, relationships, and corpus-level themes. (7:48)

  4. Core Design Principles 9:04

    To build effective video memory, systems must 'ingest once and reason many times,' store primitives (not just answers), ground every claim to a specific timestamp, let intent shape the memory, and keep the layer composable via APIs. (9:04)

Watch on YouTube Full article