How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh
Summary
The talk details how Kepler built a verifiable AI system for financial services, addressing the core limitation that Large Language Models (LLMs) are inherently non-deterministic probability machines. To achieve reliable work product—such as DCF models or investment memos—the system must augment the LLM with a deterministic substrate. This architecture enforces three tenets: Atomic Provenance (tracking every number's source), Scope Determinism (separating reasoning from computation), and Reconciliation/Derivation Chains (ensuring numerical accuracy through verifiable steps). The goal is to shift AI's edge from content generation to verifiable, traceable output.
Key takeaways
-
LLMs are Probability Machines, Not Deterministic Calculators
4:17
AI models excel at next token prediction (writing) but fail when deterministic accuracy is required, such as arithmetic or pulling specific figures from filings. Using LLMs for verification alone is insufficient because they are non-deterministic.
-
The Need for Verifiability in Finance
7:07
In finance, information must be traceable to its source (provenance). The challenge is moving beyond simple citation (an after-the-fact audit) to true deterministic verification of a number's correctness.
-
The Three Tenets of Verifiable AI
15:26
Kepler’s platform ensures numerical accuracy through three mechanisms: Atomic Provenance (writing references instead of numbers), Scope Determinism (separating the model's reasoning from deterministic computation), and Reconciliation/Derivation Chains (tracking every step to produce a final number).
-
AI Must Be Modeled Like a Portfolio Manager (PM)
20:01
The system must not let the LLM perform computation. Instead, it uses deterministic tools to calculate figures and pull data from structured sources like XBRL or filings, ensuring the model only dictates *what* needs to be computed, not *how*. This is crucial for producing reliable work product.
Technical details
-
Atomic Provenance
The model writes a reference to the number's source rather than writing or manipulating the number itself. Any extracted number must pass a deterministic check; if it cannot be independently verified, it is stripped out.
-
Scope Determinism
The system splits the LLM's function into non-deterministic reasoning (the model) and deterministic computation (external tools). The model decides what to compute, but external code executes the calculation (e.g., pulling data from a PDF or parsing XBRL).
-
Derivation Chains & Reconciliation
To calculate complex ratios (like gross margin) that don't exist in a single filing, the system must track the entire chain of events and data points used. This process allows for the creation of financial models (e.g., DCF) where every number is tied back to its individual source.
Mentioned resources
- Kepler
- Anthropic
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.