How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh
The talk details how Kepler built a verifiable AI system for financial services, addressing the core limitation that Large Language Models (LLMs) are inherently non-deterministic probability machines. To achieve reliable work product—such as DCF models or investment memos—the system must augment the LLM with a deterministic substrate. This architecture enforces three tenets: Atomic Provenance (tracking every number's source), Scope Determinism (separating reasoning from computation), and Reconciliation/Derivation Chains (ensuring numerical accuracy through verifiable steps). The goal is to shift AI's edge from content generation to verifiable, traceable output.
Key takeaways
-
LLMs are Probability Machines, Not Deterministic Calculators
4:17
AI models excel at next token prediction (writing) but fail when deterministic accuracy is required, such as arithmetic or pulling specific figures from filings. Using LLMs for verification alone is insufficient because they are non-deterministic.
-
The Need for Verifiability in Finance
7:07
In finance, information must be traceable to its source (provenance). The challenge is moving beyond simple citation (an after-the-fact audit) to true deterministic verification of a number's correctness.
-
The Three Tenets of Verifiable AI
15:26
Kepler’s platform ensures numerical accuracy through three mechanisms: Atomic Provenance (writing references instead of numbers), Scope Determinism (separating the model's reasoning from deterministic computation), and Reconciliation/Derivation Chains (tracking every step to produce a final number).
-
AI Must Be Modeled Like a Portfolio Manager (PM)
20:01
The system must not let the LLM perform computation. Instead, it uses deterministic tools to calculate figures and pull data from structured sources like XBRL or filings, ensuring the model only dictates *what* needs to be computed, not *how*. This is crucial for producing reliable work product.