# Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler

## Executive summary

Kepler addresses the critical challenge of using Large Language Models (LLMs) for mission-critical financial analysis, where numerical accuracy and auditability are paramount. The core architectural innovation is ensuring that models cite numbers but never write them. The system integrates deterministic code for calculations, a citation registry built from over 30 million SEC documents, and a post-processing filter that strips any figure that cannot be verifiably traced back to a source. This approach moves beyond standard Retrieval-Augmented Generation (RAG) to provide a guaranteed, auditable ground truth for financial data.

## Key takeaways

- Deterministic Architecture for Finance: The system architecture ensures that the LLM's role is limited to citing sources, while all mathematical operations are handled by deterministic, human-written code. A post-processing filter enforces this by stripping any unverified figures, making the platform structurally incapable of producing hallucinated numbers.
- Structured Fact Annotation: To ensure traceability, every fact is annotated with metadata, including its value, unit, scale, and period. This allows the system to correctly contextualize numbers (e.g., identifying if a 'billion' refers to 2023 revenue or a 2025 filing).
- Advanced Retrieval and Security: Kepler utilizes a hybrid search approach (combining BM25 and zero-entropy ANN embeddings) on TurboPuffer. Security is maintained through tenant isolation, where every document hit is reauthorized in Postgres to prevent data leakage across different client environments.

## Technical details

- Citation Registry and Data Ingestion: The platform processes massive corpora of SEC data (over 30 million documents) to build a citation registry. This registry registers deterministic numerical facts, allowing users to click on any number and trace it back to its core source document.
- Hybrid Search and Re-ranking: Information retrieval is achieved by combining BM25 and zero-entropy embeddings (1280 dimensions) within a namespace. The process involves reciprocal re-rankings and a cross-encoder to deconflict and prioritize the most trusted sources across multiple documents (e.g., 10-K vs. earnings call transcripts).
- Model Limitations and Guarantees: The system provides a 'potent guarantee' by assigning unique identifiers to every fact. The platform refuses citations that cannot be verified, resulting in a high rate of rejected citations (e.g., 10,981 citations refused in internal evaluations), confirming that regular RAG platforms are insufficient for numerical accuracy.

## Practical implications

- The platform provides a verifiable, auditable ground truth for financial research, eliminating the risk of LLM hallucination in quantitative analysis.
- It enables the creation of complex financial models (like DCFs) that require traceable, accurate data points, moving beyond simple semantic search.
- The architecture is designed to handle both public filings (10Ks) and private, internal corporate data (SharePoint, Box, One Drive) within a secure, isolated namespace.

## Topics

Financial Technology, Large Language Models (LLMs), Data Architecture, Information Retrieval, SEC filings (10K, 10Q), TurboPuffer, Gemini Force

Source: https://www.youtube.com/watch?v=dZgA6btnb_s