# How Bridgewater Turned 100 Million Documents Into an Edge - Sam Green & Aaron Linsky, Bridgewater

## Executive summary

Bridgewater Associates built 'The Vault,' a data platform designed to transform over 100 million unstructured documents from 135 external sources and internal firm data into a structured, first-class asset for investment research. The system addresses the distinct needs of human analysts (requiring high precision and signal-to-noise ratio) and AI agents (requiring massive scale and recall). Key technical advancements include implementing advanced filtering techniques (like Turbo Puffer's native filtering) and developing proactive agent workflows, such as the Investor Workbench, which automatically captures synthesized learning and compounds knowledge back into the corpus.

## Key takeaways

- Human and Agent Data Needs Diverge: Humans require high precision and a low signal-to-noise ratio because they have limited reading capacity (approx. 300 words/minute). Agents, conversely, are less sensitive to precision, scale rapidly, and consume data much faster (approx. 30,000 words/minute). A successful platform must be built to serve both sets of needs separately.
- The Greatest Impact is Unified Tooling: The most significant gains are achieved when a single tool can simultaneously meet the requirements of both human users and autonomous agents, rather than requiring separate systems.
- The Goal is Proactive Agent Participation: The ultimate objective is moving beyond reactive tools (where humans query the system) to building foundations that enable agents to become proactive participants, guiding high-value professionals toward areas needing attention.

## Technical details

- Data Corpus and Governance: The Vault indexes over 100 million documents from 135 external sources and internal memos. Crucially, all data is aligned to an internal ontology, enriched with metadata like countries, concepts, markets, and the specific time period discussed, ensuring systematic study over time.
- Search Performance Benchmarking: Initial search performance was poor (Precision at 10 was around 60%). After implementing 'Agentic Search,' the precision at 10 improved significantly to 90-95%, measured by hand-benchmarking 200 examples.
- Advanced Filtering Techniques: The platform addresses filtering performance by comparing three methods: Post-filtering (fast, but low recall), Pre-filtering (high recall, but slow), and Turbo Puffer's native filtering. The latter provides fast performance similar to post-filtering while maintaining recall close to pre-filtering.
- Scaling Requirements: The system must scale beyond human usage (e.g., 100 queries/hour) to handle agent swarms, which can easily exceed 100 queries per second.

## Practical implications

- The system migrates familiar user habits (e.g., subscribed lists, folders) into the Vault (saved searches, saved collections) to ensure adoption.
- The platform enables new workflows like dynamic topic feeds and allowing users to hand collections over to agents for further action.
- The Investor Workbench provides agents with structured summaries and direct evidence, allowing investors to assess recommendations and automatically capturing the resulting synthesis back into the Vault for compounding knowledge.

## Topics

Unstructured Data Processing, AI Agents, Vector Databases, Information Retrieval, Systematic Research, Data Governance, The Vault, Turbo Puffer, Pocket Analyst, Investor Workbench

Source: https://www.youtube.com/watch?v=CLve4Nx_RwI