# How to Build Financial Agents That Work Across the Web - Jay Sahnan & Arik Bird, Browserbase

## Executive summary

The presentation details the use of browser agents to extract sensitive financial data from websites that lack APIs or are protected by anti-bot measures. The solution involves running scaled, reliable browser instances (often requiring forking Chromium) in a data center environment. Key advancements include Stagehand, an SDK that enhances web automation by allowing natural language prompts and optimizing the DOM context, significantly improving reliability and reducing token usage compared to traditional tools like Playwright.

## Key takeaways

- Browser Agents for Hard-to-Reach Data: Browser agents are necessary for any automation task that requires interacting with the web (e.g., logging into portals, fetching receipts) when APIs or Machine-Readable Codes (MRCs) are unavailable. This is critical for financial institutions like Ramp, Bill, and Stripe.
- Scaling Browsers in the Cloud: Running Chromium in a data center requires specialized infrastructure. Browserbase addressed the inherent instability of standard Chromium by forking it, removing automation flags, and ensuring the process remains alive to maintain reliable, scaled operation.
- Stagehand for Adaptive Automation: Stagehand is an SDK that improves upon standard automation frameworks (like Playwright) by accepting natural language prompts. It processes the DOM and accessibility tree through an LLM to determine actions, making the automation resilient to UI changes and optimizing context for cost and speed.

## Technical details

- Browser Agent Architecture: A browser agent must replicate the full browser environment (Chromium/Safari/Dear) to access web content. Scaling this requires replicating the browser's entire fingerprint (OS, fonts, screen size, time zone) reliably in a data center, moving beyond single-laptop setups.
- Chromium Forking and Stability: To ensure reliability at scale, Browserbase forked Chromium, making specific changes such as setting the automation flag to `false` and aligning the identity (e.g., changing 'headless Chrome' to 'Chrome'). They also implemented a mechanism to keep the browser process alive.
- Stagehand Primitives: Stagehand exposes three key primitives for developers: `observe` (finding elements), `act` (performing actions like clicking), and `extract` (converting messy, unstructured web data into structured formats like a Zod schema).

## Practical implications

- Build engineers can use browser agents to automate data ingestion from legacy or highly protected web portals, significantly reducing reliance on manual data entry (e.g., saving 30,000+ manual hours).
- Implementing Stagehand allows for building highly resilient, adaptive automation pipelines that can handle UI changes and complex, multi-step workflows across thousands of different vendor portals.
- The ability to extract unstructured data into defined schemas (Zod) streamlines the process of converting raw web content into usable, structured data for downstream agents.

## Topics

Web Scraping, Browser Automation, Financial Technology (FinTech), Chromium, LLM Agents, Data Ingestion, Browserbase, Stagehand, Playwright

Source: https://www.youtube.com/watch?v=DZ24Xbm9Joc