How to Build Financial Agents That Work Across the Web - Jay Sahnan & Arik Bird, Browserbase
Summary
The presentation details the use of browser agents to extract sensitive financial data from websites that lack APIs or are protected by anti-bot measures. The solution involves running scaled, reliable browser instances (often requiring forking Chromium) in a data center environment. Key advancements include Stagehand, an SDK that enhances web automation by allowing natural language prompts and optimizing the DOM context, significantly improving reliability and reducing token usage compared to traditional tools like Playwright.
Key takeaways
-
Browser Agents for Hard-to-Reach Data
Browser agents are necessary for any automation task that requires interacting with the web (e.g., logging into portals, fetching receipts) when APIs or Machine-Readable Codes (MRCs) are unavailable. This is critical for financial institutions like Ramp, Bill, and Stripe.
-
Scaling Browsers in the Cloud
5:34
Running Chromium in a data center requires specialized infrastructure. Browserbase addressed the inherent instability of standard Chromium by forking it, removing automation flags, and ensuring the process remains alive to maintain reliable, scaled operation.
-
Stagehand for Adaptive Automation
18:28
Stagehand is an SDK that improves upon standard automation frameworks (like Playwright) by accepting natural language prompts. It processes the DOM and accessibility tree through an LLM to determine actions, making the automation resilient to UI changes and optimizing context for cost and speed.
Technical details
-
Browser Agent Architecture
27s
A browser agent must replicate the full browser environment (Chromium/Safari/Dear) to access web content. Scaling this requires replicating the browser's entire fingerprint (OS, fonts, screen size, time zone) reliably in a data center, moving beyond single-laptop setups.
-
Chromium Forking and Stability
334s
To ensure reliability at scale, Browserbase forked Chromium, making specific changes such as setting the automation flag to `false` and aligning the identity (e.g., changing 'headless Chrome' to 'Chrome'). They also implemented a mechanism to keep the browser process alive.
-
Stagehand Primitives
Stagehand exposes three key primitives for developers: `observe` (finding elements), `act` (performing actions like clicking), and `extract` (converting messy, unstructured web data into structured formats like a Zod schema).
Mentioned resources
- Browserbase
- Stagehand
- Playwright
- Chromium
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.