Hill-Climbing Skills: Improve Agents Without Changing the Model — Shubhankar Srivastava, Browserbase
Summary
The talk introduces 'hill-climbing skills,' a methodology to significantly improve the reliability and performance of browser agents without requiring changes to the underlying large language model weights. The core concept is the AutoBrowse loop, which pairs a 'Doer' agent (executing browser actions) with a 'Teacher' agent (analyzing traces, screenshots, and logs). The Teacher learns from the Doer's failures and successes, writing these 'lessons' into a persistent `strategy.md` file. This artifact guides subsequent runs, allowing agents to discover complex, site-specific hacks (like using query parameters instead of clicking through forms) and dramatically reducing cost and improving success rates in non-deterministic environments.
Key takeaways
-
The Doer-Teacher Architecture
15:32
The AutoBrowse loop consists of two agents: the Doer, which performs the browsing actions using the `browse` CLI, and the Teacher, which analyzes the Doer's traces and identifies improvements. This collaborative process continues until a successful, converged solution is found. (9:32)
-
Skill Persistence via Strategy Files
15:32
Learnings are persisted in a `strategy.md` file, which acts as a reusable artifact for future agents. This file contains hacks, constraints, and optimal navigation choices, preventing the agent from 'relearning' tasks from scratch. (9:32)
-
Addressing Reliability and Cost
18:50
By automating the discovery of optimal paths (e.g., using a direct URL query parameter instead of clicking through a calendar widget), the system achieves massive savings in cost and turns, demonstrating improvements of up to 16x cheaper on different tasks. (11:30)
-
Verifiable Outcomes are Critical
29:10
For the AutoResearch loop to function, two conditions are required: a replayable environment and a verifiable, quantifiable outcome (metric). The difficulty lies in defining a metric when the goal is subjective (e.g., finding the 'best' flight vs. finding 'a' flight). (17:50)
Technical details
-
AutoBrowse Loop Mechanism
1000s
The loop takes a verifiable problem statement and iteratively refines the solution. The Teacher reads the Doer's traces (including screenshots and accessibility tree snapshots) to identify what went wrong or what was inefficient, appending these learnings to the `strategy.md` file. (10:00)
-
Agent Components and Tools
1200s
The system utilizes the `browse` CLI and is built on the Browserbase platform. The output skill is a portable artifact (a markdown file) that can be used by any agent or SDK, not just within the Browserbase ecosystem. (12:00)
-
Security and Guardrails
2130s
To prevent prompt injection and malicious actions, the system employs multiple layers of defense: infrastructure-level controls (network firewalls, safe mode), skill-level guardrails, and running generated skills against a security reviewer agent. (21:30)
-
WebMCP (Web Component Programmability)
2230s
WebMCP is described as a concept that makes websites programmable, acting like an API but within the browser itself. The speaker notes that Browserbase has recently shipped support for this technology. (22:30)
Mentioned resources
- Browserbase
- AutoBrowse Loop
- skills.sh
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.