# Hill-Climbing Skills: Improve Agents Without Changing the Model — Shubhankar Srivastava, Browserbase

## Executive summary

The talk introduces 'hill-climbing skills,' a methodology to significantly improve the reliability and performance of browser agents without requiring changes to the underlying large language model weights. The core concept is the AutoBrowse loop, which pairs a 'Doer' agent (executing browser actions) with a 'Teacher' agent (analyzing traces, screenshots, and logs). The Teacher learns from the Doer's failures and successes, writing these 'lessons' into a persistent `strategy.md` file. This artifact guides subsequent runs, allowing agents to discover complex, site-specific hacks (like using query parameters instead of clicking through forms) and dramatically reducing cost and improving success rates in non-deterministic environments.

## Key takeaways

- The Doer-Teacher Architecture: The AutoBrowse loop consists of two agents: the Doer, which performs the browsing actions using the `browse` CLI, and the Teacher, which analyzes the Doer's traces and identifies improvements. This collaborative process continues until a successful, converged solution is found. (9:32)
- Skill Persistence via Strategy Files: Learnings are persisted in a `strategy.md` file, which acts as a reusable artifact for future agents. This file contains hacks, constraints, and optimal navigation choices, preventing the agent from 'relearning' tasks from scratch. (9:32)
- Addressing Reliability and Cost: By automating the discovery of optimal paths (e.g., using a direct URL query parameter instead of clicking through a calendar widget), the system achieves massive savings in cost and turns, demonstrating improvements of up to 16x cheaper on different tasks. (11:30)
- Verifiable Outcomes are Critical: For the AutoResearch loop to function, two conditions are required: a replayable environment and a verifiable, quantifiable outcome (metric). The difficulty lies in defining a metric when the goal is subjective (e.g., finding the 'best' flight vs. finding 'a' flight). (17:50)

## Technical details

- AutoBrowse Loop Mechanism: The loop takes a verifiable problem statement and iteratively refines the solution. The Teacher reads the Doer's traces (including screenshots and accessibility tree snapshots) to identify what went wrong or what was inefficient, appending these learnings to the `strategy.md` file. (10:00)
- Agent Components and Tools: The system utilizes the `browse` CLI and is built on the Browserbase platform. The output skill is a portable artifact (a markdown file) that can be used by any agent or SDK, not just within the Browserbase ecosystem. (12:00)
- Security and Guardrails: To prevent prompt injection and malicious actions, the system employs multiple layers of defense: infrastructure-level controls (network firewalls, safe mode), skill-level guardrails, and running generated skills against a security reviewer agent. (21:30)
- WebMCP (Web Component Programmability): WebMCP is described as a concept that makes websites programmable, acting like an API but within the browser itself. The speaker notes that Browserbase has recently shipped support for this technology. (22:30)

## Practical implications

- Build engineers can use this methodology to automate the creation of complex, site-specific operational skills, reducing manual reverse-engineering effort for new websites.
- The concept can be applied to non-web domains (e.g., internal ops dashboards, databases) provided a replayable environment and a quantifiable metric can be established.
- Implementing a 'healing loop' where production failures trigger an AutoResearch run allows for continuous skill maintenance as websites change (e.g., anti-bot measures).
- The architecture suggests building a modular 'harness' that can orchestrate multiple specialized skills (e.g., OpenTable skill + Google Reservations skill) to solve a complex task.

## Topics

AI Agents, LLM Engineering, Web Scraping, Build Automation, Prompt Engineering, CI/CD, Browserbase, AutoBrowse Loop, skills.sh

Source: https://www.youtube.com/watch?v=p7gaxmCUiIU