The Dark Arts of Web Automation: Teaching Agents to Use Websites Like Humans — Corey Gallon, Rexmore
This talk details advanced web automation techniques for AI agents to interact with websites that actively resist bots. The core methodology involves treating the agent as a 'meat bag with a mouse,' leveraging the Chrome DevTools Protocol (CDP) through a Command Line Interface (CLI). Automation is structured around a 'Sense, Act, Verify' loop and progresses up a three-rung 'Meatbag Ladder'—moving from simple API calls to simulated human input (jitter, overshoot)—to defeat modern security measures like Cloudflare Turnstile and reCAPTCHA v2.
Key takeaways
-
CLI vs. MCP for Automation
3:30
While capability is comparable (both achieved tasks 83% of the time), a CLI approach significantly outperforms an MCP server in speed, reuse, and cost. Specifically, a CLI sequence can be programmed once and reused infinitely without model calls, making it faster and up to 75 times cheaper in token cost than repeated model interaction.
-
The Sense-Act-Verify Loop
9:07
Effective automation requires a continuous loop: Sensing (via DOM, accessibility tree, or screenshot), Acting (clicking/typing), and Verifying (checking network traffic or screen state). This process must be repeated until the desired action is confirmed.
-
The Meatbag Ladder Methodology
10:38
Web resistance requires escalating techniques. Rung One uses synthetic JavaScript clicks (API-like). Rung Two employs real CDP input events, which the page recognizes as trusted inputs. Rung Three simulates full human behavior, including jitter and deliberate overshoot, to bypass advanced bot detection.
-
Solving reCAPTCHA v2
18:18
The final challenge (reCAPTCHA v2) is solved by splitting the work: deterministic code handles the rapid, repeatable actions (trusted clicks, iframe piercing), while a specialized AI agent provides the 'eyes and brain' to interpret visual challenges (e.g., identifying specific objects in a grid).