Most browser evals look clean in a sandbox and fall apart on the live web. We wrote a walkthrough with @browserbase on how to write browser evals that hold up on the live web.

Sites update, graders reward faked outputs, and bot detection kill your runs before the agent even