Anyone who has run browser automation at scale knows the feeling: it works on your machine, it works in CI, and then it falls over in production for reasons that never reproduce. We spent a long time chasing “flaky browsers” before realizing most of the flakiness wasn’t in the browsers at all — it was in how work was routed to them.
The problem isn’t the browser
A single Chrome instance is remarkably reliable. Point a hundred jobs at a pool of them through a naive load balancer, though, and you inherit a different class of failure: sessions pinned to dead workers, retries that stampede, and cost that scales with your worst-behaved job rather than your average one.
What actually helped
- Route on health, not round-robin. A worker that just OOM’d should not get the next job.
- Make session affinity explicit. Replaying a captured session against a different worker is a guaranteed heisenbug.
- Cap the blast radius. One runaway job should never be able to starve the fleet.
This is the thinking behind browser-gateway — a control plane that sits in front of your browser providers and routes, observes, and controls cost across all of them. More on the internals soon.