Walmart automation benchmark

Search for milk and verify one product in the cart.

Walmart shows why challenge encounters and final task outcomes need separate counts. The Grok browsing study uses different rules.

One task in the 500-task selection · consumer-0020

Browser Use Cloud

Oct 20, 2026

A Robot or human check cleared during a wait. The cart then showed one milk product.

CAPTCHA report: solved

Success

1/1 task

Oct 20, 2026

The cart showed one milk product. No CAPTCHA appeared.

CAPTCHA report: none

Success

1/1 task

Oct 21–22, 2026

Muse reports this outcome. Original parallel action evidence remains incomplete.

CAPTCHA report: not reported

Success

1/1 task

Oct 23, 2026 snapshot

The saved snapshot contains no started attempt for this task.

CAPTCHA report: not reported

Unstarted

Oct 12–13, 2026

Separate 50-site browsing study. It requires a product or result page, not a verified cart.

CAPTCHA report: not comparable

Uncleared challenge

Each result describes one historical attempt. Different dates, models, and challenge rules limit direct comparison.

A CAPTCHA encounter does not always stop a browser agent. Automatic clearance and final task completion need separate records.

This benchmark tests public browsing and cart verification. It stops before checkout. It does not measure purchases or completed checkout automation.

A request error, login requirement, or browser permission failure does not establish a website bot block.