BROWSER USE

- Browser Use Agents: give Browser Use a task and receive completed work. API V4 is current for new integrations.
- Browser Infrastructure: connect your agent or automation to managed browsers through SDK, REST, or CDP. Starts at $0.02/browser-hour.
- Developer tools: Open Source, Browser Harness, SDK, and MCP support the two products above.

[Developer Index](https://browser-use.com/index.md)
[Product Map](https://browser-use.com/llms.txt)
[Full Product Context](https://browser-use.com/llms-full.txt)
[Pricing](https://browser-use.com/pricing.md)
[Cloud Docs](https://docs.browser-use.com/cloud/quickstart)
[Open Source Docs](https://docs.browser-use.com/open-source/introduction)

---

# Expedia scraper skill for AI agents

Source: https://browser-use.com/skills/expedia. 3 files from github.com/browser-use/browser-harness, MIT licensed. Written for browser-harness: code calls its helpers (http_get, js, goto_url). Sites change; these notes describe what worked when last updated.

---

<!-- https://github.com/browser-use/browser-harness/blob/main/agent-workspace/domain-skills/expedia/api.md (updated 2026-10-09) -->

# Expedia — typeahead API, property URLs, affiliate shortlink chain

Field-tested 2026-06 against `www.expedia.mx` (applies to other POS domains —
`www.expedia.com`, etc. — with the same paths).

## Hotel typeahead API (no auth, no cookies)

```
GET https://www.expedia.mx/api/v4/typeahead/{url-encoded query}
    ?client=SearchForm&lob=HOTELS&personalize=false&regiontype=2047
    &dest=true&features=ta_hierarchy&maxresults=10
```

- Works with plain `curl_cffi` chrome impersonation (`impersonate="chrome120"`)
  — no session, no PerimeterX/Akamai challenge observed at low volume.
- Response `sr[]` items of `"@type": "gaiaHotelResult"` carry exactly what you
  need to identify a property: `hotelId` (the `.h{id}.` in property URLs),
  `regionNames.shortName`/`fullName`, and `coordinates.lat/long`.
- Querying `"{hotel name} {city}"` reliably puts the right property first;
  retry with the bare hotel name if the combined query returns nothing.
- The contributor reports a 10× speed improvement over search-page extraction. The report includes no paired timings.

## Property page URLs

- Canonical shape: `https://www.expedia.mx/{slug}.h{hotelId}.Informacion-Hotel`
  (`.Hotel-Information` on the English POS). **Routing is by `.h{id}.` only —
  the slug is cosmetic** and Expedia 301s to the canonical slug (it even
  rewrote `Roma-…` → `Rome-…` in tests).
- Useful query params: `startDate=YYYY-MM-DD`, `endDate=YYYY-MM-DD`,
  `rm1=a{adults}[:c{age}...]` (e.g. `rm1=a2:c8:c5`), `expediaPropertyId={id}`.
  These pre-fill dates and occupancy on the page.
- Search fallback: `https://www.expedia.mx/Hotel-Search?destination={name},
  {city}&startDate=…&endDate=…&adults=N&rooms=1`.

## Trap: property pages 429 datacenter-ish IPs

`GET` on property pages returned HTTP 429 from a residential-but-curl client
while the **typeahead kept returning 200**. If you only need to *construct*
URLs for a human to open, never fetch the property page at all; if you must
read one, do it through the user's real browser session (coordinate clicks),
not `http_get`.

## Affiliate shortlink chain (`expedia.tpx.lv`)

`expedia.tpx.lv/<code>` links are **Travelpayouts** partner shortlinks. The
redirect chain (observed):

```
expedia.tpx.lv/<code>
  → deals.vio.com?label=…-<TP marker>&ofd=book_uri=destination=<expedia URL>
  → r.vio.com (same params)
  → prf.hn/click/camref:…/pubref:…/destination:<expedia URL>   (Partnerize)
  → r.bttn.io (Button attribution)
  → www.expedia.mx/<City>-Hoteles-<Name>.h<id>.Informacion-Hotel?…&affcid=…PHG…
```

- The Travelpayouts marker is recoverable from the `label=`/`utm_content=`
  params on the vio.com hop.
- To mint these programmatically, don't reconstruct the chain — use the
  Travelpayouts links API:
  `POST https://api.travelpayouts.com/links/v1/create` with
  `{"trs": <project id>, "marker": <partner id>, "shorten": true,
  "links": [{"url": "<full expedia URL>", "sub_id": "…"}]}` and the
  account's API token (header `X-Access-Token`). Limits: 100 req/min/marker,
  ≤10 links per request. Returns `partner_url` shortlinks. Full-length brand
  URLs only (no shortened input).
- HEAD requests die mid-chain (some hops 405 HEAD); use GET with
  `allow_redirects` or walk `Location` headers manually.

---

<!-- https://github.com/browser-use/browser-harness/blob/main/agent-workspace/domain-skills/expedia/automation.md (updated 2026-04-28) -->

# Expedia — Browser Automation

Field-tested against expedia.co.in on 2026-04-27 using `browser-harness` CDP
helpers (`goto`, `js`, `click`, `type_text`, `screenshot`).

---

## TL;DR

**Build your search via URL parameters, not the UI.** The date picker, destination
autocomplete, and traveller widgets are fragile—coordinate clicks frequently
dismiss them or mis-target. Encode everything you can into a `goto()` URL, then
use JS clicks only for what the URL can't express (child ages, price filters).

---

## Hotel Search URL Template

```
https://www.expedia.co.in/Hotel-Search?destination={DEST}&startDate={YYYY-MM-DD}&endDate={YYYY-MM-DD}&rooms={N}&adults={N}&children={N}&childrenAges={age1,age2,...}
```

Example — 2 adults, 2 children (ages 5 and 7), Tokyo, June 2026:

```python
goto("https://www.expedia.co.in/Hotel-Search?"
     "destination=Central+Tokyo,+Tokyo+Prefecture&"
     "startDate=2026-06-01&endDate=2026-06-07&"
     "rooms=1&adults=2&children=2&childrenAges=5,7")
```

**Note:** `childrenAges` in the URL may not always populate the age dropdowns on
the results page. Verify with a screenshot and set them via JS if needed.

---

## Date Picker — DO NOT USE

The calendar widget is extremely unreliable with coordinate-based clicks:

- Clicking a date cell frequently **closes the entire picker** instead of
  selecting the date.
- The picker has month-navigation arrows that are tiny targets.
- "Flexible dates" mode has a different DOM structure with pill-shaped month
  selectors that also mis-fire.
- Dozens of retry attempts across multiple strategies all failed.

**Workaround:** Always pass dates via URL parameters (`startDate`, `endDate`).

---

## Travellers Widget

The travellers stepper panel works with JS `.click()` on the increment/decrement
buttons.

### Opening the panel

```python
js("""
(()=>{
  let btn = document.querySelector('button[data-testid="travelers-field-trigger"]')
           || [...document.querySelectorAll('button')].find(b => b.textContent.includes('traveller'));
  if(btn){ btn.click(); return 'opened'; }
  return 'not found';
})()
""")
```

### Incrementing children count

```python
js("""
(()=>{
  let span = [...document.querySelectorAll('span')].find(s => s.textContent.trim() === 'Children');
  if(!span) return 'no Children label';
  let container = span.closest('div').parentElement;
  let buttons = container.querySelectorAll('button');
  let plus = buttons[buttons.length - 1];  // last button is "+"
  plus.click();
  return 'incremented';
})()
""")
```

### Setting child ages

Child age dropdowns are `<select>` elements with `aria-label` like
"Child 1 age", "Child 2 age", etc.

```python
js("""
(()=>{
  let selects = document.querySelectorAll('select');
  // selects[0] = Child 1 age, selects[1] = Child 2 age, etc.
  let setter = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, 'value').set;
  selects[0].value = '5';
  selects[0].dispatchEvent(new Event('change', {bubbles:true}));
  selects[1].value = '7';
  selects[1].dispatchEvent(new Event('change', {bubbles:true}));
  return 'ages set';
})()
""")
```

### Closing the panel

```python
js("""
(()=>{
  let btn = [...document.querySelectorAll('button')].find(b => b.textContent.trim() === 'Done');
  if(btn){ btn.click(); return 'done'; }
  return 'no Done button';
})()
""")
```

---

## Price Filter

On the results page, the nightly-price filter has two text inputs and two range
sliders.

| Element | Selector |
|---------|----------|
| Min text input | `#price-min` |
| Max text input | `#price-max` |
| Min range slider | `input[type="range"][aria-label*="Minimum"]` |
| Max range slider | `input[type="range"][aria-label*="Maximum"]` |

### Setting max price

The most reliable method is to click the input, select all, type the value, and
press Enter:

```python
click(x, y)  # coordinates of #price-max
js("document.getElementById('price-max').select()")
type_text("20000")
press_key("Enter")
```

Setting the value purely via JS (`dispatchEvent`) does trigger a re-search but
coordinate-click + type is more reliable for actually applying the filter.

---

## Key Lessons

1. **URL-first strategy** — Encode destination, dates, room count, adults,
   children, and child ages in the URL. Only use UI interaction for things the
   URL cannot express.

2. **JS `.click()` over coordinate clicks** — For buttons inside panels
   (traveller stepper, Done), find elements by text/attribute and call `.click()`
   in JS. Coordinate clicks on overlay panels are unreliable.

3. **`dispatchEvent` with `{bubbles: true}`** — Required for React-controlled
   inputs (selects, text fields). Without bubbling, React state won't update.

4. **Wait after navigation** — After `goto()` or pressing Search, call
   `wait_for_load()` + `time.sleep(3)` before interacting. Expedia loads
   results asynchronously.

5. **Indian locale** — `expedia.co.in` shows prices in ₹ (INR). The price
   filter values include the ₹ symbol and commas in the text input but the
   underlying range slider uses plain integers.

---

<!-- https://github.com/browser-use/browser-harness/blob/main/agent-workspace/domain-skills/expedia/flights-search.md (updated 2026-10-09) -->

# Expedia.ca — Flight Search Automation

Field-tested 2026-05-22 using browser-harness + Mac Chrome CDP with
residential Canadian ISP IP.

---

## Anti-Bot Protection

**Expedia uses DataDome (or similar behavior-based anti-bot).** Key facts:

- Blocks datacenter IPs with HTTP 429 ("Too Many Requests") and captcha
  pages ("Show us your human side…")
- Even headless Chromium with stealth init scripts gets blocked after
  1-2 requests from the same IP
- **With residential IP + real Chrome:** Works reliably — Canadian ISP IP
  through real Mac Chrome (not headless) passes DataDome checks without
  captchas or rate limiting
- **From datacenter IPs:** Use a cloud browser with residential proxies
  (Browser Use cloud, ScrapingBee, BrightData)

---

## Search URL Format

Expedia flight search can be initiated via a **direct URL** — this is the
**primary recommended approach**. It avoids the fragile datepicker,
airport autocomplete, and traveller widget entirely:

```
https://www.expedia.ca/Flights-Search?flight-type=roundtrip&mode=search&trip=roundtrip&leg1=from:YYZ,to:ICN,departure:2027/07/03TANYT&leg2=from:ICN,to:YYZ,departure:2027/07/25TANYT&passengers=adults:1&options=cabin:economy&sort=price%3Aa
```

URL parameters:
- `leg1` — outbound: `from:{IATA},to:{IATA},departure:{YYYY/MM/DD}TANYT`
- `leg2` — return: same format
- `passengers` — `adults:N,children:N`
- `options` — `cabin:economy|premium|business|first`
- `sort` — `price:a` (ascending), `price:d` (descending),
            `duration:a`, `departure:a`, `arrival:a`

### Navigation in browser-harness

Open the search in a task tab. The pre-filled URL loads results directly; no Search button click is needed:

```python
search_url = "https://www.expedia.ca/Flights-Search?flight-type=roundtrip&mode=search&trip=roundtrip&leg1=from:YYZ,to:ICN,departure:2027/07/03TANYT&leg2=from:ICN,to:YYZ,departure:2027/07/25TANYT&passengers=adults:1&options=cabin:economy&sort=price%3Aa"
new_tab(search_url)
wait_for_load(timeout=25)
if not js('!!document.querySelector("[data-stid*=listing]")'):
    raise RuntimeError("Flight listings are absent; inspect the search tab before continuing")
```

**CRITICAL: Do NOT click the Search button.** The pre-filled URL already
returns search results. Clicking Search just reloads the same page, which
can reset applied filters and clear extracted results entirely.

---

## UI Search Flow (when URL approach fails)

When navigating to `https://www.expedia.ca/Flights` directly, the page has:

### Step 1: Select trip type

Radio buttons: "Roundtrip" / "One-way" / "Multi-city"

Selector: `button[data-testid*="trip-type"]` or `a[data-stid*="trip-type"]`

### Step 2: Origin (Leaving from)

Selector: `input[aria-label="Leaving from"]` or `#origin-airport`

Fill: clear first, then type `YYZ`, wait for autocomplete dropdown,
press Enter (or click first result).

```python
origin = page.locator("#origin-airport")
origin.fill("")
origin.type("YYZ", delay=100)
page.wait_for_timeout(1000)
page.keyboard.press("ArrowDown")
page.keyboard.press("Enter")
```

### Step 3: Destination (Going to)

Same pattern, field `#destination-airport` or `input[aria-label="Going to"]`.

### Step 4: Dates

Dates are a date-picker widget. **Do NOT use the date picker via clicks**
(highly unreliable, see lessons below). Instead:

- Set date fields via JavaScript to bypass the calendar widget:

```python
page.evaluate("""() => {
  const depart = document.querySelector('input[aria-label*="Depart"]');
  const ret = document.querySelector('input[aria-label*="Return"]');
  if (!depart) return false;
  // React controlled input — need native setter
  const nativeSetter = Object.getOwnPropertyDescriptor(
    window.HTMLInputElement.prototype, 'value'
  ).set;
  nativeSetter.call(depart, '2026-07-01');
  depart.dispatchEvent(new Event('input', { bubbles: true }));
  depart.dispatchEvent(new Event('change', { bubbles: true }));
  if (ret) {
    nativeSetter.call(ret, '2026-07-23');
    ret.dispatchEvent(new Event('input', { bubbles: true }));
    ret.dispatchEvent(new Event('change', { bubbles: true }));
  }
  return true;
}""")
```

### Step 5: Travellers & Cabin

Toggle button: `button[data-testid="travelers-field-trigger"]`

Inside the panel:
- Adults stepper: +/- buttons within a container
- Cabin class: select element with options (Economy, Premium, Business, First)
- Close with "Done" button: find `button` containing text "Done"

### Step 6: Click Search

Search button locators:
- `button[data-testid="search-button"]`
- `button:has-text("Search")`
- Any `button` inside the search form

After clicking, wait for results:
```python
page.wait_for_url("**/Flights-Search**", timeout=15000)
page.wait_for_selector('div[data-testid="listing"], div[role="listitem"]', timeout=30000)
```

---

## Results Page

After search, the page shows a list of flight combinations.

### Flight cards

Each flight option is a card. Selectors in the URL-preload results page:
- `div[data-stid*="listing"]` — **primary working selector** with
  browser-harness

Selectors for the form-submitted results page:
- `div[data-testid="listing"]`
- `div[role="listitem"]`

### Extracting flight data from cards

Using browser-harness `js()` with the URL-preload approach (up to 200
listings):

```python
result = js("""Array.from(document.querySelectorAll('[data-stid*="listing"]'))
    .slice(0, 200).map(card => card.innerText).filter(Boolean)""")
if not result:
    raise RuntimeError("No flight cards appeared; inspect the page before reporting results")
```

This example captures at most 200 rendered cards. Verify additional pages or lazy loading before claiming complete results.

Using Playwright for structured extraction (form-submit result page):

```python
results = page.evaluate("""() => {
  const cards = document.querySelectorAll('div[data-testid="listing"], div[role="listitem"]');
  return Array.from(cards).slice(0, 20).map(card => {
    const priceEl = card.querySelector('[data-testid="price"], .uitk-type-500');
    const durationEl = card.querySelector('[data-testid="duration"]');
    const stopsEl = card.querySelector('[data-testid="stops"]');
    const airlineEl = card.querySelector('[data-testid="airline"]');
    const times = card.querySelectorAll('[data-testid="time"]');
    return {
      price: priceEl?.innerText?.trim(),
      duration: durationEl?.innerText?.trim(),
      stops: stopsEl?.innerText?.trim(),
      airline: airlineEl?.innerText?.trim(),
      depTime: times[0]?.innerText?.trim(),
      arrTime: times[1]?.innerText?.trim(),
    };
  }).filter(c => c.price);
}""")
```

### Sorting

Sort options: dropdown or button group — look for "Sort by" trigger,
then options: "Cheapest" / "Fastest" / "Best" / "Departure" / "Arrival".

### Stop Filters — Critical Quirk

**Expedia's stop checkboxes are exclusive — not inclusive.** Each
checkbox shows exactly that many stops, not "up to that many":

| Checkbox | Shows |
|----------|-------|
| Nonstop | 0 stops only (not "0 or 1") |
| 1 stop | Exactly 1 stop only |
| 2+ stops | 2+ stops only |

**To search for 0 OR 1 stop:**
1. Click "Nonstop" checkbox
2. Wait **5+ seconds** for page refresh
3. Click "1 stop" checkbox
4. Wait **5+ seconds** for page refresh

Each click triggers a full page reload with new results. Do NOT assume
they'll stack — wait for the reload to complete between clicks.

#### Finding stop filter checkboxes

Filter element IDs are dynamic (e.g., `NUM_OF_STOPS-0-:rs:`) with random
suffixes. Find by `aria-label` match:

```python
stops = js("""
JSON.stringify(
    Array.from(document.querySelectorAll('input[type=checkbox]'))
        .filter(i => (i.getAttribute('aria-label')||'').match(/stop/i))
        .map(i => ({id: i.id, label: i.getAttribute('aria-label'), checked: i.checked}))
)
""")
```

### Other Filters

Common filter panels:
- **Price range:** min/max inputs or slider
- **Airlines:** Checkbox list
- **Times:** Checkbox groups for morning/afternoon/evening

---

## Click-through to Fare Selection

To get the REAL total price (not the listing estimate):

1. Click a flight card/button → opens fare details or booking page
2. Look for "Continue" or "Select" button: `button:has-text("Select")`
3. On the fare selection page, extract the total:
   ```
   div[data-testid="price-summary"]
   div[data-testid="fare-total"]
   h4:has-text("Total")
   ```
4. Take screenshot to verify price

---

## Key Lessons

1. **URL-first strategy** — encode everything you can into the search URL
   parameter format. Avoid interacting with the datepicker, airport
   autocomplete, and traveller widget (all fragile).

2. **Direct URL loading** skips the landing page bot check entirely in
   many cases. Navigate to `Flights-Search?...` URL directly rather than
   clicking through from the homepage.

3. **Do NOT click Search with URL pre-load** — the pre-filled URL already
   returns results. Clicking Search reloads the same page and resets any
   applied filters.

4. **Residential IP + real Chrome works** — from a home ISP connection
   with a non-headless Chrome instance, Expedia's DataDome passes without
   captchas. Cloud browser with residential proxies only needed from
   datacenter IPs.

5. **Stop filters are exclusive** — each checkbox toggles a specific stop
   count. To show 0 AND 1 stop, click both sequentially with 5+ second
   waits for the page refresh between clicks.

6. **Filter IDs are dynamic** — use `aria-label` matching instead of
   hard-coded IDs for stop checkboxes and other filter elements.

7. **`goto_url()` + `new_tab()` fallback** — navigate the existing tab
   silently first; if results don't load, open a new tab as fallback.
   This avoids unnecessary macOS window popups.

8. **Fresh browser context per search** — don't reuse cookies/session
   across repeated searches from the same IP, as bot score accumulates.

9. **Use non-headless mode** with xvfb on headless servers — headless
   mode is heavily fingerprintable. The `--disable-blink-features=AutomationControlled` flag + `navigator.webdriver` override is
   essential.

10. **Wait for async loads** — after search or filter application, wait
    for the URL to change to `/Flights-Search*` and for flight elements
    to appear. 15-30s timeout recommended. Filter changes take 5+ seconds.