BROWSER USE - Browser Use Agents: give Browser Use a task and receive completed work. API V4 is current for new integrations. - Browser Infrastructure: connect your agent or automation to managed browsers through SDK, REST, or CDP. Starts at $0.02/browser-hour. - Developer tools: Open Source, Browser Harness, SDK, and MCP support the two products above. [Developer Index](https://browser-use.com/index.md) [Product Map](https://browser-use.com/llms.txt) [Full Product Context](https://browser-use.com/llms-full.txt) [Pricing](https://browser-use.com/pricing.md) [Cloud Docs](https://docs.browser-use.com/cloud/quickstart) [Open Source Docs](https://docs.browser-use.com/open-source/introduction) --- # Xiaohongshu scraper skill for AI agents Source: https://browser-use.com/skills/xiaohongshu. One file from github.com/browser-use/browser-harness, MIT licensed. Written for browser-harness: code calls its helpers (http_get, js, goto_url). Sites change; these notes describe what worked when last updated. --- # Xiaohongshu — Search and Sort URL patterns: - Home / discovery: `https://www.xiaohongshu.com/explore` - Search results: `https://www.xiaohongshu.com/search_result?keyword=...` ## Search flow - Prefer direct navigation to the desktop search results page over automating the home-page search box. - Reliable primary path: `https://www.xiaohongshu.com/search_result?keyword=&source=web_explore_feed` - This route loads the normal desktop results page and avoids home-page input flakiness. - The search results page can also appear with variants such as `type=51` or other `source` values after in-app navigation; do not treat those as suspicious if the rendered results are correct. - The top search box on `explore` can work, and searching from the home page has transitioned to `search_result` without a login wall in some sessions. - The page exposes duplicate search inputs in the DOM with the same placeholder `搜索小红书`. - The home-page search input can behave like a tightly controlled app field: direct DOM value assignment may be cleared immediately, and harness `type_text()` may fail to populate it even when the input is focused. - Treat the home-page input as best-effort only. Use it when a human-like interactive flow matters, but for automation default to constructing the `search_result` URL directly. ## Sort behavior - On the current desktop results layout, `最新` is **not** a top-level tab beside `综合`. - Open the `筛选` control in the upper-right of the results header to access sort options. - Inside `筛选`, `排序依据` contains: - `综合` - `最新` - `最多点赞` - `最多评论` - `最多收藏` - The `排序依据` row can render duplicate DOM nodes for the same pill text, including non-interactive clones. - Raw global text search for `最新` can hit the wrong node first. Scope to the `排序依据` section and then choose the visible interactive `.tags` node. - Prefer semantic filtering such as `aria-hidden != "true"` or section-scoped visible `.tags` selection over style-specific checks. - When `最新` is active, the `筛选` trigger changes to `已筛选`. - The rendered feed and the `已筛选` / active-pill UI are more reliable than `window.__INITIAL_STATE__.search.searchContext.sort` for confirming latest sort. ## Stable cues - Search channel tabs near the top: `全部`, `图文`, `视频`, `用户` - Sort panel labels: `筛选`, `排序依据`, `最新` - Filter sections also visible in the panel: `笔记类型`, `发布时间`, `搜索范围`, `位置距离` ## Interaction notes - DOM `.click()` opened the `筛选` panel reliably. - DOM `.click()` on the visible `最新` pill inside the open `排序依据` section reliably activated latest sort. - The reliable DOM pattern was: - find the `排序依据` section / `.filters` block - search within that block for `.tags` - choose the one whose text is `最新` and which is the visible interactive node - call `.click()` on that visible node - Example selector strategy: - find `.filters` whose first label is `排序依据` - inside it, pick `.tags` where `textContent.trim() === "最新"` and `el.getAttribute("aria-hidden") !== "true"` - `getClientRects().length > 0` alone may be insufficient to distinguish the working node from a duplicate. - A broad `document.querySelectorAll("*")` text match for `最新` is not reliable on this page because it may click the hidden duplicate instead of the visible control. - Coordinate click on the visible `最新` pill also worked and remains a valid fallback if DOM targeting gets confused by future UI changes. - After selecting `最新`, the grid briefly showed skeleton placeholders before the refreshed results appeared. - The search page stores the currently rendered note cards in `window.__INITIAL_STATE__.search.feeds._value` as an array of feed entries. For ordinary note cards, the useful fields were: - `id` - `xsecToken` - `noteCard.displayTitle` - `noteCard.user.nickname` - The feed array can contain non-note inserts such as hot-query modules. Filter for entries with `noteCard` before treating an item as a note result. ## Post opening - Do **not** assume a raw results link like `https://www.xiaohongshu.com/explore/` is directly openable. - Opening that raw `/explore/` URL in a fresh tab can redirect to the web `404` / app-only gate even when the same post is openable from search results. - To open a post from search results, click the visible card image / card in-page first. - That click navigation can land on a tokenized URL like `https://www.xiaohongshu.com/explore/?xsec_token=...&xsec_source=pc_search`, which is a more reliable note URL than the raw `/explore/` form. - Once the tokenized URL is obtained from the click flow, it can be revisited in-session for extraction. - If the search results state is already loaded, you can reconstruct the tokenized note URL directly from a feed item without re-clicking: - `https://www.xiaohongshu.com/explore/?xsec_token=&xsec_source=pc_search` ## Post extraction - On tokenized post pages opened via `pc_search`, `document.body.innerText` can be a useful first-pass extraction source because it often includes the rendered note text, hashtags, timestamp, engagement counts, and visible comments. - Verify that the note content actually rendered before trusting `document.body.innerText`, because the page can also include substantial navigation, footer, and comment noise. - Prefer `document.body.innerText` as a fallback or initial probe before writing fragile per-element selectors for post content. ## Gotchas - Do not assume `Enter` alone finished the workflow until you verify the URL changed to `search_result` or the result grid appeared. - Do not assume the visible `综合` tab controls all sorting; on this layout, time ordering is hidden inside `筛选`. - Do not assume the first DOM node whose text is `最新` is the clickable one; this panel duplicates pills and the hidden clone can absorb naive text-based targeting without changing state. - Do not assume a successfully opened post can be reproduced by stripping query params; preserve the `xsec_token` when reopening results-derived post URLs.