How to Extract Structured Data From Websites With an LLM (2026)

Write one JSON Schema for the record you want and use it on every site. For static pages, fetch clean page text (Markdown) and have the LLM fill the schema. Use a browser agent only for pages that need a login, clicks or a JavaScript flow. Validate every record against the schema, retry or flag the ones that fail, and store the source URL with each record.

We make Browser Use. Facts about the other tools come from their own docs and pricing pages, checked October 26, 2026, and linked where used.

Reliability checklist

• Schema: required fields with types, and enums where values are fixed. Write it in JSON Schema or generate it from a Pydantic or Zod model.
• Constrained output: turn on your model's structured outputs. OpenAI's makes responses adhere to your JSON Schema; Anthropic's guarantees field types and required fields.
• Validate in your code: parse every record with Pydantic or Zod. Valid JSON is not the same as correct data.
• Null, not a guess: make missing fields nullable but still required, and tell the model to return null when the page doesn't show the value.
• Evidence: keep source_url on every record, plus a short quote from the page for fields you'll act on.
• Dedupe: on a stable key such as the canonical URL or SKU.
• Be a polite client: rate-limit per domain, follow robots.txt and the site's terms.
• Spot-check: compare a random sample of records with the live pages before you trust a run, and again after a site changes.
• Re-run with saved code: for a job you repeat, keep the working extraction script instead of prompting from scratch each time.

Which tool fits

• Known URLs, static pages: fetch + Markdown + LLM, or Firecrawl JSON mode. A browser agent adds model steps and browser time you don't need here.
• Self-hosted, any model, Python: Crawl4AI.
• Code-first browser automation with AI steps, in TypeScript, Python or Go: Stagehand.
• Logins, filters, pagination, multi-step forms: a browser agent such as Browser Use, or Firecrawl's Interact.

Browser Use example

Pass the schema as output_schema. Browser Use checks the agent's final answer against it; if it can't match, the run fails with Could not produce output matching outputSchema instead of returning output that doesn't match the schema (docs).

# Python 3.10+
# pip install "browser-use-sdk>=3.12"
from browser_use_sdk.v4 import BrowserUse
from pydantic import BaseModel
 
class Product(BaseModel):
    name: str
    # required, may be null
    price: float | None
    currency: str | None
    source_url: str
 
class Products(BaseModel):
    products: list[Product]
 
# reads BROWSER_USE_API_KEY
client = BrowserUse()
run = client.runs.create(
    "Open https://shop.example.com/laptops and go "
    "through every results page. For each product "
    "return its name, price, currency and product "
    "URL as source_url. Use null for any value "
    "the page does not show. Do not guess.",
    output_schema=Products.model_json_schema(),
)
run = client.runs.wait_for_completion(run.id)
if run.status.value != "completed":
    # flag the site, retry later
    raise RuntimeError(run.error or run.status.value)
# validate again in your code
products = Products.model_validate(run.output)

Limits: API runs only, JSON Schema 2020-12 by default, up to 64 KiB, 64 nesting levels and about 4,000 values.

For a job you repeat, have the first run save a tested script in a workspace. Later runs pass workspace_id, fetch fresh data from the live site and repair the script if the site changed. Each run still uses model tokens (rerunnable scripts).

More

• Turn one page into Markdown, free, no signup: Website to Markdown.
• Browser Use vs Firecrawl, measured: head-to-head.
• Scraping pages behind a login or filters: AI web scraper.
• Basic vs interactive scraping and tool costs: The Ultimate Guide to Web Scraping.