Finance and markets

macrotrends.net

Macrotrends - Data Extraction

Notes an AI agent reads before it works on macrotrends.net: what to open, what to call, what breaks. Written by browser-harness contributors after real runs, and kept on GitHub under the MIT license.

Give it to your agent

Any coding agent

Start the task with this line. The Markdown version is the same notes without the page around them.

Read https://browser-use.com/skills/macrotrends.md, then <your task>

Browser Harness

Save the files into your workspace and turn domain skills on. The code below calls its helpers, such as http_get and js. What Browser Harness is.

d="${BH_AGENT_WORKSPACE:-$HOME/.config/browser-harness/agent-workspace}/domain-skills/macrotrends"
mkdir -p "$d" && cd "$d"
curl -sLO https://raw.githubusercontent.com/browser-use/browser-harness/main/agent-workspace/domain-skills/macrotrends/scraping.md
export BH_DOMAIN_SKILLS=1

https://www.macrotrends.net - long-term historical financial and economic charts. Three access patterns depending on page type; all work with plain http_get, no browser required.

All results validated against live site on 2026-04-18.

Do this first: pick your access pattern

GoalPatternLatencyVariable
Stock OHLCV price historyDirect iframe PHP~190msdataDaily
Stock market cap (daily)Direct iframe PHP~200mschartData
Stock fundamentals (PE, revenue, margins)Direct iframe PHP~140mschartData
S&P 500 / composite index chartschart_iframe_comp.php~90msoriginalData
Economic indicators (rates, yields, CPI)/economic-data/ JSON API~150msdata[] array
Gold, commodity pricesEither path (both work)~150msdata[] or originalData

Never use the browser for Macrotrends read-only tasks. All endpoints are accessible via http_get with the default Mozilla/5.0 UA. For pages that occasionally 403, switch to a Chrome UA (see gotchas).


Pattern 1: Stock price history (OHLCV)

Construct the iframe URL directly - no need to fetch the main page first.

import json, re
from helpers import http_get
 
def get_stock_ohlcv(ticker: str, years_back: int = None) -> list[dict]:
    """
    Returns daily OHLCV records for any US stock.
 
    ticker:     uppercase ticker symbol, e.g. 'AAPL', 'MSFT', 'TSLA', 'NVDA'
    years_back: number of years of history (1=~250 records, 15=~3772 records).
                Omit (None) to get ALL available history (AAPL goes back to 1980).
    """
    url = f"https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/stock_price_history.php?t={ticker}"
    if years_back:
        url += f"&yb={years_back}"
 
    html = http_get(url)
    m = re.search(r'var\s+dataDaily\s*=\s*\[', html)
    if not m:
        raise ValueError(f"No dataDaily found for ticker {ticker!r}")
 
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
# Usage
records = get_stock_ohlcv('AAPL', years_back=15)
# [{'d': '2011-04-18', 'o': '9.771', 'h': '9.9547', 'l': '9.593', 'c': '9.9433', 'v': '18.275'}, ...]
 
latest = records[-1]
# {'d': '2026-04-17', 'o': '266.96', 'h': '272.3', 'l': '266.72', 'c': '270.23',
#  'v': '55.211', 'ma50': '260.554', 'ma200': '251.828'}
 
print(f"{latest['d']}: close=${latest['c']} vol={latest['v']}M shares")

dataDaily field reference

FieldMeaningType
dDate (YYYY-MM-DD)str
oOpen price (adjusted for splits)str/float
hHighstr/float
lLowstr/float
cClosestr/float
vVolume in millions of sharesstr/float
ma5050-day moving averagestr/float (appears on recent records only)
ma200200-day moving averagestr/float (appears on recent records only)

Note: All price values are strings - cast with float(). Volume is millions: 55.211 = 55.2M shares traded.

Confirmed tickers (2026-04-18)

All tested with direct iframe URL, no page fetch needed:

# All work: AAPL, MSFT, TSLA, NVDA, GOOGL, AMZN, META, NFLX, etc.
# 3772 records for yb=15 (goes back to 2011-04-18)
# AAPL full history: 11428 records back to 1980-12-12

Pattern 2: Stock fundamentals (PE ratio, revenue, market cap, margins)

Different PHP files depending on metric. Construct directly.

Market cap (daily, in billions USD)

import json, re
from helpers import http_get
 
def get_market_cap(ticker: str, years_back: int = 15) -> list[dict]:
    url = f"https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/market_cap.php?t={ticker}&yb={years_back}"
    html = http_get(url)
    m = re.search(r'var\s+chartData\s*=\s*\[', html)
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
data = get_market_cap('AAPL')
# [{'date': '2026-04-15', 'v1': 3929.35}, {'date': '2026-04-16', 'v1': 3884.67}, ...]
# v1 = market cap in billions USD

PE ratio, revenue, current ratio (quarterly/annual fundamentals)

import json, re
from helpers import http_get
 
def get_fundamental(ticker: str, metric_type: str, statement: str,
                    freq: str = 'Q', years_back: int = 15) -> list[dict]:
    """
    freq: 'Q' = quarterly, 'A' = annual
    """
    url = (
        f"https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/"
        f"fundamental_iframe.php?t={ticker}&type={metric_type}&statement={statement}"
        f"&freq={freq}&sub=&yb={years_back}"
    )
    html = http_get(url)
    m = re.search(r'var\s+chartData\s*=\s*\[', html)
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
# PE ratio
pe = get_fundamental('AAPL', 'pe-ratio', 'price-ratios')
# [{'date': '2025-09-30', 'v1': 254.146, 'v2': 7.46, 'v3': 34.07}, ...]
# v1 = stock price, v2 = quarterly EPS, v3 = PE ratio
 
# Revenue
rev = get_fundamental('AAPL', 'revenue', 'income-statement')
# [{'date': '2025-12-31', 'v1': 435.617, 'v2': 143.756, 'v3': 15.65}, ...]
# v1 = TTM revenue ($B), v2 = quarterly revenue ($B), v3 = YoY growth %
 
# Total assets
assets = get_fundamental('AAPL', 'total-assets', 'balance-sheet')
 
# Current ratio
ratio = get_fundamental('AAPL', 'current-ratio', 'ratios')

Profit margins

def get_profit_margins(ticker: str, years_back: int = 15) -> list[dict]:
    url = (
        f"https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/"
        f"fundamental_metric.php?t={ticker}&chart=profit-margin&sub=&yb={years_back}"
    )
    html = http_get(url)
    m = re.search(r'var\s+chartData\s*=\s*\[', html)
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
margins = get_profit_margins('AAPL')
# [{'date': '2025-12-31', 'v1': 47.33, 'v2': 32.38, 'v3': 27.04}, ...]
# v1 = gross margin %, v2 = operating margin %, v3 = net margin %

Dividend yield

def get_dividend_yield(ticker: str, years_back: int = 15) -> list[dict]:
    url = f"https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/dividend_yield.php?t={ticker}&yb={years_back}"
    html = http_get(url)
    m = re.search(r'var\s+chartData\s*=\s*\[', html)
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
dy = get_dividend_yield('AAPL')
# [{'date': '2026-04-17', 'c': 270.23, 'ttm_d': 1.03848, 'ttm_dy': 0.3843}, ...]
# c = stock price, ttm_d = TTM dividend ($), ttm_dy = TTM yield (%)

Stock metric URL reference

MetricPHP fileExtra params
Stock price OHLCVstock_price_history.php-
Market cap (daily)market_cap.php-
Dividend yielddividend_yield.php-
Stock splits (price history)stock_splits.php-
PE ratiofundamental_iframe.phptype=pe-ratio&statement=price-ratios
Revenuefundamental_iframe.phptype=revenue&statement=income-statement
Total assetsfundamental_iframe.phptype=total-assets&statement=balance-sheet
Current ratiofundamental_iframe.phptype=current-ratio&statement=ratios
Profit marginsfundamental_metric.phpchart=profit-margin

Base URL prefix: https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/

All take ?t={TICKER}&yb={N} (or &sub=&yb={N} for the fundamental ones).


Pattern 3: Index and composite charts (S&P 500, Shiller PE, etc.)

These pages embed chart data via chart_iframe_comp.php. The variable is originalData.

import json, re
from helpers import http_get
 
def extract_index_chart(page_id: int, url_slug: str) -> list[dict]:
    """
    page_id:  the numeric ID from the page URL, e.g. 2577
    url_slug: last segment of the page URL, e.g. 'sp500-pe-ratio-price-to-earnings-chart'
    """
    url = f"https://www.macrotrends.net/assets/php/chart_iframe_comp.php?id={page_id}&url={url_slug}"
    html = http_get(url)
    m = re.search(r'var\s+originalData\s*=\s*\[', html)
    if not m:
        raise ValueError("originalData not found — this page may use a different pattern")
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0: ei = j; break
    return json.loads(html[si:ei+1])
 
# S&P 500 PE ratio (1180 monthly records, 1927-2026)
pe_data = extract_index_chart(2577, 'sp500-pe-ratio-price-to-earnings-chart')
# [{'date': '1927-12-01', 'close': '15.9099'}, ..., {'date': '2026-03-01', 'close': '27.8925'}]
# 'close' is the PE ratio value
 
# Gold prices (1336 monthly records, 1915-2026)
gold_data = extract_index_chart(1333, 'historical-gold-prices-100-year-chart')
# [{'id': 'GOLDAMGBD228NLBM', 'date': '1915-01-01', 'close': '629.36', 'close1': '19.250'}, ...]
# 'close' = inflation-adjusted price, 'close1' = nominal USD price
 
print(f"Latest S&P PE: {pe_data[-1]}")   # {'date': '2026-03-01', 'close': '27.8925'}
print(f"Latest gold:   {gold_data[-1]}") # {'id': ..., 'date': '2026-04-01', 'close': '5177.19', 'close1': '5177.190'}

Detecting which pattern a page uses

def get_page_pattern(page_url: str) -> str:
    html = http_get(page_url)
    if 'chart_iframe_comp.php' in html:
        return 'index_chart'           # use extract_index_chart()
    elif 'generateChart' in html and 'highchartsURL' in html:
        return 'economic_api'          # use get_economic_data()
    elif '/production/stocks/desktop/PRODUCTION/' in html:
        return 'stock_iframe'          # use get_stock_ohlcv() etc.
    return 'unknown'

To get the ID and slug from a page

import re
from helpers import http_get
 
page_url = "https://www.macrotrends.net/2577/sp500-pe-ratio-price-to-earnings-chart"
html = http_get(page_url)
 
# Option A: parse from the iframe src in the HTML
m = re.search(r'chart_iframe_comp\.php\?id=(\d+)&url=([^"&]+)', html)
if m:
    page_id, url_slug = int(m.group(1)), m.group(2)
 
# Option B: derive from the page URL (works when slug matches)
import urllib.parse
parts = page_url.rstrip('/').split('/')
page_id  = int(parts[-2])   # 2577
url_slug = parts[-1]         # 'sp500-pe-ratio-price-to-earnings-chart'

Pattern 4: Economic indicator API

Pages that use generateChart() in their JS load data from /economic-data/{pageID}/{freq}. This endpoint requires a Referer header matching the page URL.

import json, datetime, gzip, urllib.request
from helpers import http_get
 
def get_economic_data(page_id: int, referer_url: str, freq: str = 'D') -> dict:
    """
    page_id:     numeric ID from the page URL (e.g. 2015 for Fed Funds Rate)
    referer_url: the full page URL — required as Referer header
    freq:        'D' = daily, 'M' = monthly (not all support both)
    
    Returns {'data': [[ts_ms, value], ...], 'metadata': {...}}
    """
    url = f"https://www.macrotrends.net/economic-data/{page_id}/{freq}"
    headers = {
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
        "Accept": "application/json, */*",
        "Accept-Encoding": "gzip",
        "Referer": referer_url,
    }
    with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=20) as r:
        raw = r.read()
        if r.headers.get("Content-Encoding") == "gzip":
            raw = gzip.decompress(raw)
        result = json.loads(raw)
    if result is None:
        raise ValueError(f"pageID={page_id} does not support freq={freq!r}")
    return result
 
# Fed Funds Rate (daily, 25319 records)
ffr = get_economic_data(2015, "https://www.macrotrends.net/2015/fed-funds-rate-historical-chart", freq='D')
print(ffr['metadata']['name'])  # 'Fed Funds Interest Rate'
print(ffr['metadata']['label']) # '%'
 
# Convert timestamps to dates
for ts_ms, value in ffr['data'][-3:]:
    dt = datetime.datetime.fromtimestamp(ts_ms / 1000, datetime.UTC)
    print(f"{dt.strftime('%Y-%m-%d')}: {value}%")
# 2026-04-13: 3.64%
# 2026-04-14: 3.64%
# 2026-04-15: 3.64%
 
# 10-Year Treasury yield (daily, 16074 records)
t10 = get_economic_data(2016, "https://www.macrotrends.net/2016/10-year-treasury-bond-rate-yield-chart", freq='D')
# Last: 2026-04-15: 4.29%
 
# Gold prices (monthly, 1336 records, 1915-present) — template=5
gold = get_economic_data(1333, "https://www.macrotrends.net/1333/historical-gold-prices-100-year-chart", freq='M')
# metadata: {'name': 'Gold Prices', 'currency': '$', 'label': ''}
 
# US Unemployment Rate (monthly, 938 records)
unemp = get_economic_data(1316, "https://www.macrotrends.net/1316/us-national-unemployment-rate", freq='M')
# metadata: {'name': 'U.S. Unemployment Rate', 'label': '%'}
 
# Debt-to-GDP ratio (monthly, 712 records)
debt_gdp = get_economic_data(1381, "https://www.macrotrends.net/1381/debt-to-gdp-ratio-historical-chart", freq='M')

metadata fields

{
    'name':            'Fed Funds Interest Rate',  # chart title
    'tableHeaderName': 'Fed Funds Interest Rate',
    'currency':        '',            # '$' for dollar-denominated series
    'label':           '%',          # units label
    'chartType':       'line',
    'mobileChartType': 'line',
    'lineWidth':       2,
    'positiveColor':   '#2caffe',
    'negativeColor':   '',
    'decimals':        '',
    'chartScale':      'linear',
    'seriesUnits':     ''
}

Available frequency codes

CodeMeaningNotes
DDailyMost series support this
MMonthlyReturns null if not available
QQuarterlyUsually null - use M instead
AAnnualUsually null - use M instead
DEFAULTDefault (usually monthly)Same data as M for most series
INDEXMONTHLYMonthly index closeSome commodity/index series
INDEXDAILYDaily indexSome series
DAILYEXCHANGERATEDaily FX rateCurrency pairs
10YD10-year dailySpecialized series

Try D first, fall back to M if you get null.

Known economic page IDs

IDURL slugDescription
1316us-national-unemployment-rateU.S. Unemployment Rate (monthly, back to 1948)
1333historical-gold-prices-100-year-chartGold Prices (monthly, back to 1915)
1381debt-to-gdp-ratio-historical-chartU.S. Debt to GDP Ratio
2015fed-funds-rate-historical-chartFed Funds Interest Rate (daily, back to 1954)
201610-year-treasury-bond-rate-yield-chart10-Year Treasury Yield (daily, back to 1962)
2577sp500-pe-ratio-price-to-earnings-chartS&P 500 PE Ratio (uses chart_iframe_comp.php)

Generic extraction helper

One function that handles all three embedded-JS patterns:

import json, re
from helpers import http_get
 
def extract_chart_var(html: str, var_name: str) -> list:
    """Extract a JS array variable from Macrotrends iframe HTML."""
    m = re.search(rf'var\s+{re.escape(var_name)}\s*=\s*\[', html)
    if not m:
        return []
    si = html.index('[', m.start())
    bc = 0
    for j, ch in enumerate(html[si:], si):
        if ch == '[':   bc += 1
        elif ch == ']':
            bc -= 1
            if bc == 0:
                return json.loads(html[si:j+1])
    return []
 
# Works for dataDaily, chartData, or originalData:
html = http_get("https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/stock_price_history.php?t=AAPL&yb=15")
daily = extract_chart_var(html, 'dataDaily')
 
html2 = http_get("https://www.macrotrends.net/assets/php/chart_iframe_comp.php?id=2577&url=sp500-pe-ratio-price-to-earnings-chart")
pe_data = extract_chart_var(html2, 'originalData')

URL construction guide

Stock pages

STOCK_BASE = "https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/"
 
# Price history OHLCV
f"{STOCK_BASE}stock_price_history.php?t={ticker}"                        # all history
f"{STOCK_BASE}stock_price_history.php?t={ticker}&yb={years}"             # last N years
 
# Market cap
f"{STOCK_BASE}market_cap.php?t={ticker}&yb={years}"
 
# Fundamentals
f"{STOCK_BASE}fundamental_iframe.php?t={ticker}&type={type}&statement={stmt}&freq={freq}&sub=&yb={years}"
# type/statement combos: pe-ratio/price-ratios, revenue/income-statement,
#                        total-assets/balance-sheet, current-ratio/ratios
 
# Metrics
f"{STOCK_BASE}fundamental_metric.php?t={ticker}&chart={metric}&sub=&yb={years}"
# metrics: profit-margin
 
# Dividend yield
f"{STOCK_BASE}dividend_yield.php?t={ticker}&yb={years}"

Economic / index pages

# From numeric ID + URL slug (read from page source or page URL)
f"https://www.macrotrends.net/assets/php/chart_iframe_comp.php?id={id}&url={slug}"
 
# Economic indicator JSON API (requires Referer header)
f"https://www.macrotrends.net/economic-data/{page_id}/{freq}"

Rate limits and anti-bot

  • No rate limiting observed at any tested volume. 10 rapid requests to the same stock iframe completed in 1.8s with no throttling, CAPTCHA, or 429 errors.
  • Default UA works (Mozilla/5.0) for most endpoints. The iframe PHP files never 403'd.
  • Chrome UA needed for some main HTML pages (not data endpoints): use when fetching /stocks/charts/... or /2015/... wrapper pages if you get 403. Switch to:
    headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"}
  • Referer required for /economic-data/{id}/{freq} - send the page URL as Referer. Without it, the request is allowed but you get a 403 on some pages.
  • No cookies, sessions, or auth tokens needed for any endpoint.

Gotchas

Main page URL ≠ data page: Some URLs redirect to different content. /1316/us-national-debt-by-year redirects to /1316/us-national-unemployment-rate. Always check the final URL with r.url if the returned data looks wrong. Use the final URL as the Referer.

yb parameter controls history depth:

  • yb=1 → ~250 records (last year)
  • yb=15 → ~3772 records (last 15 years)
  • omit → full history (AAPL: 11428 records to 1980; default for most queries)

Two iframe patterns for economic pages: Pages at macrotrends.net/NNNN/slug use either chart_iframe_comp.php (→ originalData) or generateChart + /economic-data/ API. Check the main page HTML to detect which:

if 'chart_iframe_comp.php' in html:   # use extract_index_chart()
elif 'highchartsURL' in html:          # use get_economic_data()

Gold data has two price columns:

{'id': 'GOLDAMGBD228NLBM', 'date': '2026-04-01', 'close': '5177.19', 'close1': '5177.190'}
# 'close'  = inflation-adjusted price (base year adjusts over time)
# 'close1' = nominal USD price (the raw market price)

Economic API frequency codes: Only D and M consistently return data across most series. A and Q return null for most economic indicators. Always try D first.

chartData fields vary by metric:

  • market_cap.php → {'date', 'v1'} (v1 = market cap in $B)
  • fundamental_iframe.php type=pe-ratio → {'date', 'v1', 'v2', 'v3'} (stock price, EPS, PE)
  • fundamental_iframe.php type=revenue → {'date', 'v1', 'v2', 'v3'} (TTM revenue, quarterly revenue, YoY%)
  • fundamental_metric.php chart=profit-margin → {'date', 'v1', 'v2', 'v3'} (gross%, operating%, net%)
  • dividend_yield.php → {'date', 'c', 'ttm_d', 'ttm_dy'} (price, dividend, yield%)

Bracket matching required for large arrays: The var dataDaily = [...] in stock iframes is ~450KB with 3772 OHLCV records. The re.DOTALL greedy approach works but is slow; bracket-counting (bc pattern above) is O(n) and fast.

No public API for ticker lookup: To find the company slug for a URL, check the search endpoint: https://www.macrotrends.net/production/stocks/desktop/PRODUCTION/ticker_search_list.php?v=YYYYMMDD - but the stock price iframe only needs the ticker symbol (?t=AAPL), not the slug.

scraping.md · updated Apr 28, 2026 · 2,571 words

Sites change. These notes describe what worked when they were last updated, not a guarantee they still do. If something broke, the fix is a pull request to browser-harness.