Social and communities

quora.com

Updated Apr 28, 2026

Quora - Data Extraction

Notes an AI agent reads before it works on quora.com: what to open, what to call, what breaks. Written by browser-harness contributors after real runs, and kept on GitHub under the MIT license.

Run it in Browser Use Cloud Source on GitHub

Give it to your agent

Any coding agent

Start the task with this line. The Markdown version is the same notes without the page around them.

Read https://browser-use.com/skills/quora.md, then <your task>

Browser Harness

Save the files into your workspace and turn domain skills on. The code below calls its helpers, such as http_get and js. What Browser Harness is.

d="${BH_AGENT_WORKSPACE:-$HOME/.config/browser-harness/agent-workspace}/domain-skills/quora"
mkdir -p "$d" && cd "$d"
curl -sLO https://raw.githubusercontent.com/browser-use/browser-harness/main/agent-workspace/domain-skills/quora/scraping.md
export BH_DOMAIN_SKILLS=1

https://www.quora.com - Q&A platform. One reliable access path: http_get with a Chrome UA against question, answer, topic, and profile pages. Quora SSR-renders all public data into window.ansFrontendGlobals.data.inlineQueryResults via .push() calls. No browser needed for read-only tasks.

Do this first: pick your access path

Goal	Best approach	Latency
Question metadata + first ~3 ranked answers	`http_get` question page + parse push payloads	~600ms
Single answer (full text + upvotes + views)	`http_get` answer permalink	~400ms
Answer count for a question	question page, payload with `answerCount`	same request as above
Topic metadata (id, name, follower count)	`http_get` topic page + parse push payloads	~400ms
User profile (name, follower/following, credential)	`http_get` profile page + parse push payloads	~500ms
Keyword search results	NOT available via http_get - server returns no result data	N/A

Never use a browser for read-only Quora tasks. All question, answer, topic, and profile data is server-rendered. Browser is only needed for authenticated actions (posting, upvoting, following) or for getting more than the first ~3 answers on a question page (the rest load via XHR pagination).

UA requirement: Chrome or Firefox - NOT bare Mozilla/5.0

bare "Mozilla/5.0"  -> HTTP 403
Googlebot UA        -> HTTP 403
Chrome UA           -> HTTP 200  (confirmed working)
Firefox UA          -> HTTP 200  (confirmed working)

Use this header bundle for all requests:

import urllib.request, gzip, json, re
 
CHROME_UA = (
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
    "AppleWebKit/537.36 (KHTML, like Gecko) "
    "Chrome/123.0.0.0 Safari/537.36"
)
 
def quora_get(url):
    """Fetch any public Quora page. Returns HTML string.
    Requires Chrome/Firefox UA — bare Mozilla/5.0 returns 403.
    """
    req = urllib.request.Request(url, headers={
        "User-Agent": CHROME_UA,
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        "Accept-Encoding": "gzip",
        "Accept-Language": "en-US,en;q=0.9",
    })
    with urllib.request.urlopen(req, timeout=20) as r:
        data = r.read()
        if r.headers.get("Content-Encoding") == "gzip":
            data = gzip.decompress(data)
        return data.decode()

The data format: `ansFrontendGlobals.data.inlineQueryResults`

Quora SSR embeds all page data as a series of .push("...") calls inside <script> blocks. Each call pushes a JSON-encoded string (with escaped quotes) into inlineQueryResults. There are no JSON-LD blocks, no __NEXT_DATA__, no React hydration state - only these push calls.

def extract_quora_payloads(html):
    """Extract and parse all push() payloads from a Quora page.
    Returns list of dicts (already decoded from double JSON encoding).
    """
    raw_payloads = re.findall(r'\.push\("((?:[^"\\]|\\.)*)"\)', html)
    results = []
    for raw in raw_payloads:
        try:
            # Two levels of encoding: outer JS string escape, inner JSON
            inner = json.loads('"' + raw + '"')   # decode JS string escaping
            results.append(json.loads(inner))      # decode actual JSON
        except Exception:
            pass
    return results

A question page returns 16 payloads. A profile or topic page returns 3 payloads. The payloads that matter are identified by their data keys, not by position (positions are stable across requests for the same page type, but best to key on content).

Path 1: Question page - metadata + first answers (fastest)

def quora_question(url):
    """
    Scrape a Quora question page.
    Returns:
      question: {qid, id, title, url, slug, topics}
      answers:  list of answer dicts (first ~3 ranked answers only)
      answer_count: total answer count (all answers, not just loaded)
      related_questions: list of question title strings
    Only the first ~3 highest-ranked answers are SSR'd.
    The rest require XHR pagination (browser or session cookies needed).
    """
    html = quora_get(url)
    payloads = extract_quora_payloads(html)
 
    def spans_to_text(json_str):
        """Quora stores all text as serialized span objects."""
        try:
            doc = json.loads(json_str)
            parts = []
            for sec in doc.get('sections', []):
                for span in sec.get('spans', []):
                    if span.get('text'):
                        parts.append(span['text'])
                parts.append('\n')
            return ''.join(parts).strip()
        except Exception:
            return json_str
 
    def author_display_name(author_dict):
        names = author_dict.get('names', [])
        if names:
            n = names[0]
            return f"{n.get('givenName', '')} {n.get('familyName', '')}".strip()
        return None
 
    result = {'question': {}, 'answers': [], 'answer_count': None, 'related_questions': []}
 
    for payload in payloads:
        data = payload.get('data', payload)
 
        # Question metadata — keyed by presence of 'qid' inside 'question'
        if 'question' in data and isinstance(data['question'], dict):
            q = data['question']
            if q.get('qid') and not result['question']:
                result['question'] = {
                    'qid':   q.get('qid'),
                    'id':    q.get('id'),
                    'title': spans_to_text(q.get('title', '')),
                    'url':   q.get('url'),
                    'slug':  q.get('slug'),
                    'topics': [t['name'] for t in q.get('navigationTopics', [])],
                }
 
        # Total answer count — keyed by 'answerCount'
        if 'answerCount' in data:
            result['answer_count'] = data['answerCount']
            rq = (data.get('bottomRelatedQuestionsInfo') or {}).get('relatedQuestions', [])
            result['related_questions'] = [spans_to_text(r['title']) for r in rq]
 
        # Answer nodes — keyed by node.__typename == 'QuestionAnswerItem2'
        node = data.get('node', {})
        if isinstance(node, dict) and node.get('__typename') == 'QuestionAnswerItem2':
            answer = node.get('answer', {})
            if answer.get('aid'):
                a_author = answer.get('author') or {}
                cred = answer.get('authorCredential') or {}
                result['answers'].append({
                    'aid':              answer.get('aid'),
                    'index':            node.get('index'),
                    'author_name':      author_display_name(a_author),
                    'author_profile':   a_author.get('profileUrl'),
                    'author_uid':       a_author.get('uid'),
                    'author_credential': cred.get('translatedString'),
                    'num_upvotes':      answer.get('numUpvotes'),
                    'num_views':        answer.get('numViews'),
                    'num_shares':       answer.get('numShares'),
                    'num_comments':     answer.get('numDisplayComments'),
                    'creation_time_us': answer.get('creationTime'),  # microseconds since epoch
                    'viewer_has_access': answer.get('viewerHasAccess'),
                    'perma_url':        answer.get('permaUrl'),
                    'text':             spans_to_text(answer.get('content', '{}')),
                })
 
    return result

Example output

result = quora_question("https://www.quora.com/What-is-the-meaning-of-life")
 
# result['question']:
# {
#   'qid': 2861,
#   'id': 'UXVlc3Rpb25AMDoyODYx',
#   'title': 'What is the meaning of life?',
#   'url': '/What-is-the-meaning-of-life',
#   'slug': 'What-is-the-meaning-of-life',
#   'topics': ['Philosophy', 'The Big Unanswered Questions', 'Meaning of Life', ...]
# }
 
# result['answer_count']:  413
 
# result['answers'][0]:
# {
#   'aid': 2779675,
#   'index': 1,
#   'author_name': 'Shubhankar Srivastava',
#   'author_profile': '/profile/Shubhankar-Srivastava',
#   'author_uid': 5381038,
#   'author_credential': 'works at D. E. Shaw',
#   'num_upvotes': 589,
#   'num_views': 24085,
#   'num_shares': 0,
#   'num_comments': 8,
#   'creation_time_us': 1373364681312036,   # divide by 1e6 for seconds
#   'viewer_has_access': True,
#   'perma_url': '/What-is-the-meaning-of-life/answer/Shubhankar-Srivastava',
#   'text': 'Every morning in Africa, a deer wakes up...'
# }

Convert creation_time_us to datetime

from datetime import datetime, timezone
ts_sec = result['answers'][0]['creation_time_us'] / 1_000_000
dt = datetime.fromtimestamp(ts_sec, tz=timezone.utc)
# datetime(2013, 7, 9, 9, 31, 21, tzinfo=timezone.utc)

Path 2: Single answer permalink

Fetching quora.com/{question-slug}/answer/{author-slug} directly returns only that one answer's full data in 3 payloads instead of 16. Use this when you already know the answer URL.

def quora_answer(answer_url):
    """
    Fetch a single answer by its permalink.
    URL format: https://www.quora.com/{question-slug}/answer/{author-profile-slug}
    Returns answer dict with: aid, num_upvotes, num_views, text, author info.
    """
    html = quora_get(answer_url)
    payloads = extract_quora_payloads(html)
 
    for payload in payloads:
        data = payload.get('data', {})
        if 'answer' in data and isinstance(data['answer'], dict):
            a = data['answer']
            author = a.get('author') or {}
            names = author.get('names', [{}])
            n = names[0] if names else {}
            return {
                'aid':          a.get('aid'),
                'num_upvotes':  a.get('numUpvotes'),
                'num_views':    a.get('numViews'),
                'author_name':  f"{n.get('givenName','')} {n.get('familyName','')}".strip(),
                'author_uid':   author.get('uid'),
                'text':         _spans_to_text(a.get('content', '{}')),
            }
    return {}
 
# Example:
# quora_answer("https://www.quora.com/What-is-the-meaning-of-life/answer/Pararth-Shah")
# -> {'aid': 4734237, 'num_upvotes': 234, 'num_views': 100643, 'author_name': 'Pararth Shah', ...}

Path 3: Topic page

def quora_topic(topic_url):
    """
    Fetch topic metadata from a Quora topic page.
    URL format: https://www.quora.com/topic/{topic-slug}
    Returns: tid, name, num_followers, url, is_following, has_leaderboard.
    NOTE: The topic page itself only renders topic metadata, NOT the question feed.
    Question feed requires browser (XHR-loaded via React).
    """
    html = quora_get(topic_url)
    payloads = extract_quora_payloads(html)
 
    for payload in payloads:
        data = payload.get('data', {})
        if 'topic' in data and isinstance(data['topic'], dict):
            t = data['topic']
            return {
                'tid':           t.get('tid'),
                'id':            t.get('id'),
                'name':          t.get('name'),
                'url':           t.get('url'),
                'num_followers': t.get('numFollowers'),
                'is_following':  t.get('isFollowing'),
                'has_leaderboard': t.get('hasLeaderboard'),
                'photo_url':     t.get('photoUrl'),
                'is_locked':     t.get('isLocked'),
            }
    return {}
 
# Example:
# quora_topic("https://www.quora.com/topic/Python-programming-language")
# -> {'tid': 13292, 'name': 'Python Programming Language', 'num_followers': 10, ...}

Path 4: User profile page

def quora_profile(profile_url):
    """
    Fetch user profile data from https://www.quora.com/profile/{username}
    Returns: uid, name, credential, follower_count, following_count, profile_image_url.
    """
    html = quora_get(profile_url)
    payloads = extract_quora_payloads(html)
 
    for payload in payloads:
        data = payload.get('data', {})
        if 'user' in data and isinstance(data['user'], dict):
            u = data['user']
            names = u.get('names', [{}])
            n = names[0] if names else {}
            cred = u.get('profileCredential') or {}
            return {
                'uid':             u.get('uid'),
                'id':              u.get('id'),
                'name':            f"{n.get('givenName','')} {n.get('familyName','')}".strip(),
                'profile_url':     u.get('profileUrl'),
                'follower_count':  u.get('followerCount'),
                'following_count': u.get('followingCount'),
                'profile_image':   u.get('profileImageUrl'),
                'credential':      cred.get('experience'),
                'is_verified':     u.get('isVerified'),
                'is_anon':         u.get('isAnon'),
                'is_ai_account':   u.get('isAiAccount'),
                'deactivated':     u.get('deactivated'),
            }
    return {}
 
# Example:
# quora_profile("https://www.quora.com/profile/Pararth-Shah")
# -> {'uid': 4683832, 'name': 'Pararth Shah', 'follower_count': 5154,
#     'following_count': 83, 'credential': 'Unfinished symphony.', ...}

Gotchas

Bare Mozilla/5.0 UA returns HTTP 403 - Always use a full Chrome or Firefox UA string. The default http_get helper's "User-Agent": "Mozilla/5.0" will be blocked. Do not use http_get directly; use the quora_get wrapper above.
Googlebot UA returns HTTP 403 - Quora blocks crawler UAs. Only real browser UAs work.
Double JSON encoding - Each .push() argument is a JavaScript string literal containing JSON. To parse: first json.loads('"' + raw + '"') to decode the JS string escaping (converts \\" to "), then json.loads(inner) to parse the actual JSON object. Skipping either step produces parse errors.
All text fields are serialized span objects - question.title, answer.content, user.descriptionQtextDocument.legacyJson, etc. are all JSON strings containing a {"sections": [{"spans": [...]}]} document, not plain text. Always parse through spans_to_text().
Question page only SSR's the first ~3 answers - The answers list in the result will contain at most 3 entries (the top-ranked answers). The answer_count field shows the true total (e.g. 413). To get more answers you need browser-based XHR pagination (Quora sends additional answers via GraphQL calls that require session auth in practice).
viewer_has_access: false still includes full content - Even when viewerHasAccess is False (answers from Quora+ Spaces / tribe-only content), the content field is still present in the SSR payload and the full text is readable. The flag only controls client-side gating in the browser.
creation_time_us is microseconds, not milliseconds - Divide by 1_000_000 (not 1_000) to get a Unix timestamp in seconds. Confirmed: 1373364681312036 / 1e6 = 1373364681.3 (July 2013).
numFollowers on topic pages may be 0 even for major topics - The field reflects the logged-in user's follow state for some topics and appears to undercount. Treat as approximate.
Search pages do not yield result data - https://www.quora.com/search?q=... returns 3 payloads with viewer/network info only - no search results in the SSR payload. Search results are loaded client-side and are not accessible via http_get.
Profile pages do not include the user's answer list - The profile page SSR payload returns user metadata only. The list of a user's answers is loaded via XHR pagination. To get answers for a specific question, use the question URL directly.
IDs are base64-encoded Relay global IDs - id: "UXVlc3Rpb25AMDoyODYx" decodes to "Question@0:2861". The numeric qid/uid/aid/tid fields are more useful for constructing URLs and deduplication. Use qid and aid as stable identifiers.
permaUrl may be an absolute URL for Spaces answers - Most answers have permaUrl: "/Question-slug/answer/Author-Name" (relative). Answers posted in a Quora Space have a full absolute URL like "https://spacename.quora.com/Question-slug". Handle both forms.
No public REST or GraphQL API - Quora's internal graphql/gql_para_POST endpoint requires a valid quora-formkey header derived from the session, making it inaccessible without a real authenticated session. The SSR push-payload approach is the only reliable unauthenticated path.

scraping.md · updated Apr 28, 2026 · 1,770 words

Sites change. These notes describe what worked when they were last updated, not a guarantee they still do. If something broke, the fix is a pull request to browser-harness.