Social Media

Reddit Posts & Comments

Public Reddit posts and threaded comments from chosen subreddits or keywords.

reddit-postsredditsocial-mediapostscommentsmonitoringkeywords

0 rows in the dataset

Download the dataset

Get every row this scraper has collected as one CSV or JSONL file. If the data is older than maxAgeHours, the scraper is re-run with its configured settings first. The key needs the datasets:export scope.

DOWNLOAD_ID=$(curl -s -X POST "https://api.pandorlabs.com/v1/datasets/reddit-posts/downloads" \  -H "Authorization: Bearer $PANDORLABS_KEY" \  -H "Content-Type: application/json" \  -d '{"maxAgeHours":24,"format":"csv"}' | jq -r '.id')while true; do  STATUS=$(curl -s "https://api.pandorlabs.com/v1/downloads/$DOWNLOAD_ID" \    -H "Authorization: Bearer $PANDORLABS_KEY" | jq -r '.status')  [ "$STATUS" != "refreshing" ] && break  sleep 5donecurl -s "https://api.pandorlabs.com/v1/downloads/$DOWNLOAD_ID/file" \  -H "Authorization: Bearer $PANDORLABS_KEY" \  -o dataset.csv

The rate is 1 credit per 100 rows cached and 2 credits live, charged on the rows in the dataset when the download becomes ready. A failed refresh is not charged. The file stays downloadable for 24 hours. See pricing.

Start a run

To choose the targets yourself, send the parameters below as input. The call returns 202 with the run; the key needs the scrapers:run scope and the account a positive credit balance.

curl -X POST "https://api.pandorlabs.com/v1/scrapers/reddit-posts/runs" \  -H "Authorization: Bearer $PANDORLABS_KEY" \  -H "Content-Type: application/json" \  -d '{    "input": {      "subreddits": [        "BuyItForLife",        "ProductReviews",        "GoodValue",        "SaaS",        "devops",        "webscraping",        "dataengineering",        "AskReddit",        "technology",        "gadgets"      ],      "keywords": [        "need a website",        "looking for a developer"      ],      "timeFilter": "week",      "maxAgeDays": 0    }  }'

A run is billed at the live rate once it finishes: 2 credits per 100 rows it collected. A run that collects nothing costs nothing.

Parameters

Every parameter is optional. Anything you leave out uses the default shown here; an unknown name or an invalid value is rejected with a 400 that lists each problem.

Targeting

subreddits
string[]
Subreddits
Names only — r/ prefixes and pasted URLs are cleaned up. Use "all" to search the whole site instead.
Default
["BuyItForLife","ProductReviews","GoodValue","SaaS","devops","webscraping","dataengineering","AskReddit","technology","gadgets"]
Example
forhire, webdev, smallbusiness
keywords
string[]
Keywords
Any keyword switches the run to Reddit's search. Single words match whole words; phrases match anywhere.
Default
none
Example
need a website, looking for a developer
timeFilter
enum
Time window
How far back the search reaches.
Default
week
Allowed values
hour
day
week
month
year
all
maxAgeDays
number
Max age (days)
Drop posts older than this. 0 keeps everything the listing returns.
Default
0
Range
0 to 3650

Matching

matchIn
enum
Match keywords in
Default
title_body
Allowed values
title_body
title
body
requireAll
boolean
Require every keyword
Off: a post matching any keyword is kept.
Default
false
keywordFilterEnabled
boolean
Filter results by keyword
Off: keep everything Reddit's search returns, without re-checking it locally.
Default
true
minScore
number
Minimum score
Default
0
Range
at least -1000
minComments
number
Minimum comments
Default
0
Range
at least 0

Collection

sorts
enum[]
Listing sorts
Used when no keywords are set.
Default
["new","hot"]
Allowed values
new
hot
top
rising
controversial
searchSort
enum
Search sort
Used when keywords are set. "new" is the right choice for monitoring.
Default
new
Allowed values
new
relevance
top
comments
maxPages
number
Pages per query
100 posts per page.
Default
3
Range
1 to 10
includeComments
boolean
Collect comment threads
Off makes a run dramatically faster — one request per post is skipped.
Default
true
maxCommentsPerPost
number
Max comments per post
Default
500
Range
0 to 5000
concurrency
number
Concurrent subreddits
Raising this earns 429s faster than it collects.
Default
4
Range
1 to 16
requestDelayMs
number
Delay between requests (ms)
Default
1000
Range
0 to 30000
transport
enum
Transport
Browser fetches the same JSON from inside a rendered Reddit page via obscura. Direct HTTP is answered with a block page by Reddit's edge and is kept only for Reddit-shaped endpoints that are not behind it.
Default
browser
Allowed values
browser
http

Input as JSON Schema

The same schema the API validates against. Use it as a tool input schema or to generate a typed client.

inputSchema

{  "type": "object",  "additionalProperties": false,  "properties": {    "subreddits": {      "type": "array",      "items": {        "type": "string"      },      "maxItems": 1000,      "title": "Subreddits",      "description": "Names only — r/ prefixes and pasted URLs are cleaned up. Use \"all\" to search the whole site instead.",      "default": [        "BuyItForLife",        "ProductReviews",        "GoodValue",        "SaaS",        "devops",        "webscraping",        "dataengineering",        "AskReddit",        "technology",        "gadgets"      ],      "examples": [        "forhire, webdev, smallbusiness"      ]    },    "keywords": {      "type": "array",      "items": {        "type": "string"      },      "maxItems": 1000,      "title": "Keywords",      "description": "Any keyword switches the run to Reddit's search. Single words match whole words; phrases match anywhere.",      "default": [],      "examples": [        "need a website, looking for a developer"      ]    },    "timeFilter": {      "type": "string",      "enum": [        "hour",        "day",        "week",        "month",        "year",        "all"      ],      "title": "Time window",      "description": "How far back the search reaches.",      "default": "week"    },    "maxAgeDays": {      "type": "number",      "minimum": 0,      "maximum": 3650,      "title": "Max age (days)",      "description": "Drop posts older than this. 0 keeps everything the listing returns.",      "default": 0    },    "matchIn": {      "type": "string",      "enum": [        "title_body",        "title",        "body"      ],      "title": "Match keywords in",      "default": "title_body"    },    "requireAll": {      "type": "boolean",      "title": "Require every keyword",      "description": "Off: a post matching any keyword is kept.",      "default": false    },    "keywordFilterEnabled": {      "type": "boolean",      "title": "Filter results by keyword",      "description": "Off: keep everything Reddit's search returns, without re-checking it locally.",      "default": true    },    "minScore": {      "type": "number",      "minimum": -1000,      "title": "Minimum score",      "default": 0    },    "minComments": {      "type": "number",      "minimum": 0,      "title": "Minimum comments",      "default": 0    },    "sorts": {      "type": "array",      "items": {        "type": "string",        "enum": [          "new",          "hot",          "top",          "rising",          "controversial"        ]      },      "uniqueItems": true,      "title": "Listing sorts",      "description": "Used when no keywords are set.",      "default": [        "new",        "hot"      ]    },    "searchSort": {      "type": "string",      "enum": [        "new",        "relevance",        "top",        "comments"      ],      "title": "Search sort",      "description": "Used when keywords are set. \"new\" is the right choice for monitoring.",      "default": "new"    },    "maxPages": {      "type": "number",      "minimum": 1,      "maximum": 10,      "title": "Pages per query",      "description": "100 posts per page.",      "default": 3    },    "includeComments": {      "type": "boolean",      "title": "Collect comment threads",      "description": "Off makes a run dramatically faster — one request per post is skipped.",      "default": true    },    "maxCommentsPerPost": {      "type": "number",      "minimum": 0,      "maximum": 5000,      "title": "Max comments per post",      "default": 500    },    "concurrency": {      "type": "number",      "minimum": 1,      "maximum": 16,      "title": "Concurrent subreddits",      "description": "Raising this earns 429s faster than it collects.",      "default": 4    },    "requestDelayMs": {      "type": "number",      "minimum": 0,      "maximum": 30000,      "title": "Delay between requests (ms)",      "default": 1000    },    "transport": {      "type": "string",      "enum": [        "browser",        "http"      ],      "title": "Transport",      "description": "Browser fetches the same JSON from inside a rendered Reddit page via obscura. Direct HTTP is answered with a block page by Reddit's edge and is kept only for Reddit-shaped endpoints that are not behind it.",      "default": "browser"    }  }}

Run, poll, read rows

The run is asynchronous. Poll it until it reaches a terminal status and its billing is paid, then page through its rows. Until then the rows endpoint answers 409 (still queued or running) or 402 (payment required).

RUN_ID=$(curl -s -X POST "https://api.pandorlabs.com/v1/scrapers/reddit-posts/runs" \  -H "Authorization: Bearer $PANDORLABS_KEY" \  -H "Content-Type: application/json" \  -d '{    "input": {      "subreddits": [        "BuyItForLife",        "ProductReviews",        "GoodValue",        "SaaS",        "devops",        "webscraping",        "dataengineering",        "AskReddit",        "technology",        "gadgets"      ],      "keywords": [        "need a website",        "looking for a developer"      ],      "timeFilter": "week",      "maxAgeDays": 0    }  }' | jq -r '.id')while true; do  RUN=$(curl -s "https://api.pandorlabs.com/v1/runs/$RUN_ID" \    -H "Authorization: Bearer $PANDORLABS_KEY")  STATUS=$(echo "$RUN" | jq -r '.status')  BILLING=$(echo "$RUN" | jq -r '.billing.status')  case "$STATUS" in queued|running) ;; *) [ "$BILLING" != "pending" ] && break ;; esac  sleep 5doneecho "$RUN" | jq '{status, rowCount, billing}'curl -s "https://api.pandorlabs.com/v1/runs/$RUN_ID/rows?page=1&limit=100" \  -H "Authorization: Bearer $PANDORLABS_KEY"

See the API reference for pagination, cancelling and listing runs.

Output fields

Rows have no fixed schema; these are the fields observed in the rows collected so far. Coverage is the share of rows where the field is present.

No rows collected yet, so no fields have been observed. Run the scraper once and they will appear here.