POST /v1/batch

Scrape up to 100 URLs in a single request. Results are returned in the same order as the input array. Failed URLs are included in the response with ok: false and an error field, the whole batch never fails because of individual URL errors.

The options.concurrency you request is clamped to a per-plan cap for this endpoint (and to a hard ceiling of 20 regardless of plan):

Plan	`/v1/batch` worker cap
Free	2
Hobby	5
Pro	10
Scale	20

These are batch workers, not the account-wide concurrency figure quoted on Pricing. Batch is the only endpoint that clamps concurrency today.

Request

POST /v1/batch
Content-Type: application/json
x-api-key: YOUR_KEY

Body

Field	Type	Required	Default	Description
`urls`	string[]	Yes		URLs to scrape. Must be a non-empty array of valid `http://` or `https://` URLs, max 100.
`options.extractListing`	boolean	No	`true`	Run the listing extraction pipeline on each page.
`options.includeHtml`	boolean	No	`false`	Include raw HTML in each result. Increases response size.
`options.cookies`	string	No		Cookie string forwarded to every request (Netscape or JSON format).
`options.headers`	object	No		Custom HTTP headers forwarded to every request.
`options.concurrency`	number	No	`5`	Parallel fetch limit. Clamped to your tier cap (see table above).

Example request

curl -X POST https://api.superscraper.dev/v1/batch \
  -H "x-api-key: ss_live_xxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://www.yelp.com/biz/acme-plumbing-austin",
      "https://www.yelp.com/biz/lone-star-electric-dallas",
      "https://www.angi.com/companylist/us/tx/austin/plumbers.htm"
    ],
    "options": {
      "extractListing": true,
      "concurrency": 3
    }
  }'

Response

HTTP 200 on success. The response header X-Batch-Summary summarises the run in a compact format: ok=N;failed=M;latencyMs=X.

Body

Field	Type	Description
`results`	array	One entry per URL in input order (see result item below)
`summary.total`	number	Total URLs processed
`summary.ok`	number	Number of successful scrapes
`summary.failed`	number	Number of failed scrapes
`summary.latencyMs`	number	Wall-clock time for the entire batch in milliseconds

Result item

Field	Type	Description
`url`	string	The URL that was scraped
`ok`	boolean	`true` if the page was fetched successfully
`markdown`	string \| null	Page content as markdown
`html`	string \| undefined	Raw HTML (only present when `options.includeHtml: true`)
`listing`	object \| null	Structured listing data (same shape as `/v1/scrape`)
`statusCode`	number \| null	HTTP status code from the fetch
`error`	string \| undefined	Error message when `ok` is `false`
`metadata.fetchMethod`	string	How the page was fetched: `"plain"`, `"playwright"`, or `"stealth"`
`metadata.latencyMs`	number	Fetch time for this individual URL

Example response

{
  "results": [
    {
      "url": "https://www.yelp.com/biz/acme-plumbing-austin",
      "ok": true,
      "markdown": "# Acme Plumbing\n\n**4.8** ★ ...",
      "listing": {
        "name": "Acme Plumbing",
        "phone": "(512) 555-0187",
        "address": "1402 S 1st St",
        "city": "Austin",
        "state": "TX",
        "zip": "78704",
        "rating": 4.8,
        "review_count": 214,
        "_confidence": 0.92,
        "_extraction_method": "json-ld"
      },
      "statusCode": 200,
      "metadata": {
        "fetchMethod": "plain",
        "latencyMs": 412
      }
    },
    {
      "url": "https://www.yelp.com/biz/lone-star-electric-dallas",
      "ok": true,
      "markdown": "# Lone Star Electric\n\n**4.6** ★ ...",
      "listing": { "name": "Lone Star Electric", "rating": 4.6, "_confidence": 0.88, "_extraction_method": "json-ld" },
      "statusCode": 200,
      "metadata": { "fetchMethod": "plain", "latencyMs": 389 }
    },
    {
      "url": "https://www.angi.com/companylist/us/tx/austin/plumbers.htm",
      "ok": false,
      "markdown": null,
      "listing": null,
      "statusCode": 403,
      "error": "Fetch failed: HTTP 403",
      "metadata": { "fetchMethod": "plain", "latencyMs": 211 }
    }
  ],
  "summary": {
    "total": 3,
    "ok": 2,
    "failed": 1,
    "latencyMs": 847
  }
}

Errors

Status	Error	Cause
400	`urls` must be a non-empty array	`urls` field missing, not an array, or empty
400	`urls` may contain at most 100 entries	Array length exceeds 100
400	`Invalid URL(s)` + `invalidUrls` array	One or more entries are not valid URLs
400	`Invalid JSON body`	Malformed request body
401	`API key required`	Missing `x-api-key` header
429	`Rate limit exceeded`	Too many requests in the sliding window

Notes

Each successfully scraped URL consumes one credit, same as /v1/scrape. Failed URLs are free.
Concurrency is capped by your plan tier.
For more than 100 URLs, pass async: true (or just send more than 100) to run the batch as a job: the call returns 202 with a jobId you poll at GET /v1/jobs/:id. Async batches accept up to 10,000 URLs.

Map Brand