More endpoints
GET /v1/jobs

GET /v1/jobs

Retrieve the status and results of async crawl jobs created by POST /v1/crawl.

Two endpoints are available:

EndpointDescription
GET /v1/jobs/:idGet a specific job by ID
GET /v1/jobsList the 20 most recent jobs for your account

Jobs are tenant-isolated: you can only retrieve jobs created with your API key.

GET /v1/jobs/:id

Request

GET /v1/jobs/crawl_8f3a12bc
x-api-key: YOUR_KEY

Path parameters

ParameterTypeDescription
idstringJob ID returned by POST /v1/crawl

Example request

curl https://api.superscraper.dev/v1/jobs/crawl_8f3a12bc \
  -H "x-api-key: ss_live_xxxxxxxxxxxxxxxxxxxx"

Response

HTTP 200 on success, 404 if the job does not exist or belongs to a different tenant.

Body

FieldTypeDescription
idstringJob ID
urlstringThe seed URL that was crawled
statusstringJob status: "queued", "running", "done", "failed"
result_urlstring | nullURL to download full results when status is "completed"
pages_scrapednumberNumber of pages scraped so far
errorstring | nullError message when status is "failed"
created_atstringISO 8601 timestamp when the job was created
completed_atstring | nullISO 8601 timestamp when the job finished

Example response, in progress

{
  "id": "crawl_8f3a12bc",
  "url": "https://example.com",
  "status": "running",
  "result_url": null,
  "pages_scraped": 14,
  "error": null,
  "created_at": "2026-06-01T18:30:00.000Z",
  "completed_at": null
}

Example response, completed

{
  "id": "crawl_8f3a12bc",
  "url": "https://example.com",
  "status": "done",
  "result_url": "https://cdn.superscraper.dev/results/crawl_8f3a12bc.json",
  "pages_scraped": 50,
  "error": null,
  "created_at": "2026-06-01T18:30:00.000Z",
  "completed_at": "2026-06-01T18:31:44.000Z"
}

Example response, failed

{
  "id": "crawl_8f3a12bc",
  "url": "https://example.com",
  "status": "failed",
  "result_url": null,
  "pages_scraped": 3,
  "error": "Seed URL returned HTTP 404",
  "created_at": "2026-06-01T18:30:00.000Z",
  "completed_at": "2026-06-01T18:30:08.000Z"
}

GET /v1/jobs

List the 20 most recent crawl jobs for your account.

Request

GET /v1/jobs
x-api-key: YOUR_KEY

Example request

curl https://api.superscraper.dev/v1/jobs \
  -H "x-api-key: ss_live_xxxxxxxxxxxxxxxxxxxx"

Response

HTTP 200 on success.

Body

FieldTypeDescription
jobsarrayList of job summaries (see below), ordered by created_at descending

Job summary item

FieldTypeDescription
idstringJob ID
urlstringSeed URL
statusstringJob status: "queued", "running", "done", "failed"
pages_scrapednumberPages scraped
created_atstringISO 8601 creation timestamp
completed_atstring | nullISO 8601 completion timestamp

Example response

{
  "jobs": [
    {
      "id": "crawl_8f3a12bc",
      "url": "https://example.com",
      "status": "done",
      "pages_scraped": 50,
      "created_at": "2026-06-01T18:30:00.000Z",
      "completed_at": "2026-06-01T18:31:44.000Z"
    },
    {
      "id": "crawl_7d2e09ab",
      "url": "https://another-site.com",
      "status": "queued",
      "pages_scraped": 0,
      "created_at": "2026-06-01T17:15:00.000Z",
      "completed_at": null
    }
  ]
}

Errors

StatusErrorCause
401API key requiredMissing x-api-key header
401Invalid or revoked API key.Key not found or deactivated
404Job not foundJob ID does not exist or belongs to a different tenant
500Failed to fetch jobsDatabase error on the list endpoint

Polling pattern

Poll the job status every few seconds until status is "done" or "failed":

# Poll every 3 seconds
while true; do
  STATUS=$(curl -s https://api.superscraper.dev/v1/jobs/crawl_8f3a12bc \
    -H "x-api-key: ss_live_xxxxxxxxxxxxxxxxxxxx" | jq -r '.status')
  echo "Status: $STATUS"
  [ "$STATUS" = "done" ] || [ "$STATUS" = "failed" ] && break
  sleep 3
done
AI Agents OverviewPOST /v1/parse