ScrapeThis API Docs

Jobs

Start, list, inspect, and cancel spider jobs.

GET /api/v1/jobs

List jobs for the authenticated user. Supports filtering by status and spider.

Parameters

Name Type Description
status string Filter by status (pending, running, finished, cancelled, error)
spider_id string Filter by spider UUID
limit number Max results (default: 50)

Example Request

cURL 
"text">-green-400">curl "https://scrapethis.app/api/v1/jobs?status=running&limit=10" \
  "text-cyan-400">-H "Authorization: Bearer YOUR_API_KEY"

Response

200 OK 
[
  {
    "id": "job-uuid",
    "spider_id": "spider-uuid",
    "status": "running",
    "user_id": "user-uuid",
    "created_at": "2025-01-15T10: 30: 00Z",
    "spider": {
      "id": "spider-uuid",
      "name": "my-spider",
      "repo_path": "spiders/my_spider.py"
    }
  }
]

Error Responses

401 Invalid or missing API key
POST /api/v1/jobs

Start a new job for a spider. Provide either spider_id or spider_name.

Request Body

TypeScript 
{
  spider_id?: "text-purple-400">string;      // UUID of the spider
  spider_name?: "text-purple-400">string;    // or look up by name
  arguments?: object;      // optional spider arguments
}

Example Request

cURL 
"text">-green-400">curl "text-cyan-400">-X POST https://scrapethis.app/api/v1/jobs \
  "text-cyan-400">-H "Authorization: Bearer YOUR_API_KEY" \
  "text-cyan-400">-H "Content">-Type: application/json" \
  "text-cyan-400">-d '{"spider_name": "my">-spider", "arguments": {"url": "https://example.com"}}'

Response

200 OK 
{
  "jobId": "new-job-uuid",
  "status": "pending"
}

Error Responses

400 Missing spider_id or spider_name
404 Spider not found
GET /api/v1/jobs/{id}

Get details for a specific job, including spider information.

Example Request

cURL 
"text">-green-400">curl https://scrapethis.app/api/v1/jobs/JOB_ID \
  "text-cyan-400">-H "Authorization: Bearer YOUR_API_KEY"

Response

200 OK 
{
  "id": "job-uuid",
  "spider_id": "spider-uuid",
  "status": "finished",
  "items_scraped": 142,
  "created_at": "2025-01-15T10: 30: 00Z",
  "finished_at": "2025-01-15T10: 35: 00Z",
  "spider": {
    "id": "spider-uuid",
    "name": "my-spider"
  }
}

Error Responses

404 Job not found
DELETE /api/v1/jobs/{id}

Cancel a running job.

Example Request

cURL 
"text">-green-400">curl "text-cyan-400">-X DELETE https://scrapethis.app/api/v1/jobs/JOB_ID \
  "text-cyan-400">-H "Authorization: Bearer YOUR_API_KEY"

Response

200 OK 
{
  "ok": true,
  "message": "Job cancelled"
}

Error Responses

404 Job not found
500 Failed to cancel job