Sync vs Async

ScrapeDrive offers two modes for scraping. Both accept the same parameters — the only difference is how you get the result back.

Sync

GET https://sync.scrapedrive.com/api/v1/scrape

• Returns the raw content directly in the response body
• Max timeout: 95 seconds (default 60s)
• Use for: simple scrapes, real-time needs

curl "https://sync.scrapedrive.com/api/v1/scrape?api_key=YOUR_KEY&url=https://example.com"

Async

POST https://api.scrapedrive.com/api/v1/scrape/async

• Returns a job ID immediately (under 5ms)
• Poll for results when the job completes
• Max timeout: 130 seconds
• Use for: long-running scrapes, batch jobs, heavy JS sites

Async Flow

1. Submit the job:

curl -X POST https://api.scrapedrive.com/api/v1/scrape/async \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_KEY",
    "url": "https://example.com",
    "render_js": true
  }'

Response:

{
  "id": "01KH2CTXRJ44SD5XJ220BA0XRZ",
  "status": "queued",
  "url": "https://example.com",
  "status_url": "https://api.scrapedrive.com/api/v1/job/01KH2CTXRJ44SD5XJ220BA0XRZ"
}

2. Poll for results:

curl "https://api.scrapedrive.com/api/v1/job/01KH2CTXRJ44SD5XJ220BA0XRZ"

The job transitions from queued → processing → completed. Poll until status is completed.