Auto mode

Use Auto mode when you want one Job to progress through compatible request settings instead of manually submitting a new request after each failure. max_credits is required and limits the final charge:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com" \
  --data-urlencode "auto=true" \
  --data-urlencode "max_credits=20"

Auto starts with a compatible route selected for the target and advances serially only after failure. ScrapeDrive manages HTTP-versus-browser selection, proxy routing, location, and attempt identity. Result format, screenshots, and browser wait requirements stay fixed. ScrapeDrive reserves the highest compatible price reachable within the ceiling and charges the successful result configuration once; failed internal attempts are not charged again.

Do not combine Auto mode with render_js, proxy_pool, proxy_country, custom_proxy, or session_number. Use a Standard Job when you need to control those fields. In Auto, a screenshot or browser wait tells ScrapeDrive that the result requires a browser-compatible route.

Dynamic pages and waits

For a Standard Job, set render_js=true when the content appears only after the page runs JavaScript. Rendering is off by default. Auto Jobs must omit render_js because Auto chooses between HTTP and browser acquisition.

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/app" \
  --data-urlencode "render_js=true"

When browser rendering is active, waits run in this order:

1. wait_browser
2. wait_for
3. wait_ms

ScrapeDrive chooses the browser engine. In Standard mode, you choose whether the request starts with JavaScript rendering. In Auto mode, send a wait without render_js; that wait becomes a browser requirement for the result.

Navigation readiness

wait_browser chooses when navigation is considered ready:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/app" \
  --data-urlencode "render_js=true" \
  --data-urlencode "wait_browser=networkidle"

Wait for an element

wait_for accepts a CSS selector and waits up to 15 seconds for it to appear:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/products" \
  --data-urlencode "render_js=true" \
  --data-urlencode "wait_for=.product-list"

Add a fixed delay

wait_ms adds 0–30,000 milliseconds after the earlier wait steps. Use it for a short animation or transition that finishes after the target element appears:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/products" \
  --data-urlencode "render_js=true" \
  --data-urlencode "wait_for=.product-list" \
  --data-urlencode "wait_ms=1000"

Long fixed waits hold a browser slot and reduce throughput. Prefer wait_browser or wait_for when the page provides a reliable readiness signal.

Resource and ad blocking

For browser executions, including browser steps selected by Auto, these options are enabled by default:

• block_resources=true skips images, CSS, and fonts to load faster and use less bandwidth.
• block_ads=true skips advertisements and tracker scripts.

They are independent. For example, you can keep visual resources while still blocking ads:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com" \
  --data-urlencode "render_js=true" \
  --data-urlencode "block_resources=false" \
  --data-urlencode "block_ads=true"

Neither option affects an HTTP-only execution. An HTML-only request does not download page images, styles, or fonts in the first place.

Screenshots

Use one of these options:

Every screenshot option adds 5 credits. It requires a browser-compatible result and forces block_resources=false so the visual page can load. A selector capture returns an error when the element is not found.

device_type=desktop is the default. Set device_type=mobile when you need the page's mobile layout and user agent.

Routing and identity

The routing controls in this section apply to Standard Jobs (auto=false). Auto mode manages routing and attempt identity for you.

Country targeting

Set proxy_pool=residential and provide a two-letter ISO 3166-1 country code:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com" \
  --data-urlencode "proxy_pool=residential" \
  --data-urlencode "proxy_country=US"

Country targeting is rejected with the Datacenter pool or a custom proxy.

Sticky sessions

Reuse the same 6–32 character alphanumeric session_number across related requests when the target should see a consistent exit IP:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/page/1" \
  --data-urlencode "session_number=cartA92"

Your own proxy

custom_proxy replaces the selected ScrapeDrive proxy network for the initial Standard attempt:

curl "https://sync.scrapedrive.com/api/v1/scrape" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_KEY",
    "url": "https://example.com",
    "custom_proxy": "https://username:password@proxy.example.com:8443"
  }'

If that attempt cannot deliver the page, ScrapeDrive may use a managed route during recovery. Use a sticky session_number when keeping the same identity across Standard retries is the requirement.

The format is http(s)://username:password@host:port. Prefer a request body so proxy credentials do not appear in the URL.

Forward headers to the target

Enable forward_sdrive_headers, prefix each target header with sdrive-, and ScrapeDrive removes that prefix before forwarding it:

curl --get "https://sync.scrapedrive.com/api/v1/scrape" \
  --data-urlencode "api_key=YOUR_KEY" \
  --data-urlencode "url=https://example.com/private" \
  --data-urlencode "forward_sdrive_headers=true" \
  -H "sdrive-Authorization: Bearer target-site-token" \
  -H "sdrive-X-Custom: custom-value"

The target receives an Authorization header whose value is Bearer target-site-token, plus X-Custom: custom-value. This does not replace the ScrapeDrive api_key; it authenticates or customizes the request sent to the target website.

Timeouts

timeout_ms must be at least 10,000 and is capped by the selected delivery mode:

The timeout controls the scrape itself. Async delivery returns the job id immediately while the job continues in the background.

Webhooks

Async jobs can send their completed result to an HTTPS webhook_url. Add custom_id when you want your own reference echoed in that callback.

See Sync vs Async for an example.