API Documentation
Welcome to the W3Crawler API documentation. Our REST API allows you to programmatically access our web scraping and data extraction services.
REST API
JSON-based REST API with predictable URLs
Secure
HTTPS only, Bearer token authentication
Rate Limited
Fair usage with clear rate limits
Base URL
https://api.w3crawler.comAuthentication
All API requests require authentication using a Bearer token. You can get your API key from the dashboard after logging in.
const response = await fetch('https://api.w3crawler.com/api/scrapers/ecommerce/run', {
method: 'POST',
headers: {
'Authorization': 'Bearer YOUR_API_KEY',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://example.com/products',
options: {
maxPages: 10,
format: 'json'
}
})
});
const data = await response.json();
console.log(data);Rate Limits
API rate limits depend on your subscription plan. Rate limit headers are included in every response.
| Plan | Requests/min | Requests/day |
|---|---|---|
| Free | 10 | 100 |
| Pro | 100 | 10,000 |
| Enterprise | Unlimited | Unlimited |
Authentication
Scrapers
API Services
User & Subscriptions
GET
Try it/api/user/profileGet user profile
PUT
Try it/api/user/profileUpdate user profile
GET
Try it/api/user/subscriptionGet subscription status
GET
Try it/api/user/usageGet API usage statistics
GET
Try it/api/user/api-keysList API keys
POST
Try it/api/user/api-keysCreate new API key
DELETE
Try it/api/user/api-keys/:idRevoke API key