SEO · 6th of 19 by usage
Website Content Crawler
Crawls websites and extracts clean text, markdown, or HTML content. Ideal for LLM training data, RAG pipelines, and knowledge base building.
- Price
- $0.002 per result
- Runs to date
- 506
- Teams using it
- 67
- Last updated
- 2026-03-14
Run it from the API
POST · run and wait for the datasetcURL
curl -X POST "https://api.apify.com/v2/acts/crawlerbros~website-content-crawler/run-sync-get-dataset-items" \
-H "Content-Type: application/json" \
-d '{ "maxItems": 50 }' \
-G --data-urlencode "token=$APIFY_TOKEN"Billed $0.002 per result. The call returns the dataset directly, so there is nothing to poll for a run of this size.
Website Content Crawler vs the other SEO scrapers
All 19 are ours and all are maintained, so the choice is about fit rather than quality. 1 of them cost less per result.
| Scraper | Price | Runs | Rating |
|---|---|---|---|
| Website Content Crawlerthis page | $0.002 | 506 | not yet rated |
| Website Screenshot Generator | $0.002 | 7,428 | not yet rated |
| Similarweb Scraper | $0.002 | 1,405 | not yet rated |
| Traffic Generator | $0.002 | 1,215 | 5.0 (1) |
| SEMrush Free Website Stats Scraper | $0.002 | 1,007 | not yet rated |
| Ahrefs Free Website Stats Scraper | $0.002 | 909 | not yet rated |
| Website Links Graph Generator | $0.002 | 391 | 5.0 (2) |
| URL to BibTeX Converter | $0.002 | 323 | 5.0 (2) |
| Wikipedia Article Scraper | $0.001 | 246 | not yet rated |
| Ubersuggest Keyword Scraper | $0.002 | 172 | not yet rated |
| Canva Template Scraper | $0.005 | 170 | not yet rated |
| PubMed Search Scraper | $0.002 | 162 | not yet rated |
| Sitemap Sniffer | $0.002 | 143 | not yet rated |
| Find Broken Links | $0.002 | 131 | not yet rated |
| OP.GG Scraper | $0.005 | 111 | not yet rated |