Compare

Honest comparisons of crawling tools and services

Measured, dated, vendor-neutral comparisons of scraping libraries, browser automation stacks, parsing libraries, HTTP clients, proxy types, and hosted platforms.

Comparisons with dated facts, stated criteria, and no winner you cannot justify. Where a price is relevant it is dated and linked; where a measurement is ours, the method is published.

Last updated 2 November 2026

—Scrapy vs Crawlee: which crawling framework fits your stack?A vendor-neutral comparison of Scrapy and Crawlee: language, queue model, throttling, browser support, storage, ecosystem, and the cases where a plain HTTP client is better.Updated 2 November 2026 · 4 min read

—Playwright vs Puppeteer vs Selenium: browser automation for scrapingCompare browser support, languages, auto-waiting, network control, and operational cost for Playwright, Puppeteer, and Selenium. No invented benchmarks; documented differences only.Updated 2 November 2026 · 4 min read

—BeautifulSoup vs lxml vs selectolax: measured parsing speedA reproducible local benchmark of BeautifulSoup (html.parser and lxml), lxml, and selectolax on a 125 KB fixture, plus API, XPath, and malformed-HTML trade-offs.Updated 2 November 2026 · 3 min read

—httpx vs requests vs aiohttp: choosing an HTTP client for crawlingCompare sync and async Python HTTP clients on API, HTTP/2, connection pooling, timeouts, and concurrency. No invented benchmarks; documented behaviour and practical guidance.Updated 2 November 2026 · 3 min read

—Hosted scraping platforms compared: criteria, not a leaderboardA neutral framework for comparing Apify, Zyte, Bright Data, ScraperAPI, ScrapingBee, and Browserless: access model, browser support, proxies, pricing transparency, and terms.Updated 2 November 2026 · 4 min read

—Proxy types explained: datacenter, ISP, residential, and mobileHow proxy types differ in address source, detectability, cost, and consent. Includes ethical and legal considerations, configuration notes, and when not to use a proxy.Updated 2 November 2026 · 3 min read

—Scrapy vs BeautifulSoup: framework or parser?Scrapy and BeautifulSoup are not competitors. One is a crawling framework; the other is an HTML parser. Learn which job each does and how they fit together.Updated 2 November 2026 · 3 min read

Keep going

Learn the mechanicsThe six-lesson course that makes the guides readable.Use a toolRobots tester, selector tester, sitemap validator, and more.Check a definitionThirty terms, defined first and linked into the guides.