Site guides

Getting data from specific sites, the right way

A small set of high-demand site guides that start with official APIs and public data, state the terms plainly, and never teach bypassing a login or a paywall.

A deliberately small set. Each guide starts with an official API, then a public-data approach, and states the terms and legal considerations plainly. No login walls, no paywall bypasses.

Last updated 2 November 2026

—Getting Amazon product data: the official API first, and the limits of public pagesUse the Product Advertising API where it fits, understand Amazon's terms, and assess the legal and technical limits of public product pages. No login or paywall bypass.Updated 2 November 2026 · 3 min read

—Getting YouTube data: the Data API, quotas, and what not to scrapeUse the YouTube Data API v3 for public metadata, understand quota costs and OAuth scopes, and avoid downloading or scraping content the terms do not permit.Updated 2 November 2026 · 3 min read

—Getting Reddit data: the Data API, OAuth, and the limits of public pagesUse the Reddit Data API with OAuth for public posts and comments, respect rate limits and terms, and understand why scraping the website is not a free substitute.Updated 2 November 2026 · 3 min read

—Google Maps data: use the Places API, not a page scraperGet place names, addresses, coordinates, hours, and ratings through Google Maps Platform APIs, understand the billing and caching rules, and know what not to scrape.Updated 2 November 2026 · 3 min read

—Job data for crawlers: use official ATS APIs before scraping job boardsCollect public job listings from Greenhouse, Lever, Ashby, and Workable APIs, respect board terms, normalise fields, and avoid the login walls and duplicates of job boards.Updated 2 November 2026 · 3 min read

Keep going

Learn the mechanicsThe six-lesson course that makes the guides readable.Use a toolRobots tester, selector tester, sitemap validator, and more.Check a definitionThirty terms, defined first and linked into the guides.