Archive of Our Own (AO3) Scraper: Up to 1,000 Free Results a Month

Extracting public fanfiction records returns 37 fields under work listings, including full engagement stats, relationship classifications, and author details without requiring credentials. A batch of 1,000 results costs $5.00 on the free plan. The collector is built for digital humanities researchers, tag taxonomy analysts, and recommendation engineers tracking archive trends. It is not suitable for retrieving works restricted exclusively to logged-in users, as restricted items return typed error records instead of private text.

Try it before you read further. Apify's free plan includes $5.00 of usage every month with no credit card, enough for up to 1,000 results at $0.005 each before platform usage. Open Archive of Our Own (AO3) Scraper on Apify and run the prefilled example.

How reliable is Archive of Our Own (AO3) Scraper in production?

Across the last 30 days of public runs on the Apify platform, Archive of Our Own (AO3) Scraper recorded 108 runs with the following outcomes.

No run failed or timed out in the last 30 days; the 1 that did not finish was stopped by the people who started them. Keep a retry and an alert on scheduled runs all the same: a clean month is a record, not a guarantee.

What does it cost to run Archive of Our Own (AO3) Scraper?

Each result costs $0.005 on Apify's free plan, which is $5.00 per 1,000 results. Starting a run is charged separately at $0.005 per GB of Actor memory. Apify also bills the platform usage each run consumes, at the rates of your Apify plan, on top of these charges.

Worked example: collecting 10,000 results costs $50.00 in result charges before run-start fees and platform usage. No run failed or timed out in the last 30 days, so the list price is a fair budget; keep a retry in place all the same.

Total run expenses are dictated by maxItems, which places a hard cap on emitted records between 1 and 2000. Enabling chapter text via includeChapterText expands item payload size but does not increase dataset row counts. To validate schema alignment cheaply, run initial jobs with maxItems set to 20 before scaling data collection.

How do you run Archive of Our Own (AO3) Scraper from the API?

The schema marks 1 of its 45 controls as required: mode. Every value in the payload below comes from the published schema's own prefills, which means you can paste it, swap the token, and get a real result.

Call the synchronous endpoint to start a run and receive dataset items in one request:

curl -X POST "https://api.apify.com/v2/acts/crawlerbros~archive-of-our-own-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"search","searchQuery":"time travel","fandomTag":"Harry Potter - J. K. Rowling","tagName":"Fluff","collectionName":"yuletide2023","tagSearchQuery":"fluff"}'

The same run from Python, using the official client:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
  "mode": "search",
  "searchQuery": "time travel",
  "fandomTag": "Harry Potter - J. K. Rowling",
  "tagName": "Fluff",
  "collectionName": "yuletide2023",
  "tagSearchQuery": "fluff"
}

run = client.actor("crawlerbros~archive-of-our-own-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

And from Node.js:

import { ApifyClient } from 'apify-client'

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' })

const input = {
  "mode": "search",
  "searchQuery": "time travel",
  "fandomTag": "Harry Potter - J. K. Rowling",
  "tagName": "Fluff",
  "collectionName": "yuletide2023",
  "tagSearchQuery": "fluff"
}

const run = await client.actor('crawlerbros~archive-of-our-own-scraper').call(input)
const { items } = await client.dataset(run.defaultDatasetId).listItems()
console.log(items)

That endpoint blocks until the run completes. Fine while you are testing a handful of records, risky once a run takes minutes: a dropped connection loses the response even though the run itself finished. Switch to an asynchronous start with polling or a webhook before you schedule anything.

Which Archive of Our Own (AO3) Scraper inputs matter, and which can you skip?

The central input control is mode, which selects among search, tag exploration, author indexing, or direct URL extraction. Most jobs only require pairing mode with either searchQuery, fandomTag, or tagName while leaving specific bounds like hitsMin, commentsMin, and wordCountMax empty. Fine-grained facets should only be configured once you have verified the baseline volume of your selected tag.

• mode (string): What to fetch. Default: "search".
• searchQuery (string): Free-text query searched across title, summary, tags and body. Required for mode=search. Also usable as an optional refine filter on byFandom/byTag/byAuthor - leave blank there to avoid narrowing results. Default: "".
• fandomTag (string): Exact AO3 fandom tag to browse, e.g. Harry Potter - J. K. Rowling, Marvel Cinematic Universe, Naruto.
• tagName (string): Exact AO3 tag to browse - a relationship, character, or freeform tag, e.g. Fluff, Hurt/Comfort.
• collectionName (string): AO3 collection slug to browse, e.g. yuletide2023 (from archiveofourown.org/collections/yuletide2023).
• tagSearchQuery (string): Find AO3 tags whose name contains this text - useful for discovering the exact canonical tag name/spelling before using it in fandomTag, tagName, or the refine filters.
• titleQuery (string): Restrict to works whose title contains this text.
• creators (string): Restrict to works by this creator (pseud or username).
• fandomNames (string): Comma-separated fandom names to refine results, e.g. Harry Potter - J. K. Rowling, Marvel Cinematic Universe.
• relationshipNames (string): Comma-separated relationship tags to refine results, e.g. Harry Potter/Draco Malfoy.
• characterNames (string): Comma-separated character tags to refine results.
• freeformTags (string): Comma-separated additional/freeform tags to refine results, e.g. Fluff, Slow Burn.

The other 33 controls, with their defaults, are listed in the input schema on Archive of Our Own (AO3) Scraper on Apify.

Fixed-choice controls: mode accepts 9 values (default search), including search (Search works (full-text + facets)), byFandom (Browse by fandom), byTag (Browse by tag (relationship / character / freeform)), byAuthor (Works by an author (username)).

What does Archive of Our Own (AO3) Scraper return?

Output items deliver complete work-level metadata, including word counts, chapter tallies, kudos, bookmarks, and detailed categorization tags. They do not contain private user bookmarks or restricted, login-only fiction bodies. When an inaccessible URL is queried directly, the actor outputs a structured error record rather than dropping the entry silently.

Output per work

• workId, sourceUrl
• title, authors[], authorUrls[]
• fandoms[], relationships[], characters[], freeformTags[]
• rating, warnings[], category[]
• language, summary
• wordCount, chapterCount, chapterCountTotal, isComplete
• comments, kudos, bookmarks, hits
• seriesTitle, seriesUrl, seriesPart
• collections[], authorNotes, giftFor[] (mode=byUrls)
• lastUpdated (listing modes) or publishedDate / updatedDate (mode=byUrls)
• chapterTitles[], chapterText, chapterTextChapterNumber/chapterTextScope (mode=byUrls, opt-in only)
• chapterNotes, chapterEndNotes (mode=byUrls, single-chapter opt-in scope, multi-chapter works only)
• recordType: "work", scrapedAt

Output per series (mode=bySeries)

• seriesId, sourceUrl, seriesTitle
• creators[], creatorUrls[]
• beginDate, updatedDate, description
• wordCount, worksCount, isComplete, bookmarks
• recordType: "series", scrapedAt

Output per tag (mode=tagSearch)

• tagName, sourceUrl, tagType (Fandom / Character / Relationship / Freeform / UnsortedTag / etc.), useCount
• recordType: "tag", scrapedAt

Output per author (mode=authorSearch)

• username, pseud, displayName, profileUrl
• worksCount, worksUrl, bookmarksCount, bookmarksUrl, iconUrl (when a real, non-default avatar is set)
• recordType: "author", scrapedAt

These are the documented fields. Optional ones can be empty on a given record, so measure how often each field your deliverable depends on is populated across a real sample before automating the handoff.

How do you build the workflow end to end?

Open Archive of Our Own (AO3) Scraper and work through these in order. Each step ends with something to check, so a bad configuration surfaces on a small run rather than a scheduled one.

1. Run mode tagSearch with tagSearchQuery set to your target fandom or relationship to verify the exact canonical spelling and confirm useCount.
2. Execute a test run with mode set to byFandom or byTag using the exact canonical string in fandomTag or tagName, keeping maxItems at 20.
3. Inspect the emitted records to confirm recordType equals work and check that numeric engagement metrics like hits, kudos, and bookmarks contain values.
4. Refine your retrieval scope by applying search facets such as language, rating, completeStatus, or a date range via dateUpdatedFrom.
5. Scale up maxItems toward your project requirement, keeping in mind the hard cap of 2000 records per run execution.
6. If raw text is required, pass extracted work URLs into mode byUrls with includeChapterText set to true before parsing chapterText.

How do you apply it? Three worked playbooks

These are Archive of Our Own (AO3) Scraper's own documented use cases, each worked through as an operating pattern rather than a description.

Use case 1: Fandom research

Outcome: Track how a fandom's output, pairings, and tag trends evolve over time

Configure: Set mode to byFandom, fandomTag to 'Harry Potter - J. K. Rowling', dateUpdatedFrom to '01 January 2024', sortBy to 'revised_at', and maxItems to 200.

Working method: Start by validating the fandom name using tagSearch to eliminate zero-result queries. Run the browse mode sorted by date across set intervals, then cross-reference relationships and freeformTags arrays across time buckets to monitor evolving tag frequency.

Deliverable: A structured dataset of work records containing publication dates, relationship tags, and character counts suitable for longitudinal trend visualization.

Stop condition: Zero records returned due to an invalid canonical fandomTag string or empty arrays across relationships and characters.

Use case 2: Recommendation engines

Outcome: Build a fic recommender from tags, ratings, and kudos/hits ratios

Configure: Set mode to search, completeStatus to 'complete', wordCountMin to 5000, sortBy to 'kudos_count', sortDirection to 'desc', and maxItems to 500.

Working method: Begin by filtering for completed long-form narratives to avoid unfinished drafts. Run queries across specific category pairings, parse hits, kudos, and bookmarks, and compute normalized interaction ratios to establish a ranking threshold for recommendations.

Deliverable: A tabular dataset of completed fanfiction records enriched with calculated kudos-to-hits ratios and normalized tag taxonomy vectors.

Stop condition: Runs producing records with missing hits fields or incomplete status flags marked false despite completeStatus filters.

Use case 3: Academic study of fan culture

Outcome: Bulk-export tag taxonomies and metadata for transformative-works research

Configure: Set mode to tagSearch, tagSearchType to 'Relationship', tagSearchCanonicalOnly to true, and maxItems to 1000.

Working method: Execute tag search queries across key relationship patterns to gather canonical taxonomy structures. Compare tag frequencies against target fandom archives by cross-referencing extracted useCount values against collected work-level metadata.

Deliverable: A comprehensive JSON export mapping canonical tags, use counts, and pairing categories for archival and linguistic research.

Stop condition: The tagSearch output contains non-canonical variants when tagSearchCanonicalOnly is set to true.

What breaks, and how do you design around it?

• Over the last 30 days, 0.0% of public runs failed and 0.0% timed out. Build retries and alerting around those rates rather than assuming every run completes.

AO3 strictly requires canonical tag names, causing mistyped strings to return zero hits. Resolve this by executing tagSearch prior to batch runs to retrieve exact wrangled spellings. For large text preservation, note that allChaptersText caps narrative extraction at approximately 300,000 characters per work.

When should you not use Archive of Our Own (AO3) Scraper?

Do not use this Actor if your project requires private works locked behind user accounts or direct messaging data. AO3 restricts certain works to registered members, and this scraper deliberately skips authenticated sessions, outputting error entries for restricted items. If your focus is general literature and public domain texts rather than transformative fan works, consider the Internet Archive Search Scraper instead. Additionally, do not use this scraper as a high-frequency polling engine to mirror the entire archive, as the platform is a volunteer-run nonprofit infrastructure that relies on gentle rate pacing.

What should you check before trusting the output?

• Verify that recordType evaluates to work; abort the ingest pipeline if recordType returns error due to restricted-access works.
• Ensure workId is non-empty and sourceUrl matches a valid Archive of Our Own URL structure.
• Check that hits, kudos, comments, and bookmarks are non-negative integers; flag records where all engagement values remain zero across old works.
• Verify that relationships, characters, and freeformTags parse as valid arrays rather than raw delimited strings.
• If running mode byUrls with includeChapterText enabled, fail the step if chapterText is empty while isComplete is marked true.

None of this proves a record is correct. It gives a scheduled Archive of Our Own (AO3) Scraper run defined points where it should stop instead of quietly passing bad data downstream.

Frequently asked questions

How reliable is the scraper during scheduled runs?

In the last 30 days, the scraper completed 107 of 108 runs successfully, recording a 99.1% success rate with 0 failed runs and 0 timed out runs. Only 1 run was aborted by a user. Its built-in jitter and automatic backoff keep data collection stable across automated batches.

How much does it cost to scrape 10,000 works?

Results cost $5.00 per 1,000 results on the free plan, which translates to $50.00 in result charges for 10,000 items. Paid Apify tiers offer lower unit prices. Memory run-start fees and platform compute usage are billed separately on top of result counts.

Why did my run return zero results for an existing fandom?

AO3 filters fail unless tag spellings match archive taxonomy exactly. If a fandom or pairing tag has unusual punctuation, run mode tagSearch with tagSearchCanonicalOnly enabled first to discover the canonical wrangled string before entering it into fandomTag or tagName.

Can I extract the full text of stories in bulk?

Full story text requires mode byUrls with either includeChapterText or allChaptersText enabled. Listing modes only fetch work-level metadata. Note that allChaptersText is capped at approximately 300,000 characters per story to avoid payload exhaustion.

Does the scraper access restricted or locked works?

No. The scraper operates without user login cookies and cannot access works restricted to registered account holders. When encountering a locked or deleted item in direct lookup mode, it writes an error record containing workId and sourceUrl rather than failing.

Where to go next

When you are ready to run it, open Archive of Our Own (AO3) Scraper on Apify; the free plan covers up to 1,000 results a month.

Other Actors we maintain for related data:

• Our World in Data Scraper: Scrape Our World in Data (ourworldindata.org) - browse articles by topic or keyword, fetch chart data (CSV) by slug, or explore country profiles.
• Internet Archive Search Scraper: Searches and retrieves items from the Internet Archive (archive.org) - 44M+ books, videos, audio, software, and web archives.
• Ghost Blog Scraper: Scrape posts, pages, tags, authors, and site info from any Ghost CMS-powered blog via Ghost's public Content API - including ghost.org's own /resources, /changelog, and /help sections.
• Freebooksy Scraper: Scrape free and discounted Kindle ebook deals from Freebooksy - the leading free-ebooks deal site.
• CPAN Module Scraper: Scrape CPAN (Comprehensive Perl Archive Network) via MetaCPAN API.

Resources

• Actor documentation, input schema, and pricing: verified against the published Actor on 2026-09-27.

• Actor last updated by its maintainers on 2026-07-13.

• Run outcome figures cover the 30 day public window ending 2026-09-27.

• Archive of Our Own (AO3) Scraper on Apify