Skip to content

Guides

Scraping PDFs in Python: text extraction, tables, and scanned documents

Extract text from public PDFs with pypdf, handle multi-page documents, detect scanned pages, and know when you need OCR or a different tool.

Last updated 3 min readBy CrawlerBros Editorial

Tested with Python 3.14.2, pypdf 6.7.4, fpdf2 2.8.5 on 2 November 2026. Library versions and dates are part of the evidence, not decoration.

A PDF is a container for drawing instructions, not a structured document. Text may be encoded as glyph runs, split across lines, stored out of reading order, or absent entirely because the page is a scan. Extraction is therefore a text-recovery problem, not a parsing problem like HTML.

This guide extracts text from public PDFs with pypdf, checks for empty pages, and explains when to use OCR or a table extractor.

Install

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\Activate.ps1
python -m pip install pypdf

Tested with Python 3.14.2 and pypdf 6.7.4 on 2 November 2026. fpdf2 2.8.5 was used only to generate the test fixture, not to read it.

Extract text from one PDF

from pypdf import PdfReader

reader = PdfReader("fixture.pdf")
print("pages", len(reader.pages))
for page_number, page in enumerate(reader.pages, start=1):
    text = page.extract_text() or ""
    print(f"--- page {page_number} ---")
    print(text)

Tested against a generated two-line PDF:

pages 1
--- page 1 ---
CrawlerBros PDF fixture
Product A-1 costs $89.00. Product B-2 costs $39.50.

extract_text() returns None or an empty string when the page has no text layer. That is the signal to check for a scan.

Download a PDF and extract it

import io
import requests
from pypdf import PdfReader

response = requests.get(
    "https://example.com/report.pdf",
    headers={"User-Agent": "CrawlerBros-Example/1.0 (+https://www.crawlerbros.com/about)"},
    timeout=30,
)
response.raise_for_status()
if "application/pdf" not in response.headers.get("content-type", ""):
    raise ValueError(f"not a PDF: {response.headers.get('content-type')}")

reader = PdfReader(io.BytesIO(response.content))
text = "\n".join((page.extract_text() or "") for page in reader.pages)
print(len(reader.pages), len(text))

This shape is standard; it was not run against a live third-party PDF. The point is the content-type check and the in-memory buffer: do not write a file you do not need, and do not trust a URL ending in .pdf.

Detect scanned pages

A scanned PDF has an image per page and no text layer. extract_text() returns an empty string. Check before running OCR:

empty_pages = [
    number
    for number, page in enumerate(reader.pages, start=1)
    if not (page.extract_text() or "").strip()
]
print("pages without text", empty_pages)

If most pages are empty, the document is likely a scan. OCR is a separate pipeline: render the page to an image, run a text recogniser, and store the OCR output with its confidence and page number. OCR is slower, less accurate, and language-dependent; budget for it.

Tables

pypdf extracts text, not table structure. A table may come out as a column of words or a jumbled line. Options:

  • pdfplumber for rule-based table extraction; inspect the result, because merged cells and multi-line headers break naive assumptions.
  • Camelot for lattice/stream tables; it depends on Ghostscript and works best on ruled tables.
  • A vision model for complex layouts; treat the output as probabilistic and validate it.

Do not promise a table parser will work on every PDF. PDF has no table model.

What to store

  • the source URL and the fetch time;
  • the PDF hash (so you can tell when it changed);
  • the extracted text per page;
  • the extractor name and version;
  • an OCR flag and confidence if OCR was used;
  • the page count.

A PDF extraction without the extractor version is not reproducible. A page number is part of the citation.

  • A public PDF is not automatically free to republish. Copyright and database rights still apply.
  • Personal data in a PDF is still personal data; minimise and set retention.
  • Do not scrape a PDF behind a login or paywall you are not authorised to use.
  • Respect the site’s terms and robots.txt; a PDF URL is still a request.
  • If you need the content at scale, ask for a data licence or an export.

When not to use pypdf

  • Scanned documents: OCR first.
  • Complex tables: pdfplumber/Camelot, then validate.
  • Forms with fields: use the PDF form API, not text extraction.
  • A page that offers HTML or an API: use that instead.

Sources

Sources checked