Copyright and database rights for scraped content

Copyright protects original expression, not facts or ideas. Database rights (in the EU and UK) can protect the investment in obtaining, verifying, or presenting a database’s contents, even when the individual facts are not copyrightable. Scraping a public page does not automatically make the content free to republish.

This page is research material, not legal advice. It summarises the main doctrines; it does not apply them to your facts.

Copyright: what is protected

article text, descriptions, and reviews;
photographs, illustrations, and videos;
original page design and graphics;
code and structured data with original expression.

It does not protect:

facts (a price, a date, a stock level);
ideas, methods, or systems;
names, titles, or short phrases (generally);
data that lacks originality in selection or arrangement.

Feist Publications v. Rural Telephone Service (1991) is the canonical US case: a telephone directory’s facts were not copyrightable, and the selection/arrangement lacked the originality required. The case is often cited for the proposition that “sweat of the brow” is not enough for copyright.

What a crawl may do

Copyright regulates copying, distribution, adaptation, and public communication. Fetching a page into memory and extracting facts is not the same as republishing the article or the photos. But:

storing full copies of copyrighted text and serving them to users can infringe;
republishing photos or article text without a licence can infringe;
creating a derivative database of extracted content can raise separate issues;
quoting for criticism, review, or news may be permitted under fair use/fair dealing, but the scope differs by jurisdiction.

A crawler that extracts prices and stock levels is usually on firmer ground than one that republishes the description and images. That is a factual difference, not a licence.

The EU/UK database right

The EU Database Directive (96/9/EC) created a sui generis right for databases whose maker shows substantial investment in obtaining, verifying, or presenting the contents. It is separate from copyright.

The CJEU’s British Horseracing Board v. William Hill (C-203/02) narrowed the right: investment in creating the data does not count; investment in obtaining, verifying, or presenting existing data can. The court also addressed extraction and re-utilisation of substantial parts.

Practical consequences:

A database of facts can be protected even if each fact is not.
Extracting a substantial part can infringe, even if you change the format.
Repeated extraction of insubstantial parts can amount to a substantial part in aggregate.
The UK retained a database right after Brexit, with its own regime.

Terms and contracts

Copyright and database rights are statutory. The terms of service are contractual. A site can prohibit scraping by contract even where the content is not copyrightable. The terms guide covers that layer.

AI training and text/data mining

The copyright status of training a model on scraped content is unsettled and jurisdiction-specific. The EU’s DSM Directive (2019/790) introduced text-and-data-mining exceptions with opt-out conditions; the UK, US, and other jurisdictions have different rules and ongoing litigation. Do not treat “public web” as a free training licence. If you are building a dataset for model training, get advice and check the site’s terms and robots directives.

A practical assessment

What are you copying: facts, text, images, or a whole database?
Will you republish it, or only extract facts?
Is the selection/arrangement original?
Is there a substantial investment in obtaining/verifying the database?
Do the terms prohibit the use?
Is there a licence (open data, API terms, CC BY)?
Does your jurisdiction have a text-and-data-mining exception, and does it apply?

If you need the article text or images, get a licence. If you need facts, extract the minimum and do not redistribute the expressive content.