Common Crawl

Common Crawl (opens in a new tab)

• Common Crawl maintains a free, open repository of web crawl data that can be used by anyone.

We make wholesale extraction, transformation and analysis of open web data accessible to researchers.

Over 250 billion pages spanning 17 years. Free and open corpus since 2007. Cited in over 10,000 research papers. 3–5 billion new pages added each month.