The Common Crawl bots spent 2013 crawling the Web. Now their 102TB / 5 billion page index is available to anyone who wants it. For free. Re-use it freely too, on what is effectively a CC0 licence.