The Common Crawl bots spent 2013 crawling the Web. Now their 102TB / 5 billion page index is available to anyone who wants it. For free. Re-use it freely too, on what is effectively a CC0 licence.

Elastic Search 1.0 has just released. It’s a search-engine, open source, with four years of development under the hood. Aimed at big businesses, but free. Currently trying to swing itself around to head toward the gap that’s looming due to the shortage of human Big Data analysts. InfoWorld magazine has a straightforward overview.

Web hyperlinking to freely available online content does not amount to publishing an illegal communication, says a major new ruling by the Court of Justice of the European Union. Linking to open content cannot communicate it to a new public audience, since the content is already public.

However, the ruling says nothing of the taxing of Web links to open content, a loopy idea proposed by French socialists among others.

The US Defense Advanced Research Projects Agency (DARPA) want to build Google-style custom search engines for spooks, or as they phrase it… “domain-specific indexing of web content [with] domain-specific search capabilities”. Might be nice if they could also invite worthy non-profits like JURN to park up on their uber-servers for free, with maybe 10,000 domain URLs to play with — compared to the 5,000 URL limit Google places on its CSEs.

TED has the full set of statistics from Coursera, one of the leading MOOCs…

Cool new Radio search engine. Indexes 400,000 radio station streams in real-time, tags artist metadata to the songs they’re streaming, then lets you search and load those streams. Works fine, and is an excellent method of discovering online radio stations playing artists you like. Would be great to see this technology extended to intelligent speech radio and intellectual podcasts. It serves as a sort of podcast-and-BBC search at present, but my search for “economy” showed it to give very poor results when it strays outside of music.

A frank summary report of a recent open access event in the UK…

“The publisher-led proposal [apparently in a very limited trial] to offer walk-in access in UK public libraries to a majority of [their commercial database] journals was also dismissed as ‘lip-service’, an experiment intended to fail in order to show that there was ‘no demand’ for wider access.”

There’s also a question of how that idea would work in practice, if it was ever rolled out to all UK public libraries. Assuming password-controlled home access to the service is not allowed for library members (like it is for accessing NewsUK and other publication databases), then how many PCs in the library would be equipped to offer the service? All of them? Would the public be allowed to save their accessed PDFs to a memory stick or cloud storage, and walk out? If so, how many PDFs on each visit? If only paper print-outs of articles were allowed, at current public library costs of say 6 to 8 pence per sheet, an average long humanities article might carry a printing cost of perhaps £1.60 to £2.50. Would publishers be able to claim a cut of that printing charge, perhaps even inflating the cost in such instances to 10p or 12p per printed sheet in order to boost their cut? I wonder if such apparently ‘free’ access, in the hands of the major paywall publishers, might actually end up costing library users £4 or £5 per article — in terms of users being able to take articles home for close study and contemplation.

The DOAJ has just launched a radical web design makeover. The DOAJ seems to have morphed into a single-box search engine, with all their records made dynamic. The lack of the old Directory might be a problem for student searchers unsure of the correct keywords and spellings, or even of the existence of a journal. There is a sidebar, which sort of serves as a filterable directory — but the usability is rather poor and it is only really usable in conjunction with a keyword search. I suspect the new dynamic results approach also presents problems for Google, since it effectively (unless they do something clever with OAI-PMH harvesting) blocks the Google Search bots from indexing any of the DOAJ’s records as static Web pages?