“How Scholarly is Google Scholar? A Comparison to Library Databases” (PDF pre-print paper for College & Research Libraries journal, accepted 30th June 2008)…

“We found that Google Scholar is, on average, 17.6% more scholarly than materials found only in library databases and that there is no statistically significant difference between the scholarliness of materials found in Google Scholar across disciplines.”

Scholarly worth was apparently judged by “subject-specialist reference librarians” rather than by active research scholars. I’m not sure if results from Google Book Search (which pop up in Google Scholar) were counted, or if it was only the articles that were evaluated — the word “book” occurs only once in the article. Of course, what really counts is if a user can get access to the article they want. If not then, for most students, the article might as well have been locked in a trunk and thown in the ocean.

This encouraging 2008 research should be balanced against some 2007 findings from Germany (Philipp Mayr and Anne-Kathrin Walter, “An Exploratory Study of Google Scholar“), which found some weaknesses in the up-to-dateness of Google Scholar results, and commented on…

“weaknesses in the accessibility of Open Access content”.

… this last probably due (I would guess) to the paucity of easily-accessible metadata, and the often awkward and haphazard ways in which such journals archive their articles (hence the need for a hand-made solution such as JURN).