« Justice Breyer Hopes Someone Will Read His Dissenting Opinions | Main | Gadgets in the Library: A Practical Guide to Personal Electronics for Librarians »
March 28, 2011
No Surprise: Online Research Beats Manual Research
There is a study called A Day Without A Search Engine: An Experimental Study Of Online and Offline Search, and it's getting some press. It basically compares online and manual research as conducted in a library and highlights the differences in the time involved to arrive at results and the quality of the results. The study, from the University of Michigan, is focused on general research and not legal research.
The experiment was designed around random web queries within the United States on a single day. They were filtered so that they could be answered manually or online. One criticism of taking questions from an online source is that they may be biased towards the web as a source. There is no suggestion in the report that the designers looked at the results before selecting the questions for the study. As I'm fond of telling students, and as much as I love using the web myself, not everything is online. More on that in a bit.
The experiment results showed four things. Using 305 questions, 99.7% were answered via the web and 90.2% were answered in the non-web treatment. Online searches take one-third the time of an offline search. Web searchers used more sources than those who used the library. 70% of the non-web users consulted a reference librarian. Finally, quality of web and non-web sources were not significantly different.
Google, one can imagine, is just ecstatic with the results. Hal Varian, Google's Chief Economist, was quoted in a short interview in The Economist about the productivity gains from web research. His numbers were about 22 minutes using the library compared to 7 minutes using a Google search. He sees this as the democratization of data, enabling users with an Internet connection in remote location.
This study focused on general research, with the answers limited to facts in some cases, and others with conceptual responses that are fleshed out. I'm intrigued as to whether similar results would be obtained with legal research. I think a knowledgeable researcher can pull a cited case online, whether from a commercial service or a free resource such as the Google Scholar case database. It's the conceptual searching I'm wondering about. Lexis and Westlaw certainly have their share of topical commentary. Natural language searching is pretty sophisticated, especially in its WestlawNext incarnation.
Still, with all of this, I'd be curious of the time and the efficiency savings compared to cost of commercial services, the free web, and print. Price wasn't a factor much in the Michigan study though participants weren't limited to Google. They were allowed to use pretty much any electronic resource to which they had access. I have to assume that some of Michigan's commercial subscriptions were part of the mix. The free web has delivered more or less on the promise of unedited primary law, though real library-like organization is lacking. Searching legal concepts is one of those areas where it falls completely short, in my opinion. Given all of this, I wonder if there are recent numbers out there that really measure the efficiencies in online and manual legal research. [MG]
March 28, 2011 in Info - Antics or Metrics?, Legal Research Instruction | Permalink
Comments
These results would be very different was the study to require a legal search with a cost component. I do not think the study is a fair comparison or significantly illuminating. If finding factual or simple conceptual information is all a researcher is being asked to perform, of course, an electronic search would be faster than a manual one. After all, an electronic researcher has so many more data tags available to him than a manual researcher, who is confined to the Library of Congress headings or the sources’ indexes and digests. Conversely, legal research rarely involves merely looking up facts and, instead, requires complex conceptual searches. An electronic researcher probably has the edge over a manual researcher if the issue can be addressed with precedent from the last 25 years. This is largely due to the availability of free governmental and academic, legal research websites. But were cost to be a component, the manual researcher would definitely outperform the electronic researcher if the issue required the analysis of case law prior to 1980.
Posted by: CL | Apr 6, 2011 7:18:43 PM
This is not surprising; however the questions were probably pre-formulated web queries; however, many questions that students and scholars search to answer cannot be answered by using google... maybe google scholar, or online databases (such as Galileo,JSTOR) that an organization may subscribe to. I prefer to use both simultaneously, and I still feel that physical resources still provide me with some type of comfort, especially when conducting scholarly research.
Posted by: Dindi R. | Mar 29, 2011 1:51:53 PM
Interesting. I think the bias inherent in choosing questions from actual web queries is distinctly non-negligible. (I'm pretty sure I could craft questions based on actual reference queries that would achieve the opposite result.) However, I'm not sure how one would devise an unbiased set of questions. What I'd really like to see, though, is a study like this that compared print research vs. online research vs. an unrestricted combination of the two. I find that I often pull a print source that gives me information from which to craft web searches or run a few searches that point me to an appropriate print source.
Posted by: Anon | Mar 28, 2011 12:46:42 PM
