November 25, 2009
CSI: Google Scholar
Whether Google Scholar is going to seriously challenge the wexis's of the world is not my concern here. Nor is its current lack of serious search functionality. It is about what happens when you search Google Scholar because whether we like it or not, if it is Google, "they" will use it. Especially if Google decides to include case opinion results in the general Google search results. So, in order to be prepared, I investigated how the new kid in town might influence or change someone's research results.
My test project: Find cases in N.Y. courses on cosmetic surgery that didn't go so well.
My limits: New York cases
My search terms:
I did what I imagine most law students would do. This means my search was: cosmetic surgery
(Perhaps they might add negligence, but I'll keep it simple for the sake of the blog.)
So what happened?
I retrieved 50 cases from the three levels of court in N.Y. which included published and some unpublished decisions. Not so bad. However, four of the cases were listed twice. First with its reporter citation and then further down in the list a slip opinion. So the numerical result was really 46. Note that one of these cases also had an appellate level opinion listed; however, you would need to know the publication patterns for N.Y. reporters to realize this as Google Scholar does not use their muscle to make the connection for you.
The order that the cases were placed in remains a mystery.
Interestingly, two of the first four cases were fairly famous cases in N.Y., but not for torts centered around cosmetic surgery. They are famous for parents raising religious objections to medical procedures performed on their child. Neither of these cases, In re Siefirth nor Matter of Sampson, mention the phrase "cosmetic surgery." Seifirth was decided in 1955. The term "cosmetic surgery" was not even being used. Sampson dealt with a blood transfusion and had nothing to do with cosmetic or plastic surgery
So why are they in my results list?
- Was Google adding odd metadata like they are doing with the book scanning project?
- Was the search engine just sloppy?
I learned the answer from Andrew Larick-Plumb's now infamous post on the Case Western law library blog. Apparently, Google scholar is not searching the text of the the cases. It is searching the text of the documents listed in the "How Cited" tab that appears next to the case list and analyzes linking patterns to create a results list. Sampson showed up in the hit list because in the literuture it is often linked with Seifirth. Both are early cases that discuss the parents' right to exercise their religious freedom in choosing health care for their children.
Comparing to Wexis:
I did not compare the case results from Google Scholar to other free resources because there are no other free resources that allow you to search across all three levels of NY courts. I reran the same search in appropriate N.Y. cases databases on Lexis and Westlaw and limited my results to cases that were decided after 1949. Of course nither Seifirth nor Sampson were in the results lists from either company because they do not contain the phrase "cosmetic surgery." But neither were 10 other cases.
Westlaw gave me 45 hits and Lexis gave me 44 hits. Between Westlaw and Lexis, there were three unique cases. In Westlaw, three cases showed up that did not show up in Lexis though 2 of the 3 did show up in Google Scholar. In Lexis, there was one case that was not reported in either Westlaw or Google Scholar, and one case that was reported in Google Scholar but was not reported in Westlaw. The two cases on Lexis that were not reported on Westlaw were both reported only in the New York Law Journal which explains why it was not picked up in the case law database I chose on Westlaw (NY-CS). That one case that was unique to Westlaw was an unpublished decision from a lower N.Y. civil court which helps explain why it was not picked up by either Lexis or Google Scholar.
With respect to Google Scholar as compared to wexis, there were a total of 10 cases picked up by Google Scholar that were not reported in either Westlaw or Lexis. Only one of the nine seems to be relevant and I am not sure how to explain why the search in Westlaw and Lexis did not pick up that one case. It was a 2008 slip opinion from the trial level that was about a cosmetic surgery gone bad. None of the other 9 cases that were unique to Google Scholar had the sought-for phrase. Considering the small set of results, this is rather a large subset to sift through. I larger set of results would make it very annoying to work with.
From the other side, there were 12 cases reported on Westlaw and Lexis that should have been picked up by Google Scholar if it indexed its cases instead of creating linking patterns based on citations in literature, and had a full set. In fairness, the missed cases were unpublished decisions that were mentioned only in an Appendix to the reporter or not published at all. We are not going to miss them too much.
Certainly, this method of case discovery has some interesting benefits. It will, of course, pick up cases that are note worthy enough to deserve merit by an author. From an academic perspective, this might be a way to jump start a note topic or seminar paper. It isn't the best way to do so, but if we are looking for ways to use this new tools, this could be a good way.
Will I teach students about it? Yes, of course. And I will go through this same exercise with them so that they understand the process of the tool and where it fits in their tool bag. I think once they run into snafus like what I found with Seifirth and Sampson, they may be more discriminatory about relying on it. (VS)
In one of my assignments, we're required to find 3 articles that provide ideas about Google and its impact using ONLY Google Scholar. Although this sounds remotely easy, i couldnt find anything useful as opposed to using ordinary search.
Posted by: buy silagara online | Mar 2, 2010 9:50:06 AM
When using this automatic software to comment spam, be sure to delete this text and add something that looks like you read thier article.
Posted by: prohandymanserve.com | Feb 25, 2010 7:17:52 PM
The subscription argument does seem to get lost in the discussion. I explained to our legal writing and research faculty who were first excited about this that the only reason they could read some of the higher end articles is because they belong to a school that subscribes to them. Enthusism quickly died.
I think that Koulikov's comment is accurate at the momemt, but with the rising costs of subscription vendors, those lawyers wanting to cut corners for clients who question their online bills may use this for due diligence at the onset - and as it improves, well, who knows right? It is a wait and see game right now.
Posted by: Vicki | Nov 27, 2009 8:16:01 AM
My thinking about all of this is more along the lines that as it currently stands, Google 'Legal' does nothing of real value for a lawyer, law student, or law librarian - but it's a very valuable resource for a political science scholar who basically has no idea about what Westlaw and Lexis are, how to access them, and how to then actually use the bloody things. Or, hell, for a literature or English language scholar who wants to know see how a particular term was used in official documents at a given point in time.
Plus, of course, it will open judicial decisions up to bibliometric analysis far more than they have been so far, since most of the people who do bibliometrics right now approach it from a background in LIS, rather than in law and law librarianship.
Posted by: Mikhail Koulikov | Nov 25, 2009 11:28:22 PM
It appears to me that the more interesting and useful feature of Google Scholar is its journal search that is also part of each inquiry. It searches Hein-on-Line, SSRN and Google Books (as well as some other sources). This means that (through H-o-L) a search will turn up articles that pre-date wexis data bases for law reviews. It will pick up SSRN papers that have not yet been published and so do not appear in wexis. And it will pick up at least some books. Users need subscription access to Hein-on-Line to get past the first page of an article-- so law students will need the school's remote password if they do these searches outside the school's network.
Posted by: Howard Friedman | Nov 25, 2009 8:38:00 PM