Law Librarian Blog

What May Become the Next Normal in Pricing Online Legal Search in the 21st Century?

In the "good old days" when we aging and decrepit Boomer-gen law librarians were bright young things, very expensive legal search vendors only offered "everything." At that time, everything in the databased content inventory wasn't all that much. Soon enough, "everything" was limited to the non-academic law library market because certain resources were excluded from academic law library licenses.

Early on database vendors' pricing mechanism generated very high variable search costs in the private and government sectors. While the private sector was then still able to charge back costs to clients, the government sector could not afford this model because, well you know, government entities have budgets to maintain. The call went out for some sort of in-plan access. "Impossible!" was the vendor responses based on false technology claims. Citing the differences in academic plans compared to the entire database universe model in private and government sectors plans, "impossible" became "possible."

Thus arrived the era of in-plan access with fixed costs with an out-of-plan option that included a variable cost component. Of course, fixed costs were and still are based on a mysterious pricing matrix that takes into account database selections, number of user accounts, number of legal professionals and various print tie-in arrangements at the institutional buyer's level. Ultimately, however, it boiled down to an institutional buyer's purchasing power, the negotiation skills of that buyer's representative, and, in the private sector, the consultants sometimes employed because the NDAs incorporated in licensing agreements prohibit sharing pricing data.

Those were the "select your database, then perform your search" days. With the next gen current gen WEXIS platforms, it is perform your search  and then filter your results largely by data elements that once were segment or field searches in former menu-driven database categories. Eliminating the marketing nonsense about how this or that vendor's metadata-enhanced SE is better than the "other guy's," buyers are still left with to options: (1) in-plan only negotable fixed rate or (2) in-plan with out-of plan variable rate. Of course what is "out-of-plan" does not necessarily mean "everything else." Access to "out-of-plan" databases is negotiable and oftentimes excludes much of the vendor's databased inventory to reduce variable costs. The clear trend, however, in the private sector is in-plan only licenses for a selected set of resources at a fixed rate for classic or next gen current gen WEXIS licenses.

Now comes the silliness. What is the point of next gen current gen WEXIS federated search engines if WEXIS doesn't provide its entire databased content inventory at a fixed rate? Why not stop applying a 20th century pricing scheme to a 21st century platform? "Impossible!" WEXIS will say but not for a psudeo-technological rationalization. Why then? Because WEXIS is still applying last century's revenue generation model to the 21st century as if the legal services industry has not changed permanently.

Will the "impossible" become possible as the next normal pricing scheme for very expensive online legal search if BLaw's flat rate pricing for that Company's entire database universe becomes a competitive threat to WEXIS?  That may depend on BLaw reducing its per seat flat rate for something less than institution-wide licenses.

For a little background on what may be the next normal in the 21st century from the institutional buyer's side of this equation, see The No Sacred Cow Models: Sole Provider, Primary Provider, or Multiple Narrow-Focused Providers for Online Legal Search in the Private Sector. [JH]

New and Updated Research Guides from GlobaLex

Recent international, comparative, and foreign law research guides published by GlobaLex include:

New Guide

• Research Guide of the Legal System of Kingdom of Nepal by Md. Ershadul Karim and Sirjana Sharma Pokhrel.

Updated Guides

• An Overview of the Egyptian Legal System and Legal Research by Dr. Mohamed S. E. Abdel Wahab.
• A Guide on the Harmonization of International Commercial Law by Duncan Alford.
• Basic Primary and Secondary Information Online Sources for the North American Free Trade Agreement (NAFTA) and the United States Central American Free Trade Agreement-Dominican Republic (CAFTA-DR) Research by Francisco A. Avalos and Maureen Garmon; Update by Francisco A. Avalos.
• Researching the United Nations: Finding the Organization's Internal Resource Trails by Linda Tashbook.
• The Austrian Legal System and Laws: a Brief Overview by Johannes Oehlboeck and Immanuel Gerstner; Update by Johannes Öhlböck.

[JH]

JAG's The Operational Law Handbook 2012

From the Preface to The Operational Law Handbook 2012:

The Operational Law Handbook is a “how to” guide for Judge Advocates practicing operational law. It provides references and describes tactics and techniques for the practice of operational law. The Operational Law Handbook is not a substitute for official references. Like operational law itself, the Handbook is a focused collection of diverse legal and practical information. The handbook is not intended to provide “the school solution” to a particular problem, but to help Judge Advocates recognize, analyze, and resolve problems they will encounter in the operational context. Similarly, the Handbook is not intended to represent official U.S. policy regarding the binding application of varied sources of law, though it may reference source documents which themselves do so. The Handbook was designed and written for Judge Advocates practicing operational law.

The size and contents of the Handbook are controlled by this focus.

Did I mention that the PDF version provided in the above link is 576 pages long? To order copies call (434) 971-3339; or email usarmy.pentagon.hqda-tjaglcs.mbx.clamo-tjaglcs(at)mail.mil.

Hat tip to beSpacific. [JH]

How Open? Survey of Online State Primary Legal Resources

This is the sixth annual Open Access Week so it seems appropriate to see just how far states have progressed to provide open access to their primary legal resources online.

AALL: 2011-2012 Preliminary Analysis of AALL’s State Legal Inventories (September 2012; Updated October 2012)

Sunlight Foundation: State Regulations Online wiki page and Matt Rumsey's blog post, Survey: How Many States Publish Rules and Regulations Online?

[JH]

Zillman's Statistics Resources and Big Data on the Internet

Marcus Zillman's LLRX feature, Statistics Resources and Big Data on the Internet, is a comprehensive listing of statistics and big data resources and sites on the Internet. "This bibliography of sources", writes Zillman, "is taken from my Subject Tracer™ Information Blog titled Statistics Resources and is constantly updated with Subject Tracer™ bots." 

These sites will help you to discover a wide range of avenues through which you may locate and leverage the latest statistics resources applicable to many different types of projects, programs and research. As this site is constantly updated it would be to your benefit to bookmark and return to the above URL frequently.

[JH]

What Do You Think of Congress.gov?

Here's what Matthew Mantel thinks of the beta replacement for Thomas, New is Not Always Better -- It's Just New: Legislative Research edition, on Nota Bene.

End note. For those "how a bill becomes a law" lectures, see The Legislative Process mini-webinars hosted on Congress.gov for student assignments. [JH]

The "Duplication of Legal Publications" Issue: Recalling a "forgotten moment in the history of law librarianship in which a prominent law librarian provided leadership on a matter of concern throughout the legal profession."

Last weekend I read Dick Danner's very interesting and well-documented history of the ABA, AALL and AALS joint efforts in the late 1930s to address the problem characterized then as the "duplication of legal publications." Hopefully a couple of quotes from the opening paragraphs of Danner's The ABA, the AALL, the AALS, and the 'Duplication of Legal Publications' [SSRN; LLJ forthcoming] will stimulate interest by all three associations to join together in a concerted and coordinated advocacy effort with law librarian leadership to address today's issues despite the fact that the "forgotten moment" Danner recounts failed. One can make the case that the stakes are much higher today.

It was neither new nor unusual for lawyers to complain about having to deal with “too much law.” Historical concerns about too many law books are limited neither to common law legal systems nor to the post-Gutenberg age. Because of their reliance on precedents found in judicial opinions, common law lawyers in particular have complained about too many published opinions at least since the time of Francis Bacon in the early seventeenth century. The problem remains alive today in the background of twenty-first century controversies regarding citation of unpublished opinions in federal and state courts. American lawyers challenged by perhaps two million reported cases in the 1930s6 would likely be astounded at the number of appellate cases available since the introduction of commercial legal databases in the mid-1970s.

In the 1930s, it was not out of place for the ABA to be concerned about the problem of too many law reports. The problem of “duplication” of legal publications had been of interest to the ABA from the mid-1880s through the first two decades of the twentieth century, especially for the impacts of multiple versions of published law reports on the work of the practicing bar. In his history of the ABA, Edson Sunderland called it “one of the most baffling subjects” with which the Association dealt. By the 1930s, however, two other associations established near the beginning of the twentieth century had matured to where they too might exert influence on this and other matters of mutual concern. The American Association of Law Libraries had grown from thirty-four individual charter members in 1906 to 172 regular members in 1933.10 The Association of American Law Schools was formed in 1900, with thirty-two law schools as charter members, and had seventy-seven member schools by 1933.11 More recently, the American Law Institute (ALI) had been founded in 1923 by a group of prominent judges, lawyers, and law professors with the goal of promoting “the clarification and simplification of the law and its better adaptation to social needs.”

The 1930s initiative to solve the problem of the “duplication” of publications was significant not only because it was a joint effort by these organizations, but because it was coordinated within the ABA and with the other associations by Professor James, the Harvard Law Librarian. This article describes those efforts (and their eventual failure) in hopes of shedding light on a forgotten moment in the history of law librarianship in which a prominent law librarian provided leadership on a matter of concern throughout the legal profession.

(Citations omitted.)

Highly recommended. [JH]

The No Sacred Cow Models: Sole Provider, Primary Provider, or Multiple Narrow-Focused Providers for Online Legal Search in the Private Sector

On 3 Geeks, Greg Lambert reported that BigLaw Foley & Lardner has gone the sole legal search provider route (Lexis) recently.

For many of us in the law library world, we've been waiting for a BigLaw firm (Foley falls in at #45 in the AmLaw100 revenue rankings) to pull the trigger and go to a single legal research provider. Now we have someone to use as an example. Now that the seal is off that bottle, it will be interesting to see how many other BigLaw firms finally start looking seriously at dropping the two-vendor legal research model and start going as a Westlaw, Lexis, or even Bloomberg Law only shop.

I have no idea if Foley is the first AmLaw 100 firm to go the sole provider route. Certainly many mid-size law firms have gone sole provider since the 2008 recession because there are no sacred cows anymore. WEXIS no more; Lexis or Westlaw but not both. Do we add "or BLaw"? Not yet. With only two uber BigLaw firms on board for US-firmwide access and no public information that either have taken BLaw as a sole provider, about all one can say is that BLaw may be either one of two or three search providers or the primary provider offered to users at those firms.

The provider model is evolving from both Lexis and Westlaw to

• Sole Provider: Lexis or Westlaw but not both.
• Primary Provider: a more comprehensive plan for Lexis or Westlaw or BLaw with smaller plans as secondary provider(s) with or with limiting the number of user accounts at the firm level.

When the dust settles, I think there will also be a third model. Multiple narrow-focused licenses for Lexis Advance, WestlawNext and BLaw for select groups of institutional users based on a selection of in-plan only access to secondary sources that each practice group user population requires. Add into the institutional buyer decision matrix (1) any appropriate productivity "solutions" that tie into search needed by specific user populations of institutional buyers and (2) the eventual licensing of enhanced eBooks with their own tie-ins to the vendor's search service which will certainly cannibalize database inclusions in future search licenses.

Given that all three vendors provide essentially the same primary legal sources and that none of their next gen current gen metadata-enhanced search engines are convincingly better than any other despite marketing claims, my hunch is that eventually the dominate model in the private sector, particularly in the BigLaw sector, may end up becoming licensing multiple narrow-focused search plans from Lexis, TR Legal and  BLaw which are limited to targeted user populations based on the spectific secondary sources and tools needed online,

Of course, this is where the editorial quality of secondary legal sources becomes a competitive advantage in the context of the current WEXIS model of content commodization (think BLaw-BNA offerings) but also intelligent professional grade workflow solutions (think forthcoming WK offerings for some legal practice group specialities). [JH]

Sourcing Records on the US Constitution for Evaluating Original Intent Claims

In the "now for something completely relevant" category to the dustup between Scalia and Posner about Reading Law (2012), Gregory Maggs, Professor of Law and Co-director of the National Security and U.S. Foreign Relations Law Program at George Washington University Law School, has added a third installment in his series on sources of the original meaning of the Constitution. From the abstract of A Concise Guide to the Records of the Federal Constitutional Convention of 1787 as a Source of the Original Meaning of the U.S. Constitution [SSRN], 81 Geo. Wash. L. Rev. (forthcoming 2012):

The article describes the Constitutional Convention and the various kinds of records that were kept of its proceedings. The essay then explains, with examples, how judicial opinions and academic works draw upon the records for evidence of the Constitution’s original meaning, including both the meaning that the Framers may have subjectively intended the document to have and also other possible meanings. The essay next identifies and assesses seven important potential grounds for impeaching assertions about what the records show. Each of these potential grounds has merit in some contexts, but all of them are also subject to significant limitations or counter arguments. The essay, accordingly, recommends that anyone making or evaluating claims about the original meaning of the Constitution should proceed with caution, carefully taking into account both the possible grounds for impeaching claims and the arguments against these grounds. Appendices to this essay include an annotated bibliography and a table of the deputies who participated at the Constitutional Convention.

Earlier articles in Maggs' highly recommended concise guide series:

A Concise Guide to the Federalist Papers as a Source of the Original Meaning of the United States Constitution, 87 B.U. L. Rev. 801 (2007) [SSRN link] and

A Concise Guide to The Records of the State Ratifying Conventions as a Source of the Original Meaning of the U.S. Constitution, 2009 U. Ill. L. Rev. 457 [SSRN link]

Excellent additions to an ALR class reading list. [JH]

A Blue Book Bites The Dust

There are all kinds of “blue books” out there.  There are the Uniform System of Citations, the various reports from the Joint Committee on Taxation, and the venerable booklets in which law students write their exams.  And, if it matters, there’s the one for car valuation.  These are all safe, so far. 

I understand that Thomson Reuters is discontinuing the National Reporter Blue Book because of “insufficient market interest.”  I mentioned this to another librarian who remarked “What does it do again?”  I think that may have been the overriding result of any market research TR may have conducted on the decision to cancel the title.  For the record, it converts citations from official state reports to citations in the National Reporter System.  I don’t believe I’ve used the set in at least 25 years or more.  Parallel cites are easily available in every online legal research system and through just about every search engine out there.  The need for the National Reporter Blue Book as a print publication is nil.  I’m surprised it lasted this long.  [MG]

A Reference Desk Perspective on the Quality of Content Farm Information

From the abstract of R. Lee Sims and Roberta Munoz's The Long Tail of Legal Information: Legal Reference Service in the Age of the Content Farm, 104 LLJ 411 (2012):

The authors discuss the implications for legal reference service of a new feature of the legal information universe: the content farm. This article describes the content farm, its workings, what makes it profitable, and the market and informational forces that drive content farm creation. It also discusses how reference interactions may be altered if a patron has consulted content farm information before coming to the reference desk.

Recommended. [JH]

The Bluebook: There's an eReader app developer selling that

Bob Ambrogi reports that the first moble editon of The Bluebook has been launched for the iPhone and iPad using Ready Reference Apps' rulebook as its platform. While the rulebook app is a free download, the content is not. The Bluebook via Ready Reference Apps costs $39.99 or a one-year online subscription costs $32. Bulk licensing discounts are available. Bob notes that the print edition of The Bluebook costs $34. There is also an online web-based subscription service not produced by the app developer for the The Bluebook.  

Ready Reference Apps also sells the Federal Rules of Appellate Procedure, Bankruptcy Procedure, Civil Procedure, Criminal Procedure and Evidence priced at $1.99 each for it's rulebook platform. Bob notes that today (August 22) and only today the federal rules but not the The Bluebook will be available for free to all rulebook app users.

For more on rulebook's functionality, see Bob Ambrogi's LawSites post. Bob does not indicate plus I have my doubts that Ready Reference Apps has an exclusive license to be the sole provider of a moble edition of The Bluebook. [JH]

Beyond Copy and Paste: Intelligent Content EveryWare as the Next Wave of Technological Innovation from Wolters Kluwer

Unlike our other major vendors, Wolters Kluwer's executives and technologists in Europe present business ideas with a minimum of the usual marketing pablum on the Company's Intelligent Solutions Blog. In a recent two-part series of posts, WK's Jack Lynch, a member of the Executive Board of Wolters Kluwer whose responsibilities include global shared services, technology and business development, discusses the "next wave" business model for delivering contextualized and actionable solutions for intelligent data-intensive professional use.

I believe our business is evolving along two dimensions—1) The Computing dimension where we find ourselves today in the Post PC era barreling towards an era of Ubiquitous Computing or, what Adam Greenfield has termed, “EveryWare” and; 2) The Information dimension where today we are just beginning to contextualize information in different customer contexts as we continue to move along the data to information to intelligence continuum.

For more, see Part 1 and Part 2 of Lynch's "Next Wave" post series on the Intelligent Solutions Blog.

Angel Sancho Ferrer, Research & Development Director in Content & Online Services, Wolters Kluwer, South Europe follows up on Lynch's theme at Content in Search and Inference Engines.

The integration of content and software is a topic as broad as how to create Artificial Intelligence; and it is very deeply related with the core assets of Wolters Kluwer, with enriched content and algorithms that understand those special structures for research or workflow tools.

Expanding on the concept of "actionable content" presented by Lynch, he discusses the limitations of current search technology compared to rules-based inference engines.

Search technologies, as powerful as they have demonstrated to be, have limits. They:

• are reactive, and so depend on the quality of the query;
• cannot create information, just select the best documents and fragments (without modifying them, just copy and paste).

Rule systems, on the other side:

• Can follow a dialog-based approach to obtain more information from the user, and even from a software system, changing its internal states and strategy.
• Can create information that was not there (i.e. a computable document).

The delivery of actionable content by way of inference engines can be viewed as a solution that embeds search and professional-grade editorial content to generate made-to-order templates for work product. Think today's personal income tax preparation software only ratched-up well beyond Form 1040 rules for complex legal matters. This borders on Artificial Intelligence. But to make it work it will require specialist legal expertise with an editorial staff constantly reviewing legal developments to update inference engine rules.

One could argue the case that WK's US legal platforms are not even close to WK's "Next Wave" business development model. But they may be someday. If they do, WK presents a competitive threat to BLaw in several specialist market segments and could "nudge" BLaw and WEXIS to offer specialty-centric "solutions" that are state-of-the-art as defined by WK. [JH]

Living in the Past: Who will take the lead in establishing legal citation protocols now that the end of the print era is in sight?

The next edition of the Canadian Guide to Legal Citation "could prove to be a breakthrough edition if the editors choose to end the fiction that print law reports still matter in legal research" writes Gary Rodrigues. To do that, he recommends the Guide's editors should "introduce reality into the practice of citing court decisions by a few simple changes to the recommended Hierarchy of Sources for case law."

Rodrigues adds:

Of course, the legal publishers can cut to the chase and get ahead of The Guide to Legal Citation. Citations to electronic sources for the commercial publishers now appear in print publications for their own digital sources together with print citations.

Where it is unavoidable, citations for competing electronic databases are also referenced, i.e. where a judgment specifically refers to a competing electronic source. Ideally, legal publishers should follow the print model and provide all possible correlative electronic citations in every print publication. Without a doubt, the fear of giving a competitor an advantage makes that unlikely in the foreseeable future.

Fear of change will also delay decisions to drop print citations. Academics in particular will worry that their words will look less scholarly and less authoritative if they are not heavily footnoted by print citations. No publisher will want to be first for fear of being criticized by those living in the past.

Here in the US, legal publishers also could cut to the chase by adding paragraph numbering to court opinions in their e-text where pagination has not yet been officially eliminated for pinpoint cites. Who will be the first to take the bull by the horns by just doing it? Since what is and what is not a paragraph is a matter of interpretation, commercial vendors can copyright their individual paragraph numbering systems! Who will decide that a blockquote is (or is not a) new paragraph first just may win AALL's Best Product of the Year award someday.

Rogrigues explains why citation protocols should change because the era of print law reporting is history on Slaw in Reality Check. The same argument can be made for statutory and regulatory resources now and, in the not too distant future, for secondary legal sources. [JH]

Rai's Access to Legal Information in the Digital Age

The 2012 FCIL Schaffer Grant for Foreign Law Librarians receipent Priya Rai's powerpoint for her AALL Boston 2012 talk entitled Access to Legal Information in the Digital Age: A Comparative Study of Electronic Commercial Databases and Public Domain Resources in Law is available for download from the FCIL SIS website. Rai is the Deputy Librarian In-Charge at the Justice T.P.S. Chawla Library, National Law University Delhi, India. [JH]

(Well) I'm back in Blacks.

Aprroximately two weeks ago, I was instructing our incoming law students on how to read a judicial opinion. I implored them to keep a law dictionary within reach when reading case law, and to actually use it when they come upon a term of which they did not understand the meaning. I next went onto show them how to access some legal dictionaries online. Foolishly, I logged into WestlawNext (WLN), typed "Blacks" in the search box, and chose to enter the Blacks Law Dictionary database. At that moment, my foolishness was revealed to me. DENIED. I did not have access to Blacks Law Dictionary on WLN.

I tend to use either my personal hardcover version of Blacks that I keep in my office or my free Iphone law dicitonary app (sorry, Brian Garner, but I am not paying $54.99 for Blacks on my Iphone when I have sufficient free alternatives), so I don't usually try to access an online law dictionary through commercial sources, but I believe it to be improtant to provide ready access to ready reference material as useful and important as a law dictionary. If Ballentine's Law Dictionary has migrated to Lexis Advance, I cannot find it, and trying to locate it on LexisNexis is commonly a hassle for me (maybe I'm missing something).

I can recall a number of occassions that I tried to use Blacks Law Dictionary on WLN and was denied. So, yeah, the egg was on my face that afternoon. Luckily, I was able to somewhat save face by introducing th students to LII's Wex. Thanks, Cornell.

The good news is that I learned that Blacks Law Dictionary would soon be accessible via WLN. And when I checked it today, it was there. My first search in it: res judicata. (I've always just liked the way it sounds.) It is now on my favorites list and so next time the matter arises, I won't have to dodge the imaginary rotten tomato coming my way. So thanks for finally getting that worked out, West. Although I tend to quote my favorite philisophy professor, Arnold Rothstein, when he would say, "Better never than late," I will leave you with the more familair, "Better late than never." I'm just pleased that Blacks in back. (DCW)

A Pronouncing Dictionary of the Supreme Court

Sally Pei et al. "have undertaken to identify those cases decided by the Supreme Court of the United States that are most susceptible to mispronunciation and to determine the proper pronunciation." Details about the project at 13 Green Bag 2d 433. The pronouncing dictionary, hosted online by YLS, includes audio for the case names selected. [JH]

Braun Offers Tips on Free Online Legal Research Tools at the ABA Annual Meeting

Here's Stephanie Francis Ward of the ABA Journal interviewing Matthew Braun, legal reference librarian for the Library of Congress at this year's ABA Annual Meeting. [JH]

New Law Students: Make Friends With A Librarian

U.S. News & World Report has a short post out for new law students who will start their law school career in the next few weeks.  It offers four points for navigating law school:

1. Come prepared
2. Focus on finals
3. Make friends
4. Remove distractions

I’d like to focus for a moment on the third one.  The author suggests making connections within the law school and the wider university through activities and other diversions as a break from the law school routine.  That’s great advice as law school can be a highly competitive grind.  I’d like to make one other suggestion that may help the new student:  get to know a librarian.  Why?  Because we know stuff that students do not.

We know the cycle of the law school academic year.  It normally doesn’t shock us when 1Ls invade the library for the legal writing treasure hunt.  It’s not exactly the running of the bulls, but there are certain parallels.  For students it’s a new experience.  For us, it’s “been there done that.”  Don’t be afraid to ask questions, even for the simple stuff.  There’s usually a ready answer that can make a student’s life easier.

We also know the resources.  It may be nice to have all of those apps on tablets and phones.  And I’m sure there are plenty of new law students who have a lot of experience doing research in college.  Legal research, however, is a different animal.  Lexis, Westlaw, and Bloomberg Law are not free in the wild.  The mechanics can be a challenge despite the trend to provide Google-style interfaces.  Librarians understand how this stuff works and can help.

I’ll offer related word of advice:  not everything is online.  More important, not everything is online and is free.   It’s more often than not a pay for play world.  Luckily the law library has licensed a lot of the good stuff.  We can tell students what information is easily accessible and what is not.  We can also explain how to get remote access to stuff, like articles, hard to find documents, exams, and other materials.

Librarians know the law school.  We’re not the concierge for the school, but we know how it works.  We can tell you generally which office likely handles what responsibility.  We can tell you where are public copiers, scanners, microform readers (yes, they are still necessary) and other useful resources may be.  We can also tell you generally what your expectations should be in using them.  The library is probably the friendliest location in the law school.  Students use the library facility regularly in spite of the integration of technology into the curriculum.  We try to make the place a comfortable and quiet space to study.  Take advantage of that.

So, make friends with a librarian.  We won’t break the rules for you.  But our institutional knowledge of the law program and legal information can make a student’s life easier.  We even know where the bathrooms are located.  Don’t be afraid to ask.  [MG]

Susan Nevelow Mart and Jeffrey Luftig on The Case for Curation

From the abstract for The Case for Curation: The Relevance of Digest and Citator Results in Westlaw and Lexis [SSRN] by Susan Nevelow Mart and Jeffrey Luftig (both Univ. of Colorado):

Humans and machines are both involved in the creation of legal research resources. For legal information retrieval systems, the human-curated finding aid is being overtaken by the computer algorithm. But human-curated finding aids still exist. One of them is the West Key Number system. The Key Number system’s headnote classification of case law, started back in the nineteenth century, was and is the creation of humans. The retrospective headnote classification of the cases in Lexis’s case databases, started in 1999, was created primarily although not exclusively with computer algorithms. So how do these two very different systems deal with a similar headnote from the same case, when they link the headnote to the digesting and citator functions in their respective databases? This paper continues an investigation into this question, looking at the relevance of results from digest and citator search run on matching headnotes in ninety important federal and state cases, to see how each performs. For digests, where the results are curated – where a human has made a judgment about the meaning of a case and placed it in a classification system – humans still have an advantage. For citators, where algorithm is battling algorithm to find relevant results, it is a matter of the better algorithm winning. But no one algorithm is doing a very good job of finding all the relevant results; the overlap between the two citator systems is not that large. The lesson for researchers: know how your legal research system was created, what involvement, if any, humans had in the curation of the system, and what a researcher can and cannot expect from the system you are using.

This paper was the basis of a presentation at AALL Boston 2012. The authors have requested in the SSRN version that the paper not be cited without permission so I won't quote from it. But their work is an interesting read. Recommended.

Hat tip to Legal Research Plus. [JH]