Law Librarian Blog

What Can You Find By Rummaging in the Federal Government's Attic?

Certainly, some interesting electronic copies of federal documents provided by Governmentattic.org via FOIA requests. For example, Collection of US Supreme Court Legal Maxims, Compiled by the Department of Justice (DOJ) Civil Division Appellate Staff, [1993 through 1998 Terms] is a very interesting quick reference tool that provides 

a collection of useful quotations from all opinions of the United States Supreme Court from its 1993 through 1998 Terms. These are intended to be used as a research tool for finding authority to cite in briefs and memoranda for common propositions of law. They are grouped into five categories: (1) Doctrines of Constitutional Construction (in the 1998 Term maxims only), (2) Doctrines of Statutory Construction, (3) Supreme Court Practice, (4) Procedural Doctrines, and (5) Substantive Law Doctrines.

I think identifying legal maxims for a SCOTUS or state supreme court term could be a very productive group project for any legal research course.

Other interesting finds that caught my attention include

• US Department of Justice (DOJ) Office of Information Policy (OIP) template paragraphs used for responses to FOIA requests and appeals and OIP Administrative Appeal Letter Database and
• Numerious federal agency log requests including, for example, Mandatory Declassification Review (MDR) Request Logs for the National Security Agency (NSA), 2002-2010

For history buffs interested in Hoover era of the FBI, there are many documents like

• Federal Bureau of Investigation (FBI) File 71-HQ-5415, James Robert “Bob” Langston – Midwest Stag Film and Burlesque Showman, 1952-1953 and
• Federal Bureau of Investigation (FBI) file 62-HQ-46855: Book Reviews, 1959-1966

Plus there are numerious FBI files on President Nixon's longtime nemesis, Pulitzer Prize winning syndicated newspaper columnist Jack Anderson (use the site's search engine).

Yes, Governmentattic.org has an internal search engine but I didn't have time to see if any ETs were issued SSNs. [JH]

 

Browsing On A Sunday: Do Courts Cite Internet Legal Resources?

I’m doing a lecture on Internet legal research soon and I was wondering how valid are some of the sites I teach to the courts.  I find Google Scholar very useful in my day job.  Has anyone ever mentioned it in an appellate opinion?  Yes, it seems, once, and only very recently.  The California Supreme Court issued an opinion on January 27^th called Vandermost v. Bowen, --- P.3d ----, 53 Cal.4th 421, 2012 WL 246627 (Cal.), 12 Cal. Daily Op. Serv. 1119, 2012 Daily Journal D.A.R. 1110 (citations and search results generated from Westlaw) with this reference:

By contrast, academic observers have concluded that the Commission's maps, including the certified state Senate map, “represent[ ] an important improvement on the legislature-led redistricting of 2001. The new district boundaries kept more communities together and created more compact districts while at the same time increasing opportunities for minority representation.... [T]hese maps ... have the potential to modestly increase competition in California elections and the responsiveness of the legislative branch to changing voter preferences.” (Kogan & McGhee, Redistricting California: An Evaluation of the Citizens Commission Final Plans, supra, 4 Cal. Journal of Politics and Policy ____ (forthcoming Jan. 2012; available via Google Scholar at <http:// polisci2.ucsd.edu/vkogan/research/redistricting.pdf>, pp. 32–33 [as of Jan. 27, 2012] ).)

I think the Court should be citing the University of California San Diego, which is the actual source of the cited document at the end of the quote.  Interestingly enough, the Court tipped its research strategy in finding the document, a forthcoming publication no less.

The Cornell Legal Information Institute gets one state hit in American Home Assur. Co., Inc. v. Unauthorized Practice of Law Committee, 121 S.W.3d 831, 2 A.L.R.6th 783, Tex.App.-Eastland, November 06, 2003 (NO. 11-02-00212-CV), and seven in the federal courts.  The Oyez Project gets a reference in a federal case quoting Chief Justice Roberts’ reaction to a particular statute.  The reference is to an oral argument.  The citing case is Evans & Green, LLP v. Meadoworks, LLC, Slip Copy, 2012 WL 137885, W.D.Mo., January 17, 2012.

FindLaw gets a whopping 23 references, though some of those are involved around litigation involving FindLaw, and a few more relate to accessible resources by prisoners as part of access to a prison law library.  The news feed at Leagle.com gets at least one mention in a 2010 Michigan case.  The name “Leagle” shows up either as a personal name or as a misspelling.  Justia gets five hits, though a few are in reference to litigation involving the site.  As for government citation, GPO Access gets two citations, while the more current FDsys.gov has nothing yet.  Regulations.gov gets five hits.

The point for me is these sites have enough respectability to be cited by the courts, meaning they have enough respectability to be used and cited by others.  We teach students to use the original source, and court rules tend to enforce that concept.  But with courts starting to go to legal content on the free Internet, we can’t discount some of these Internet legal sites, at least as a matter of reputation.  [MG] 

Reminder: Registration for AALL Webinar on Educational Storytelling as an Instructional Technique Due by Feb. 16

Back in the day, I would use one of my own legal research stories for a research class lecture or while working alongside an anxious young firm associate performing research when appropriate. In my case, storytelling was oftentimes a parable that sounded in the research log and writing process to serve as a warning -- "you may think you have finished research and can now just write but the odds are high that during the analytical process that goes into writing, you will be returning to perform more research until the work product is ultimately finalized."

Quoting from the description for AALL's Feb. 22, 2012 Developing and Using Stories to Teach Legal Research webinar:

[S]torytelling ... serves as an effective communication technique in trial practice, business administration, and library science.  While little used in law librarianship, storytelling has the potential to transform the way law librarians convey legal research techniques and legal information to law library patrons.  This comprehensive webinar will introduce educational storytelling, and include a demonstration of a legal education story as well as methods for collecting, developing, and evaluating potential stories.

If storytelling is or has become a technique little used by law librarians, it can be a valuable technique that IMHO should be used in the classroom and in patron assistance. It certainly isn't always appropriate but the technique ought be in the law librarian instructional toolbox. Again quoting from the webinar description:

This comprehensive webinar will introduce educational storytelling, and include a demonstration of a legal education story as well as methods for collecting, developing, and evaluating potential stories.

(Emphasis added.)

This isn't a matter of telling "war stories." The key here is developing stories which are instructional for the patron in the library and other stories for group audiences. In the context of legal research instruction in law schools, storytelling needs to dovetail into the intent of the course. Is the course designed to teach reseach skills to students so they can conduct legal research during their academic career? Is the course designed to teach practice-oriented legal research to students?  

Moderated by Mark Estes, the webinar's speakers are Vanessa Christman (formerly Uribe), El Dorado County Law Library Director, and Beth Wrenn-Estes who is a full-time lecturer in the School of Library and Information Science at San Jose State University. Should be interesting.

Developing and Using Stories to Teach Legal Research
February 22 at 11 a.m CST
Cost: $30 for AALL members; $60 for nonmembers; $150 for site registration (per physical site)
Registration and payment due by February 16 at 5:00 p.m. (CST)

For additional information and to register, go here. Do note, space is limited. [JH]

Reminder: Contributions to the 20th Annual RIPS-SIS Teach-In Kit Due Friday, Feb.10th

Want to help promote law libraries and enhance legal research instruction? AALL's Research Instruction and Patron Services-SIS publishes its popular Teach-In Resource Kit online in conjunction with National Library Week each April. The deadline for submiting contributions for this year's edition in Friday, Feb. 20, 2012. So there is still plenty of time to send in your contributions as email attachments to RIPS-SIS Teach-In Kit co-chair, Laura Ax-Fultz, Access Services Librarian, Dickinson School of Law, Penn State Univ., lja10(at)psu.edu.

Teach-In Kit instructional materials can include:

• course syllabi
• research guides
• lecture notes
• handouts
• assignments
• lesson plans
• PowerPoint shows
• basic instructional guides
• examinations and guided quizzes
• crossword puzzles and trivia quizzes

For examples of past submissions, see last year's Teach-In Kit.

The Teach-In Kit producers welcomes ideas and suggestions and encourages you to contact them to discuss your contributions. Contact information here. [JH]

Congressional Lawmaking: A Perspective on Secrecy and Transparency

From the summary of the CRS Report, Congressional Lawmaking: A Perspective On Secrecy and Transparency, (R42108) (November 30, 2011):

Openness is fundamental to representative government. Yet the congressional process is replete with activities and actions that are private and not observable by the public. How to distinguish reasonable legislative secrecy from impractical transparency is a topic that produces disagreement on Capitol Hill and elsewhere. Why? Because lawmaking is critical to the governance of the nation. Scores of people in the attentive public want to observe and learn about congressional proceedings. Yet secrecy is an ever-present part of much legislative policymaking; however, secrecy and transparency are not “either/or” constructs. They overlap constantly during the various policymaking stages. The objectives of this report are four-fold:

• first, to outline briefly the historical and inherent tension between secrecy and transparency in the congressional process;
• second, to review several common and recurring secrecy/transparency issues that emerged again with the 2011 formation of the Joint Select Deficit Reduction Committee;
• third, to identify various lawmaking stages typically imbued with closed door activities; and
• fourth, to close with several summary observations.

[JH]

New Legal Skills Textbooks from Carolina Academic Press

Federal Legal Research
by Mary Garvey Algero, Spencer L. Simons, Suzanne E. Rowe, Scott Childs, Sarah E. Ricks
Forthcoming January 2012

Federal Legal Research explains how to conduct research in the U.S. Constitution and in federal cases, statutes, and administrative regulations. The book begins with an overview of the sources of law and the research process. That chapter is followed with an in-depth discussion of American legal research strategies and techniques for both print and online sources. The book covers secondary sources and practice guides, updating with Shepard’s and KeyCite, and legislative research. A separate chapter focuses on legal ethics and court rules.

Federal Legal Research is effective for teaching legal research in first-year classes that integrate research, writing, and analysis as well as in upper-level courses with a more bibliographic approach. Moreover, the book will provide accessible information about federal legal research for practitioners, paralegals, librarians, college students, and even laypeople.

Effective Lawyering: A Checklist Approach to Legal Writing and Oral Argument, 2d ed.
by Austen L. Parrish, Dennis T. Yokoyama
Forthcoming January 2012

Effective Lawyering takes a unique approach to legal writing and oral advocacy. Many excellent legal writing books exhaustively detail how to write effectively. Those books — which are written primarily for first-year law students and are often several hundred pages long — meticulously explain the dos and don’ts of effective advocacy and provide numerous exercises for students to complete. Effective Lawyering, which can serve as a useful supplement to these lengthy introductory texts, takes a different approach. The book assumes the reader has learned the basics of legal writing, and at most needs only to be reminded about them. The book also assumes that most practitioners (and, for that matter, law students) lack the time to read lengthy discussions of all the subtleties of legal method.

Advocacy to Zealousness: Learning Lawyering Skills from Classic Films
by Kelly Lynn Anders
Forthcoming January 2012

Advocacy to Zealousness: Learning Lawyering Skills from Classic Films includes 26 skills, listed in alphabetical order and appearing in separate chapters, which should ideally be in every lawyer’s toolbox. Each is an example of professionalism, and all are possible for every current or future lawyer to attain or sharpen. They also speak to our needs in the legal profession in the new millennium. Interspersed throughout the text are subtle references to diversity, both in the practice of law, as well as in various film references and storylines.

Each chapter includes a discussion of the skill and its use for lawyers, a synopsis of the film associated with the skill, film discussion questions, and brief exercises for improvement. The format encourages readers to either methodically review each chapter in alphabetical order, or skip around as needs and interests dictate. The book concludes with a comprehensive index.

Images, be they positive or negative, are powerful and long lasting. Those found in popular films are often our only points of reference until we meet the real thing — or assume the role ourselves. Many professors already use film clips in their classes, but, until Advocacy to Zealousness, there was no singular point of reference for films selected solely for the purpose of fostering and sharpening lawyering skills. Legal skills training needn’t be dry or cumbersome. With creativity and planning, it can instead be educational, memorable, and enjoyable — while also remaining comprehensive and relevant — thereby enhancing the practice of law and the legal profession as a whole.

Wouldn't some of the above legal skills titles make excellent AALL p- and enhanced e-book imprints? See Dear Santa, Please Give AALL a Clue on How to Make Money. [JH]

Top Ten Evidence Issues for 2012 and 2011 (Could provide issues for ALR assignments)

For litigators at the federal trial level, is there anything more important that evidence issues? Federal Evidence Blog has published two posts covering Top Ten Evidence Issues for 2012 and 2011, retrospective and prospective. After assigning the 2011 recap analysis, the 2012 issues could make for good advanced legal research assignments. [JH]

Let's Start with Basic Skills: Teaching Law Students Reading and Reasoning Skills

Obviously there has been much more change-talk than action in improving the legal academy curriculum. Perhaps that is because the legal academy is a bit befuddled about where to start. Even if we accept the premise that law schools try to teach students to "think like lawyers" (instead of to "practice like lawyers") the profession is still an intellectual activity that must teach law students reading and reasoning skills as fundamental to research and communication skills by way of work product, client relations, and court room advocacy.

In Scott Fruehwald's recent Legal Skills Prof Blog post, he calls attention to Dorothy Evensen, James Stratman, Laurel Oates, and Sarah Zappe, Developing an Assessment of First-Year Students' Critical Case Reading and Reasoning Abilities: Phase 2 (LSAC 2008) as demonstrating that the legal academy isn't doing it's job of teaching law students to think like lawyers because students aren't acquiring the law-centric reading and reasoning skills they need.

In Do We Need to Change Our Teaching Methods? Fruehwald writes

To follow the nuances of this report, you will need to read it in detail, which I recommend that everyone who is interested in the future of legal education should do. In any case, it demonstrates that we cannot say that our traditional methods of law teaching are working, and we need to develop and use new methods to teach basic skills.

Highly recommended. Those of us who have been in the legal research instruction "business" for sometime know that you really can't provide a meanful education in how to conduct legal research until students know how to read and reason in a lawyerly manner. Anyone who thinks the tool box approach to teaching legal research a/k/a as the presentation of legal bibliography is good enough, should be tarred and feathered. I'm certainly in favor of this medieval act of retribution and disgrace for acadmic law librarian and legal research instructors at this stage in the 21st century; if they are only given enough class time to teach this way, "just say no."

Hell, let's extend the whole tar and feather thning to everyone in the legal academy who just continues to engage in "change-talk" without doing something about it. This has been talked to death. Time to move forward. [JH}

PACER Users Training Site

Hat tip to Deborah K. Hackerson, Law Librarian & Legal Research Adjunct, Univ. of St. Thomas School of Law, for her FYI Legal Skills Prof Blog post about PACER's training site. Do note that according to Nota Bene's Nov. 13, 2011 post, authored by law librarians at Houston Law's O'Quinn Law Library, the site leaves something to be desired:

[N]ot all database content is available in the training site. Currently it includes information and documents from real cases filed in the Western District of New York between 1/1/2007 and 7/1/2007.

[JH]

Providing Users with Search Algorithm Information

Matt Cutts, Distinguished Engineer, recaps ten changes to Google's algorithm implemented during the last couple of weeks on Google's official Inside Search blog including, for example:

Snippets with more page content and less header/menu content: This change helps us choose more relevant text to use in snippets. As we improve our understanding of web page structure, we are now more likely to pick text from the actual page content, and less likely to use text that is part of a header or menu.

Better page titles in search results by de-duplicating boilerplate anchors: We look at a number of signals when generating a page’s title. One signal is the anchor text in links pointing to the page. We found that boilerplate links with duplicated anchor text are not as relevant, so we are putting less emphasis on these. The result is more relevant titles that are specific to the page’s content.

Refining official page detection: We try hard to give our users the most relevant and authoritative results. With this change, we adjusted how we attempt to determine which pages are official. This will tend to rank official websites even higher in our ranking.

Improvements to date-restricted queries: We changed how we handle result freshness for queries where a user has chosen a specific date range. This helps ensure that users get the results that are most relevant for the date range that they specify.

Very expensive legal SE vendors take note. The "secret sauce" is not publicly disclosed by providing this sort of information. But it could help users and law librarian-instructors acquire a better understanding of today's new legal SEs. Since some of our very expensive legal vendors are re-engineering their SEs for the Google generation, perhaps they should steal another idea from Google by creating their own official "inside search" blogs. [JH]

The New Search Model: "Everyone plugs into the search engine, only a few understand how it works"

A definition of the algorithm:

The algorithm is an effective method expressed as a finite list of well-defined instructions.

And that is the problem today's new legal search engines present for users and instructors of online legal research in the context of online legal bibliography (OK, by referring "legal bibliography" I admit I'm pretty damn old school in my perspective). The "push" is toward contextually sensitive legal search. Download the Cognizant 20-20 Insights white paper, "The Future of Legal Search: Meeting Lawyer Requirements by Delivering More Contextually-Sensitive and Relevant Results" (Sept. 2011) from the link provided in Stanford Law's  Paul Lomio Legal Research Plus post. Highly recommended.

The objective is to read a legal researcher's mind by way of search engine technology that incorporates usage patterns generally or individually. Based on recent renditions like WestSearch and Bloomberg Law, we aren't there yet. But we are already facing the consequences of today's SEs that are striving to get there.

Reflecting on three decades of teaching ALR courses, Stanford Law's Bob Berring writes:

We used to catch them in ALR as second and third year law students. They had context from a year of law school and many had worked or interned in a law office or public interest office for a summer so they knew that they needed to know more. In 2011, the motivation is waning. The thrust of what Google does removes the searcher from the search process. WESTLAW/NEXT and LEXIS/Advanced mimic Google’s search scheme as much as possible. We are asking people who want to drive a car to understand how the engine works.

... Everyone plugs into the search engine, only a few understand how it works.

Not that I shall give up. My plan is to go down fighting, but I think that I know how this movie ends.

For more, see Berring's Legal Research Training’s End on Slaw.

I personally want to believe that our user population is smarter but we do have to address the issue that a generation of users may not give a damn about how today's SEs work --- at least not until their over-reliance on them based on their ignorance of the finite nature of their programming leads to inadequant client representation. No judge is going to let them off the hook when they plead "mercy" because they relied on WestlawNext, Bloomberg Law or Lexis Advance. Their professional responsibility will not be one qualified by reliance on an online legal search service's limitations. Knowing how these new SEs actually work by way of legal research instruction in law school as well as in practice settings by experienced and expert professionals -- that would be us -- is crucial for legal researchers.

My user population is typically "older." That doesn't mean they aren't (1) smart enough to grapple with the new SEs or (2) don't know better than to over-rely on advanced SE results. In fact, they have far more legal expertise and experience in performing online legal search than the typical ALR student. But if the user population doesn't how the finite set of commands drives these new SEs, how to work them or work around them to perform research, the wiser course is not to rely on them until there is no choice but to do so because the vendors' legacy systems are shutdown.

And they will be shutdown eventually. If WLN's adoption rate by Big Law is any indication -- only half of the AmLaw 100 have bought into the WLN upgrade-- legacy services will be killed off sooner rather than later. Want to bet on the over-under that WEXIS legacy services will be "history" five years after their new services have been fully implemented?

Do note for over-under betting purposes where we stand:

• In the premium online search market, IntelliConnect is essentially fully implemented already, albeit with versioning improvements in an attempt to recapture a user population that fled to BNA Online after the first release of IntelliConnect. Bloomberg Law's current search engine replaced its earlier version.
• In the generalist online search market, WLN still isn't ready for prime time, well we all know that by now, but eventually it will be the only search service TR Legal sells. Lexis Advance's market segmented rollout of Advance's multiple versions isn't completed yet.

I think it is fair to say that the only people who really know how today's very expensive legal search engines work are vendor programmers who create the unseen algorithms. See Traffickers of Very Expensive Online Legal Search: How do we use and teach today's legal search services when we don't know how the search engines work? The other hurdle is the mindset of legal researchers Berring describes in his Slaw post.

I wonder (read am concerned] if aging and decrepit law librarians like me will be viewed someday as luddite reactionaries by a future generation of legal researchers who don't care that they don't know how the search engines they are using work. It looks like the draft script authored by traffickers of very expensive online legal search want this to be how this movie's ends. My hunch is some vendors will say that legal researchers are finding what they need, so what's the big deal?

The big deal is that if the new model of search, at least as exemplified by WLN and BLaw, is being programmed to try to read a legal researcher's mind to find the "perfect" source document or small set of source documents that hits the mark. SE developers are creating a unrealistically high goal for themselves and the users of their services. The state-of-the-art in search technology simply isn't "there" yet. Trying to program for the "contextually sensitive" by embedding usage patterns globally or at the user account level is questionable at best at this time. One would hope that vendors would provide an "off switch" for that (and other elements) in their new SEs instead of a switch-back button to their legacy SEs like one has because we know legacy systems won't be around forever. [JH]

The Clinical Legal Association is Soliciting Contributions for Follow-up Book to Best Practices in Legal Education

The Clinical Legal Association, Best Practices Implementation Committee is planning a follow-up publication to Best Practices for Legal Education by Roy Stuckey and others.  The vision of the book is to build on ideas for implementing best practices, and to develop new theories and ideas on Best Practices for Legal Education.  If you are interested in authoring a section for the new publication, contact Professor Deborah Maranville at maran(at)uw.edu or Professor Antoinette Sedillo Lopez at lopez(at)law.unm.edu as soon as possible. Then by Dec. 1, 2011, send one of the profs a 3-5 page abstract identifying the knowledge, skills and values as well as the learning objectives and methodology of your innovative teaching idea. The Editorial Board will meet at the AALS meeting in January to select pieces for inclusion in the book.

Hat tip to Jim Levy at Legal Skills Prof Blog. [JH]

Creating Video Tutorials of Today's Online Search Services: The example of Fastcase endorsing best practice videos authored by law librarians

Got to hand it to Fastcase for calling attention to video tutorials created by law librarians:

We recently came across a series of well-crafted Fastcase Video Tutorials created by our partners at the Jenkins Law Library. We were impressed with their work and we think you will be too.

Based on this statement plus my personal experience with some of the folks at Fastcase, I'm assuming the Jenkins Law Library is the independent author of the tutorials listed below. They were intended for the Jenkins Law Library's Fastcase user population. The videos take a no nonsense approach to the topics covered. The fairly visible hand of product promotion offered in some vendors' in-house tutorials is not seen here. One may say Fastcase has "adopted" these videos because they have been endorsed as "best practices in Fastcase search."

I'm thinking the folks at WestlawNext, Lexis Advance, Bloomberg Law and IntelliConnect might learn something about providing law librarian created video tutorials of their very expensive online legal search services that are not tainted by their marketing departments. My hunch is there are law librarians "out there" willing to do this for free (except for waiver of any search charges) to help fill in the huge gaping hole in reliable instructional materials for those new search services.

The glitch here, however, is law librarians have to understand how the search engines work. The crux of that matter is that today's search engines embed propretary information. My suggestion to very expensive online legal search vendors is that they reveal enough of that information to law library staffers who are willing to author their independently produced tutorial videos.

No one is under the illusion that the recipes for the secret sauces of the unseen algorithms are going to be disclosued by our vendors to the public at large. However, only the naive will believe vendor claims about "how great" their new SEs work. And vendor published testimonials are simply absurd. Further, no one is going to believe vendor hand-picked law librarians for some wacky sort of testimonial "tutorial" video series.

There is a fairly simple solution for starting to address the problem of vendor SE programming gurus knowing how today's crop of SEs work while their user populations don't. If law librarians offer to create a series of "best practices" videos, offer them sufficient details of the secret sauce under a NDA. Then let them create their video tutorials with the sole restriction that the secret sauce cannot be explained in specificity in the videos they produce. At least this would be a step in the right direction.

In the Fastcase tutorial videos that were produced by Jenkins Law Library, my favorite is the one that goes over Fastcase's Authority Check feature. I doubt the Jenkins Law Library staff were given the secret sauce but Fastcase has provided enough of a description of how Authority Check works already, at least for my aging and decrepit brain to understand (perhaps not so for must smarter law librarians who specialize in legal research).

For folks who want some no nonsense video tutorials on Fastcase, see below. For vendors of very expensive online search, the challenge is open to them to let some sunlight in by what I consider would be a "best practices" for independent development of tutorial videos by law librarians based on the terms and conditions above. Reasonable?

Jenkins Law Library Video Series as described by Fasecase

• In part one, get the basics on the next generation of legal research offerings within the Fastcase database and where to start based on your search requirements. It's a great refresher course!
• In part two, learn the basics of a Boolean keyword and natural language search to yield more accurate search results.
• In part three, learn how to get the most of the results page using Fastcase's smarter sorting tools.
• In part four, learn about viewing a case and generating an Authority Check report to find later citing cases.

OK, I know, Fastcase couldn't resist the urge to toss in some promotional language in their brief descriptions of the videos but ... that's as far as that goes. All of the videos can be viewed on Fastcase from this link. [JH]

GPO Access Goes Archive Only Once Today's Edition of Daily Updated Content Has Been Uploaded

From Cynthia Etkin's Nov. 3, 2011 message posted on multiple listservs:

GPO Access Goes Archive Only

On Friday, November 4, 2011, the U.S. Government Printing Office (GPO) draws one step closer to shutting down GPO Access. Once the Friday editions of daily updated content (e.g., Federal Register, Congressional Record) have been uploaded, GPO will cease updating GPO Access in terms of both database content and HTML pages. This will mark the start of the archive only phase of GPO Access and new content will only be loaded to FDsys. During this phase, GPO Access will remain publicly accessible as a reference archive.

In order to make the switchover from GPO Access to FDsys as seamless as possible for users, GPO is in the process of creating one-to-one redirects from GPO Access content to the FDsys equivalent. This will ensure that bookmarks, Web links, URLs in print publications, and other GPO Access references point to valid Web resources. Once this has been completed, GPO Access will be taken offline. A date has not yet been established for the final shutdown of GPO Access; however, it is slated for fiscal year 2012.

Libraries should take this opportunity (if they have not already done so) to review their Web sites, presentations, brochures, and other materials that reference GPO Access and work to update or replace these materials. This includes imagery and URLs.

* Download FDsys logos <http://www.gpo.gov/fdsysinfo/logo.htm>
* View instructions on how to create links to FDsys content <http://www.gpo.gov/help/index.html#fdsys_url_structure.htm>

Thank you for your patience and assistance while we make this transition.

If you have questions or comments, please use the askGPO help service at: <http://gpo.custhelp.com/>. Select "FDsys" as the category and "GPO Access - FDsys Transition Question" as the sub-category.

Cynthia Etkin
Sr. Program Planning Specialist
Office of the Superintendent of Documents
U.S. Government Printing Office

[JH]

CALI Call for Proposals to Create State Specific Legal Research Lessons

From the announcement:

The Center for Computer-Assisted Legal Instruction (CALI) is on a mission get a state specific legal research lesson covering primary and secondary materials for each of the 50 states.   We're close, but we still need at least one lesson for the following states:  Alabama, Alaska, Michigan, Mississippi, Missouri, Montana, New Hampshire, New Jersey, New Mexico, New York, North Dakota, Oregon, Rhode Island, South Carolina, South Dakota, Vermont, Virginia and Wisconsin. 

If you are a librarian at a CALI member institution and would like to write a lesson for one of these states, we are currently accepting proposals.  The deadline for current round of lesson proposals is November 4, 2011. Authors receive monetary compensation, free registration to the CALI conference and editorial support though the lesson writing process.  Full proposal requirements and details can be found here.

[JH]

Traffickers of Very Expensive Online Legal Search: How do we use and teach today's legal search services when we don't know how the search engines work?

One can go back to when the issue of CALR was first much debated in the late 1970s, to view what some might characterize as a "luddite" response to the advent of very expensive online legal search but many of those articles sounded in the Great Unknowing at the time. The authors simply were not accustomed to thinking in terms of performing legal research in the context of selecting databases, using logical operators, overriding SEs predefined ordering of logical operators, performing segment searches, etc., and teaching others how to do the same. We are repeating history with a new Great Unknowing in very expensive online legal search. Unlike in the past, this is because advances in SE programing are metadata-rich proprietary information. This is not a luddite reaction calling for a return to terms and connections but our vendors are not providing sufficiently detailed information to replace our old Venn diagram understanding of yesterday's commercial legal search engines with something that maps today's more advanced SE programming.

Take for example, WestSearch. Does any practicing law librarian or legal research instructor really know how WestSearch's algorithms actually work? A fair number of specific WestSearch anomalies have been spotted by experienced legal researchers (that would be law librarians) that have been published in the blogosphere; you'll find some if you look for posts published 12-18 months ago when law librarians were given trial WLN accounts as well as posts published when academic law librarians were preparing to start instructing students in WLN. More recently, Anon, for example, has tested TR Legal marketing claims for WestSearch at The WestSearch Straitjacket For Legal Research - Thinking Beyond The Keyword, Part I and Part II. Ron Wheeler, for example, has closely examined some issues by way of trial-and-error testing, particularly the crowdsourcing "usage pattern" facet of WestSearch at Does WestlawNext Really Change Everything?: The Implications of WestlawNext on Legal Research, 103 LLJ 359 (2011). Do note that Ron Wheeler never got a definitive answer from TR Legal's WLN developers to the question, will crowdsourcing not miss what Wheeler calls "esoteric" materials? In other words, we do not have any idea if WestSearch will produce truely comprehensive search output for the diligent legal researcher. I do not rest any easier learning from Mike Dahn in his interview with Jason Wilson on rethinc.k that after testing, the contribution of the "usage patterns" component in WestSearch was reduced in WestSearch's algorithm.

None of those law librarian reports generate the kind of confidence one would hope from a very expensive 21st century search engine like WestSearch. All one can say is that this sort of trial and error process has been prompting the question: why did I get this WLN output in this search results display? This Great Unknowing also presents another important question, namely, how does one teach WLN to students, to members of the the bench and bar? Ah yes, all this is "proprietary," so much so that our vendors' reps and their sales managers who are trying to sell these services don't really know how WestSearch works.

TR Legal is a trafficker in online legal search using algorithms we know very little about except that some metadata is baked-in and crowdsourcing, oh, my bad, "usage patterns" are factors in WestSearch's algorithms. We have no clue what every factor is and how each factor is ranked because this is "proprietary." Therein lies the real problem with 21st very expensive online legal search.

For me, the issue is much broader than "usage patterns." What exactly are all the elements of the WestSearch algorithm and what is the ranking of each? How does each factor contribute to WLN search results. We don't know? Perhaps it wouldn't be an issue if long-time Classic Westlaw users didn't experience WLN search result display shock and then clicked on a button to switch to Classic Westlaw. What happens when Classic Westlaw disappears? Why? Because trial-and-error just isn't going to cut it. Hell, it can be damn expensive in conducting WLN research in the private sector.

This is hardly a new issue.

More than two decades ago, Robert Berring, speaking of electronic databases, wrote:

The danger of the high-end products is that each step in the research process that is carried out automatically by the front end system, is a step taken away from the purview of the researcher. Each decision that is built into the system makes the human who is doing the search one level further removed from the process.

Berring’s words should serve as a reminder to librarians and teachers of legal research. We must strive to understand as much of the research process as possible, even the steps carried out by online algorithms, so that we can develop and teach effective strategies for achieving our research goals.

(Citation omitted; emphasis added.) Quoting from the last paragraph of Ron Wheeler's LLJ article.

While not a new issue, I think it has become any increasingly more important one as greater complexity is being baked into today's very expensive legal search engines.

If we look at TR Legal's Patent Application, we can see the big picture. Download it here. If I was still teaching legal research, I would make it required reading for a class on WestSearch and I would use the graphics published in it, like the one displayed right (click to enlarge), as the 21st century but substantially less instructive version of Venn diagrams.

I'm very willing to embrace new search engines but the devil is in the details and those details are represented by the "lightening bolts" is some of the patent application's diagrams. According to Symbolism Wiki, a bolt of lightning "is a symbol of loss of ignorance. It also represents the punishment of humans from the Gods." When an experienced legal researcher has an all too human WTF reaction to WLN search results, it may be punishment from the gods of programmers in their equally all to human software routines. Our ignorance of WestSearch is based on not knowing how the algorithm (technically, algorightms) work. Mike Dahn's description in his rethinc.k interview, quoted below, does nothing to increase our knowledge with needed specificity:

To dramatically improve search beyond what standard keyword based search engines can do, our WestSearch algorithms primarily rely on our editorial enhancements, things like the Key Number System, KeyCite, Headnotes, Statutes Notes of Decision, and the language correlations we have in our proprietary indices –  like “see also” references.  We’ve literally been building up this collection of editorial enhancements for over a hundred years, and it provides both extraordinary search results and a significant competitive advantage over what others can do in the marketplace.

About the objectives of WestSearch in its development stage, Dahn explains:

One of our concerns was about user experience – we wanted researchers to get very noticeably better results – better enough to pay a premium for our new product.  It couldn’t just be arguably better – it had to be noticeably better.  Our other concern was a competitive one. We were investing a lot of money in WestlawNext, and in the search engine specifically.  If employing usage data drove most of the benefits in terms of precision and recall, then our competitors could turn around quickly and do similar things. We needed to find out what mattered most and why.

Note that Dahn recognizes that upgrading very expensive online legal search is required to stay competitive. I personally don't believe a subscriber base needs to be paying for the corporate cost of staying competitive by paying a premium even if the search results were "noticeable better." That's how corporations maintain their competitiveness even in a duopoly. With TR Legal's profit margin plummeting from 33% to 25-26%, one might think the pricing gurus would consider eliminating a WLN premium since "noticably better" results might shore up its subscriber base, might even increase it.

But in this case, it is arguable to even say WestSearch results are even "arguable better" than Classic Westlaw except, perhaps, in caselaw research. Criticism from experienced legal researchers (ah, that would be us), have discussed some nasty results in federal and state statutory and regulatory research and in an apparent bias against secondary sources. We also know that West's traditional topical analysis (e.g., West Key Numbering System) now incorporated as metadata is caselaw top heavy. Perhaps that is why some WLN users find WestSearch "good" for caselaw, not so for statutory and regulatory law. Of course, we are merely speculating... . And, of course, those reports are products of trial-and-error research by law librarians.

TR Legal has had their WestSearch staff tackle some of those "anomalies" and WLN searches performed to test marketing claims. When they come up with different results that claim to contradict law librarian results, should we simply assume their claims are correct because TR Legal says so? I'm thinking it would be very interesting to see proof of those claims in ALR assignment fashion, namely the entire research log.

If we assume TR Legal's claims are true for the sake of argument (or proven to be true by law librarians replicating the research logs WestSearch staff used), what does that tell you? It tells me the WestSearch staff knows how the SE algorithms work, unlike us, their customer base, who do not have the same amount of detailed information.

Comprehending how legal search engines work must go far beyond trial and error. Law librarians are legal research specialists working in real time. Cost aside, we can perform online legal research using any SE as long as we can understand the output results based on a clear understanding of how the SE works. We can modify our online search results based on knowing how the SE actually works. The Great Unknowing is that today we don't know how 21st century SE algorithms work. Nor is TR Legal responding to this need in any substantive way. Want to sell me on the benefits of WLN, better provided more detailed information on how the algorithms produce the results I get (that is to say got when I gave it a test drive).

Law librarians and other legal information professionals are focusing on WestSearch, but WLN isn't the only 21st search engine that presents these issues. Bloomberg Law's SE "learns"? OK, how? Good luck trying to get an answer to that question. I guess we will have to wait and see if Lexis Advance learns from WLN's mistakes. Might be time to do some patent research.

Mike, buddy, no one is asking for the exact recipe of the secret sauce in WestSearch. The only people who would understand that would be search programming brainiacs employed by other very expensive online legal search vendors and they have/are probably reversing engineering WestSearch to dissect it. But just saying that X, Y and Z are part of the equation and that the "usage pattern" factor is part of that equation but was demoted in priority during testing isn't good enough. Every professional law librarian, including a former law librarian, knows that. [JH]

What Was the Legal Status of "Wanted - Dead or Alive" in the Wild West Days? (Assuming those posters weren't a creation of Hollywood)

Kenneth Anderson wants to know:

• First, to what extent was this actually a historical practice [in 19th century frontier days], or is it really just a creation of Hollywood westerns– any scholarship on this, or actual examples?
• Second, from the standpoint of the law in force in that period, on what basis was this kind of “dead or alive” language used?
• Third, what was the relevant Constitutional doctrine of that day and did it have a role to play?  Why or why not?

See also the comments to his post on The Volokh Conspiracy.

What a great ALR assignment! Certainly could take law school students beyond WEXIS to perform research. [JH]

The WestSearch Straitjacket For Legal Research - Thinking Beyond The Keyword: Part I

Thomson Reuters - Legal (TR Legal) advertises sweeping claims for the benefits of WestlawNext (WN) at Customers. WestlawNext.com. Given the scale of the advertising blitz, these claims merit careful examination. Though it remains too early to reach a conclusion, initial evidence from its advertising suggests that TR Legal oversells WN for the benefits of its search engine, WestSearch. Despite benefits unique to its design, WestSearch has not only limitations from its own design, but the same type of keyword limitations that any other search engine has. As TR Legal markets WestSearch as if it should supplant other uses of WN for legal research. Its marketing reveals a one-size-fits-all reductionism at cross-purposes with WN’s versatility, but in perfect harmony with WN’s increased costs.

Lawyers must think like lawyers to correctly and comprehensively answer legal questions. Legal research therefore requires legal analysis and related problem-solving skills. As Kent Olson observes, “[l]egal research is not simply gathering information, but being able to analyze that information and grasp its significance. It is a process that requires a siginficant amount of time and focus.” (Principles of Legal Research (2009), at 8) Yet marketers for WN attach supreme value to speed in legal research. By failing to distinguish WestSearch and WN, they claim that WN - when they mean WestSearch - allows legal researchers to accomplish their research “fast” but without any loss of “confidence.” ("Meet The People Who Made [WN] Possible") WN retains less obvious features essential to research goals other than speed. As WN’s marketers try to attract interest in WestSearch, WN’s developers have left legal research educators back doors to help their students learn how to overcome WestSearch’s shortcomings, even if the adaptations will, in some instances, almost double “retail” costs.

In the first part of this series, we will separate the marketing hype from the realities of WestSearch. We will find that it does not necessarily advance effective legal research, even if it (modestly) advances effective keyword searching. We will identify some of its limitations, using them to underscore the value of “traditional” research methods still available on WN where WestSearch falls short. Examples alllow us to anticipate increased “retail” costs for traditional research uses of WN. In the second part of the series, we will examine a study for TR Legal that allegedly demonstrates the time-saving benefits of WestSearch. By underscoring WestSearch’s limitations, the study’s biases will help us better understand its benefits. We will also better appreciate the perils of reducing legal research to keywords in multiple databases. Legal researchers, of course, can not wear this kind of straitjacket. They must think beyond keywords.

TR Legal’s marketing campaign features comments by customers, including such law firm partners as Daniel Kelly, of Reinhart Boerner Van Deuren, and Kathryn Conde, of Nutter McClennen & Fish. Kelly says that WN "gives us a quick view of the question that we have to answer and then it leads us right down the road, each step of the way, to figure out exactly what the answer needs to be and how to present that." Conde claims that WN saves her clients money by producing "quick and correct answers" to legal questions. Other smiling customers, including librarians and associates, endorse WN (or rather WestSearch) in the same enthusiastic ways. Oddly enough, almost of them work at Reinhart Boerner or Nutter McClennen.

At Customers. WestlawNext.com, customers can "go behind" WN by exploring "The Science of WestSearch." Its developers associate characteristics with WestSearch that no other search engine has. Mike Dahn, Vice President of New Product Development, says that it ends "the tyranny of the keyword,” even though it depends on user-selected keywords. How are these apparently conflicting ideas reconciled?

To understand what Dahn means, we will begin with a fact: keyword searching has had a record of limited effectiveness.  Keyword searches in large databases fail to uncover all, and only, the documents that keyword searchers need to correctly answer their questions. Keyword searchers often end up with too many unhelpful documents or miss too many helpful ones. The constraints arise from the limitations of keywords, the incomplete knowledge of keyword searchers, and the “keyword irreducibility” of complicated legal questions.

First, keyword searches work only as well as keywords track the right concepts. The concepts meant may diverge from the keywords selected. In response to the problem, Dan Dabney helped develop KeyCite. Before joining West Publishing Company, he identified three aspects of this problem in The Curse of Thamus: An Analysis of Free-Text Document Retrieval 78 Law Libr. J. 5, 17-20 (1986). Whether as individual words or words in expressions, keywords imperfectly correspond to concepts because of ambiguity, synonymy, and syntactic complexity. Document authors use different words or expressions for the same concept (synonymy), the same words or expressions for different concepts (ambiguity), and words or expressions whose complex syntactic relationships designate conceptual relationships (syntactic complexity). Examples to follow will illustrate these types of language-related problems.

Second, keyword searches work only as well as the keyword searcher can identify all of the legal concepts needed to correctly answer a question. If you miss applicable concepts because you lack adequate context for the question, then you will also neglect keywords that might be useful to include in your search.

Third, a keyword search works only as well as a legal question lends itself to keyword compression. A complicated question involves a wide range of concepts, facts, and relationships among them. No one keyword search will prove sophisticated enough. And any number of keyword searches may not suffice to cover the entire conceptual and factual territory.

Its developers designed WestSearch to end “the tyranny of the keyword” by mitigating problems of synonymy, inadequately understood contexts, and complicated questions. (Peter Jackson & Khalid Al-Kofahi, Human Expertise And Artificial Intelligence In Legal Search (2010), at 3, 6) WN users may start their research by entering "simple descriptive words, terms and connectors, or even a citation" in WestSearch. (TR Legal, Westlaw Next Search Brochure (2010)) They need not identify a specific source or type of source. WestSearch’s algorithm weights and links topically related legal documents for importance and relevance, whether or not (1) the documents have the keywords entered; (2) the user has entered keywords to cover all of the applicable concepts; and (3) a complex question can be adequately compressed in keywords.

WestSearch represents a “further evolution “ from the Natural Language search engine of “Westlaw Classic” (WC) and other conventional search engines, which rank documents for relevance according to keyword properties. Conventional search engines deploy  “term-frequency-inverse-document- frequency”. Thus the less frequently a WC case database has the keywords of a Natural Language query, the more of these keywords will appear in top-ranked cases, the more closely together the keywords will appear in them, and the more often top-ranked cases will use these keywords. (Robert A. Berring and Elizabeth C. Edinger, How To Find The Law (12th ed. 2005) at 94, n.4) WestSearch’s design displaces the primacy of keyword properties in ranking search results.

So how does WestSearch work? WestSearch developers have issued a number of patent applications for the new technology. We can begin to understand WestSearch’s ranking based on at least two of them. The following explanations therefore represent a simplification.

WestSearch ranks legal documents for relevance to the issue and importance in use. To rank documents for importance, WestSearch has been designed to capture more than KeyCite’s data on citation frequency. It also captures crowdsourcing by WN users, or patterns in documents they link to, view, print, or save in folders. One crowdsourcing factor concerns the extent to which WN users click though document hyperlinks, especially among WN users who tend to do most of the click-throughs. (Jackson and Al-Kofahi, at 5; U.S. Patent Application No. 20100332520, published Dec. 30, 2010)

To rank documents for relevance, WestSearch requires a set of “starter” documents, such as cases, that satisfy keyword criteria of a Boolean, or “plain English,” search query. (U.S. Patent Application No. 20080033929,  “Exemplary Operation” at [0028 - 0050], published Feb. 7, 2008) “Term-frequency- inverse-document-frequency” supplies one of the “search capabilities”  for retrieving starter cases.” (Patent Application No. 20080033929, at [0021]; TR Legal, WestSearch: Westlaw Next Search Technology (2011), at 3) Search algorithms automate comparisons between starter cases and other cases. Comparisons involve extent of textual similarity between starter cases and other cases. The comparisons also involve shared “meta-data” or document characteristics - the number and importance of shared Key Numbers; the number of shared citations to cases and statutes; and the number of shared citing cases. Each compared case receives a rank, or relevance score, based on these comparisons and the case’s meta-data. The meta-data of a ranked case include the number of sources citing it; the number of sources cited by it; the number of starter cases related to it by shared Key Numbers, shared citations, shared citing sources, or other links; the number of relationships between it and starter cases; and the number of these relationship types. Thus the top-ranked cases may be linked by their Key Numbers to each other, and frequently cited for the same points of law. They may also be linked by shared citations and citing cases.

The ranked cases, in turn, have “meta-data” features that support further links to primary and secondary legal authorities. Based on relative weights, or ranking scores, attached to their meta-data features, ResultsPlus determines what additionally linked primary and secondary authorities appear in the results list, and in what order of relevance. (Patent Application No. 20080033929, at [0015]). ResultsPlus “suggests materials related to retrieved cases, including ALR annotations and Am Jur articles.” (WestSearch: Westlaw Next Technology, at 3) “Suggested” materials also include treatises, law reviews, and legal briefs and trial documents. (TR Legal, ResultsPlus Fact Sheet (2008); U.S. Patent Application No. 20050228788, published Oct. 13, 2005) Inventors of ResultsPlus developed it from an automated classification technology called the “Categorization and Routing Engine, or CaRE. (TR Legal Press Release, ResultsPlus for Statutes Search Now Available on Westlaw, Mar. 29, 2003) CaRE aids TR Legal editors when they assign Key Numbers to headnotes by “suggesting” Key Numbers for unclassified headnotes.  (To learn more about the CaRE invention, see U.S. Patent No.7,065,514 (issued Jun. 20,2006).

Cases represent an “exemplary embodiment” of the WestSearch technology. The technology has comparable applications specific to other categories of legal documents, such as statutes, regulations, administrative decisions, briefs, law reviews, and and treatises. For each “content category,” a search query retrieves a set of starter documents. Search algorithms automate comparisons between starter documents and other documents, ranking each compared document by extent of textual similarity and shared meta-data, and by the document’s weighted, meta-data characteristics. ResultsPlus then links the ranked documents with other primary and secondary sources. (See Jackson’s comments on ResultsPlus in: Jason Wilson, On WestlawNext, State of the Art & Steve Jobs: A Conversation With Peter Jackson, Chief Scientist for Thomson Reuters (Jun. 24, 2010)). Thus WestSearch has been designed to rank search results not just across, but within these “content categories.” It “is not simply a global or federated search of multiple databases containing different document types. Rather, each content type has its own search engine, employing a total of thirteen different algorithms.” (Human Expertise And Artificial Intelligence In Legal Search, at 6)

Nevertheless, if we rely on the Brochure, WestSearch generally works so well that its users can be confident that the top-ranked search results include only relevant and important documents, and that they miss no relevant and important documents. So if we credit the advertising, WestSearch’s developers have untied the Gordion knot of online searching. "The algorithm [used in the search engine] retrieves legal documents of high recall, as well as high precision." (WestSearch: Westlaw Next Search Technology, at 4) Because the search engine ranks relevance in the right ways, its users can expect to retrieve most of WN’s documents on the legal issue of interest (high recall). Because the search engine ranks document importance in the right ways, its users can also expect that most of a search’s results will include such documents (high precision), particularly among the top-ranked results. (On WestlawNext, State of the Art & Steve Jobs: A Conversation With Peter Jackson, Chief Scientist for Thomson Reuters)

Unfortunately, “the tyranny of the keyword” can also stand for an alien marketing ideology - the belief, contrary to common sense, that keyword searching exhausts legal research methods. If we think beyond the keyword, we will realize the difference between effective legal research and effective keyword searches - and the difference in cost. So suppose a single use of WestSearch typically achieved high precision and high recall. WN may not "transform" legal research even in two instances where its developers tell us it should. We have no evidence that running a WestSearch makes it any easier, faster, or cheaper to answer complicated questions than starting with secondary sources or consulting experienced practitioners. We also have no evidence that alternatives to WN work less well at helping legal researchers identify all the legal principles at issue when they lack subject expertise. In fact, we have the following reason to think WN works too well in one undesirable respect, by inflating costs.

Olson recommends that researchers learn more about unfamiliar topical contours of their questions. They should do  “some preliminary research … to understand the context of a particular problem and to get some sense of the rules of an area of law.”  (Legal Research In A Nutshell (10th ed. 2010), at 32) Subject treatises, for example, can help them in traditional areas of law, and legal encyclopedias may help if no subject treatises are available. Law reviews, legal newspapers, and legal blogs can afford insights about new or emerging areas of law. Preliminary research may also help researchers learn the topic’s “legal jargon” to prepare better keyword searches. (Finding the Law, at 13-14

Of course, WN search results may alert legal researchers to issues they would otherwise overlook. But they need not start with a WN search to remedy this deficiency. Secondary sources in a law library appear more likely than WN searches to save legal researchers time, and their clients unnecessary search costs, as long as researchers have ready access to a law library, and know how to effectively use the sources or receive instruction. At "retail," transactional rates, a WN search costs $60, and then $42 to view a section of a “premium state [or] speciality title.” (TR Legal, WestlawNext Pricing Guide for Commercial Plans (Feb. 2010)) WN search results may include a larger collection of relevant secondary sources. But you may incur unnecessary cost to run even one search on WN, and view several documents, at least if you can gain the needed context from a treatise or legal encyclopedia in your institution’s law library. Moreover, it will cost almost twice as much in WN than in WC to view “speciality title” sections. (TR Legal, Westlaw Pricing Guide For Private Price Plans (Apr. 2010))

Though hardly decisive, available evidence at least suggests that WestSearch will not achieve high recall and precisions for certain types of questions, even though they frequently arise in legal research. WestSearch’s design causes at least two important problems; the complexities of language cause others for all search engines.

WestSearch will not accommodate research to find and understand the required legal concepts, if finding and understanding them depends on factors other than those ranked by WestSearch's algorithms. In Does WestlawNext Really Change Everything?, Ron Wheeler challenges WN's effectiveness for questions whose answers depend on "the obscure or less popular results" that crowdsourcing would not uncover. (103 Law Libr. J. 359, 366 (2011)) In addition, he says that "[t]he legal researcher using WestlawNext and doing broad searching will be exposed not only to fewer cases, but also to a far narrower range of related content." (372) In other words, as a legal researcher learns more about a legal question or problem, the researcher may lose opportunities to better define it by missing relevant legal authorities or secondary commentary.  So WestSearch may not achieve high recall for broad search queries. In Part II, we will see why it may not, based on Patent Application No. 20080033929.

Moreover, WestSearch does not close significant gaps between concepts and keywords used to express them, even if it narrows such gaps. The slipperiness of language limits the usefulness of all keyword search platforms, requiring legal researchers to think less like keyword searchers and more like lawyers. Let us now turn to examples of these limitations. The examples involve three types of questions quite common to legal research. Where WestSearch might lead to (expensive) dead ends, other research methods may lead to correct and comprehensive answers. Thus if a researcher understands a legal problem’s context, the researcher may start by identifying a promising source for an answer, and then navigate its content through its map-like system of organization. In our examples, legal researchers will need to use research aids, such as annotated codes, legal encyclopedias, and treatises, and will need to use associated navigational devices, such as indexes, topical outlines and tables.

These alternative methods will cost more to use in WN than in WC. We will base our transactional cost comparisons on two pricing plans for private-sector customers: Westlaw Pricing Guide For Private Price Plans (Apr. 2010) and WestlawNext Pricing Guide for Commercial Plans (Feb. 2010). Our price comparisons will hardly settle opinion on WN’s overall cost impact, especially because almost no WC or WN customer pays “retail.” But they will appear to fit the wonder-bra principle of cost shifting: squeeze the costs in in one direction and they pop out in another.

Consider first the problem of synonymy. Suppose a terminally ill man stores sperm for his wife to use after his death. The couple wants to ensure that a child born under these circumstances has the same right to inheritance as if the child had been born while his father was alive. Do California couples have a legal procedure to protect this form of inheritance? They do, under Cal. Prob. Code §249.5, but that provision references “posthumous conception,” “genetic material,” and a maximum period when the “child was in utero.”. Technical language of this kind abounds in statutes and regulations. Use of at least one of these phrases increases the odds of a successful keyword search. So would starting with a specific source, such as West’s Ann. Cal. Code. But an attorney new to this area of law may overlook such helpful phrases when searching WN’s entire collection of California legal materials. Suppose the attorney enters store sperm terminally ill in WestSearch, limiting the content to just “California” as the jurisdiction. The attorney will not find Cal. Prob. Code §249.5 in the search results, and may end up wasting money by viewing documents not on point. But the attorney could instead use the index to West’s Ann. Cal. Prob. Code, if available, at a savings of at least $60 and perhaps considerably more. Under the entry for “Children and Attorneys,” the attorney would find the entry “Posthumous conception, probate proceedings, Prob 249.5 et seq.” Whatever their shortcomings, the index entries require the user to think like a lawyer.

Now consider the problem of ambiguity with respect to a sample question. Under what circumstance, if any, can an attorney appearing as amicus curiae file an appeal on behalf of a party? This unlikely circumstance occurred in Matter of Additional Magistrates for St. Louis County, 580 S.W.2d 288 (Mo. 1979). A court issued an order making appellant “a party as amicus.” The Additional Magistrates court decided that the appellant retained standing to appeal, because the lower court treated the appellant as a party. For keyword searching, the ambiguity arises from  the pedestrian sense of “party to a case” and the more exceptional sense of “amicus as a party.” A skillful WN user would need to run several searches of federal and state case law  just to eliminate irrelevant cases, such as cases  in which courts barred amici from raising new issues on appeal. Nevertheless, even a few advanced WestSearches (whether Boolean or in “plain English”) will not likely uncover Additional Magistrates. (If Additional Magistrates was frequently cited or “crowdsourced” among WN users, WN searches might stand a better chance of success.) The cost of several searches may approach $200. But a legal encyclopedia - 4 AmJur. 2d. Amicus Curiae §6 - affords a legal researcher a crucial alternative, because it has a citation to Additional Magistrates. Browsing the “Amicus Curiae” outline of AmJur. 2d will point a researcher to “II. Rights and Powers of Amicus Curiae - §6 Generally.” That amorphous “Generally” would tempt the curiosity of an experienced researcher. But temptation has its costs. Retrieving this section in WN will cost about twice as much it would in WC.

Finally, legal researchers may need to answer syntactically complex questions. Dahn claims that WestSearch can “break down the words and phrases in legal documents … [and] recognize the syntactic relationships among them.” (WestSearch: Westlaw Next Search Technology, at 3) But his claim does not survive the test of syntactic complexity. Dabney offers us an example of a syntactically complex question:

“If a person a waives his or her right to trial by jury in one trial, can a jury trial still be demanded in a subsequent new trial of the same matter? The key words for this question, 'trial,' 'jury,' 'waiver,' and 'retrial,' are common in judicial opinions, but discussions of the specific point of law of the question are relatively rare. A computer cannot reliably find cases that are on point because too much of the meaning of the desired cases is tied up in the syntactical relationships between the words, which are not 'understood' by the computer." (78 Law Libr. J. 5, 19-20)

To try answer a syntactically complex question, a keyword searcher must first identify legal phrases or keyword combinations specific to the legal concepts involved in the question. Sometimes no specific legal expressions apply, or the researcher can not anticipate expressions that would apply. Sometimes the applicable legal expressions do not help. Legal language often causes these difficulties.

Although Dabney was discussing Boolean searching, his observation also applies to Natural Language searching in WC and “plain English” searching in WN. Entering "simple descriptive terms" in WestSearch will no more help a researcher answer Dabney's question than using Natural Language or Boolean searches in WC. To correctly and comprehensively answer the question, a researcher should instead identify a likely secondary source and use its table of contents, topical outline, or index. Suppose you are a California attorney. Anticipating criminal law as the likely context, you could start with Witkin & Epstein, Cal. Criminal Law (3d ed. 2000). Using WN or WC, you could browse the table of contents to find the relevant discussion:

Chapter XIV. Criminal Trial - X. RIGHT TO AND SELECTION OF JURY - A. Right to Jury Trial - 4. Waiver of Right. - h. [§ 459] Effect of Waiver on Retrial.

WN users can still access the table of contents of Cal. Criminal Law and other secondary sources. But it will cost them more to do so. Under a private plan’s  "retail," transactional rate, it costs $24 to retrieve Cal. Criminal Law § 459 from WC; however, it costs $42, or almost twice as much, to retrieve this section from WN.  This difference in cost has significant consequences for charges to clients and cost recovery, because the difference will have iterations every time WN users rely on WN’s print-based features. Thus Cal. Criminal Law § 459 cites to People v Solis, 66 Cal.App.4th 62, 77 Cal.Rptr.2d 570 (1998). The case would lead you to another helpful secondary source:

Although there are no published opinions in California specifically addressing whether a jury trial waiver remains in effect for a subsequent retrial of the same case, the contention has been considered and rejected in many other state and federal courts. (United States v. Groth (6th Cir.1982) 682 F.2d 578; United States v. Lee, supra, 539 F.2d 606, 610; People v. Mixon (1994) 271, 111. App.3d 999, 208 Ill.Dec. 385, 387, 649 N.E.2d 441, 443; People v. Hamm (1980) 100 Mich. App. 429, 298 N.W.2d 896, 898; State v. Di Frisco (1990) 118 N.J. 253, 571 A.2d 914, 930; see also Annot., Waiver of Right to Trial by Jury as Affecting Right to Trial by Jury on 573-573 Subsequent Trial of Same Case in Federal Court (1984) 66 A.L.R.Fed 859, 869, § 7 and cases cited.)” (77 Cal.Rptr.2d 570, 572-73).

(Links omitted)

It costs $24 to retrieve the ALR document from WC, but $46 to retrieve it from WN. And if the ALR led to other secondary sources, the cost difference repeats itself.

In summary, we have seen how three types of legal questions can disable keyword searching as a useful research method. The questions tend to to disassociate or sever the ways in which search algorithms match keyword combinations with documents having the right legal concepts. Therefore, they typically require legal researchers to think beyond the keyword. Answering them correctly may require source-specific, research methods more finely tuned to use of analogies, distinctions among similar or related legal rules, and other forms of legal reasoning. The research may take more than a few minutes, especially if the researcher must use multiple research aids and methods. Fortunately, WN, no less than WC, still provides users research aids to leverage the print-based organization of particular databases. But using them on WN will cost more.

By overemphasizing WestSearch as the principal or exclusive way to use WN, WN's marketers reduce legal research to "simple descriptive terms,” applied to searches across a wide range of different types of sources. Yet the analysis legal research often requires can not be reduced to “simple descriptive terms.” So WN’s design reveals a troubling reductionism that  perpetuates another form of “keyword tyranny.” As it has been marketed, the WN model represents a radical departure from the model of traditional research that Wheeler describes in his article. The “traditional model” requires a researcher to understand the context of a legal question, the likeliest sources for answering it, and the structure of these sources. Under the traditional model, researchers also understand the trade-offs involved in using Boolean and Natural Language searches, search engines, and print-based or “pre-coordinate” methods of research.

In Part II of this series, we will examine TR Legal’s evidence that  WestSearch supports fast research by achieving high recall and precision. TR Legal relies on an “independent” study whose biases reveal WN’s reductionist tendencies and invite worries that the study’s designers lack independence. But the study will help us assess WestSearch’s benefits.

[Anon.]

Resistance? No Doubt, but the Timing is Ripe to Make an Attempt: Reforming Legal Research & Writing Instruction Requires Transforming the Status Quo Model of Law School Curriculum and Faculty Governance

Alas, I seriously doubt any such reform will by mandated by the ABA's review of accredition standards since not interfering with the how the legal academy should produce "pratice ready" grads beyond aspirational goals for "outcomes" appears to be the ABA's response to widespead criticism from the bench, bar, and recent grads. Well, what the hell should you expect from the ROFL claims that the ABA Section of Legal Education and Admission to the Bar is "separate and independent of the American Bar Association, as required by the Department of Education’s regulations governing accrediting agencies" while including way too many members of the legal academy with vested interests in maintain the status quo to the maximium extent possible.

Plus there is the equally ROFL claims from some law profs who in a nutshell claim that it is their law school administators' fault, not theirs because the Dean and the University make all the important decisions. Ever attend faculty meetings? I have. It was part of the job requirement. Faculty members routinely chimed in with their opinions on how to run the law school before and after the Dean made his decisions. The tenured faculty were not afraid to express their opposition to proposed administrative actions. (Tenure-track profs, less so). At this law school, it got so bad the senior law faculty did what was necessary to get this dean, who still had about six months left in his first contract, fired.

Do note that not all law profs are so willing to hide behind the shield of blaming their deans and university administrators by claiming a disconnect in responsibility for this state of affairs. OSU law prof Douglas Berman writes

So, even as I grow more aware/attune to the "law school scam" and suffering grads, I still have a hard time viewing law professors as avaricious Ponzi schemers eager to drive students into a lifetime of debt to fund a lavish lifestyle.  Instead, I see a group of well-meaning service-providers (law professors/schools) working earnestly to provide what they consider a valuable non-economic service ("teaching students how to think like a lawyer") to some people who are paying a lot of money (law students) problematically believing they are getting a valuable economic service ("becoming a practicing lawyer").  

To the extent that deceptive placement stats fuel this disconnect between the "law school service" most law professors seek to provide and what some (many? most?) law students actually want and expect, more honest employment data should help considerably.  But if one fears (as I do) that much bigger societal and human psychology forces are in play, more honest employment data is just a first step on a long journey toward a sounder legal education system.

(Emphasis added.) See Berman's Identifying the disconnect at the center of the "law school scam" post for more.

Bottom line: the battle to produce "practice ready" law school grads will be fought law school by law school by law school. The war will only be "certified" as won by the ABA's so-called independent Section of Legal Education and Admission to the Bar after the fact. Certainly, AALL's contributions to this process provide little nudge-generating momentum despite our association's claims that AALL is working closely with the ABA to make changes. Even the The Boulder Statement on Legal Research Education is devoid of the specificity needed, as well-intented as it was.

The only alternative to the law school-specific battlefields is for Congress to get involved by way of hearings that includes but goes well beyond the usual establishment witness list (eg ABA, AALL, etc.). I'm thinking Senator Grassley and Senator Boxer may be sufficiently dissatisfied with responses to their inquires to the ABA. But unless they call Chief Justice Roberts to testify, the matter may not produce the sort of media attention senators like. See, for example, Mark Giangrande's recent post, The ABA's Latest Response To Senator Grassley (I like the idea of holding accreditors of law schools to justifying their standards and actions by someone with enough gravitas to pull it off.  Perhaps that conversation the Section is seeking with Senator Grassley might take the form of a Senate Judiciary Committee hearing at some point. That would be a hoot.}

In Think [And Practice] Like a Lawyer: Legal Research for the New Millennials, [SSRN], Aliza B. Kaplan, Associate Professor of Legal Analysis & Writing, Lewis & Clark Law School, and Kathleen Darvil, Access Services–Reference Librarian and Adjunct Assistant Professor of Law, Brooklyn Law School, write

To properly prepare law school graduates for today’s legal marketplace, research instruction needs to be revamped. Emphasis should be placed on researching within law’s hierarchy, using a broad range of tools, regardless of whether that tool is available online or in print. Cost-effective research strategies should be taught to students, stressing the utility of popular secondary sources and other efficient search strategies and techniques. In order for new attorneys to be competent, competitive, and an asset to their employers, law schools should seek out the advice of practitioners and law firm librarians in the development of the legal research curriculum.

The authors offer several general recommendations on how to reform the legal research curriculum to advance the objective of teaching law students to think and practice like lawyers:

[W]e recommend that legal research instruction should use 1) multimedia technologies, 2) exploratory and collaborative strategies, 3) relevant subject matters, 4) and cost-effective tools. It should be integrated throughout the curriculum and taught by experts, which would convey its importance and assure adequate training. Making some timely and necessary changes to legal research instruction will integrate more practical skills into the law school curriculum and help us better prepare our Millennial students to be skilled practicing lawyers.

Do note that in reviewing criticisms from legal employers about the current state of legal education, one can conclude that in the context of generating practice-ready work product, the case can be made that law schools are not even successful at graduating students who think like lawyers, let alone who are "practice ready" in terms of their research skills training. Could this be because thinking and practicing like a lawyer require more than listening to doctrinal class lectures and more that requiring students to take a separate mandatory 1L LRW? Most of us inside and outside the legal academy know that the answer is and has been "yes" for decades.

The authors write:

Today’s legal research instruction neither trains students for the complexities of practitioner research nor reinforces good research habits and skills. Research instruction is generally mandated only in the first year of law school, which is insufficient to develop the skills necessary for legal practice. It also sends the message that research is not an important skill. Moreover, many legal writing professors who teach research do not have expertise in legal research, which would not be the case with any other specialized subject in most law schools. In order to best prepare our students to practice law, legal research instruction should be integrated throughout the law school curriculum, taught by those with the appropriate level of expertise, and should incorporate the resources and technology used by practitioners.

Sometimes specifics can use as important illustrations. In this case, the authors offer a couple of illustrations about how legal research instruction can be integrated thoughout the law school curriculum. Take for example, reading assignments that move beyound the casebook; for a course like CivPro, the authors suggest assigning reading in Moore's Federal Practice and/or Wright & Miller to expose students to the types of publications they may actually use in the real work. For research paper exercises in doctrinal course, assigning hot tops to exposure students to recent developments titles, like some of BNA's reports, because they will find them to be valuable current awareness services when students are practicing law. The authors also address some commonly used productivity solutions, like LexisNexis CaseMap, that could become a learning-by-using teachable moment students may carry with them into their careers.

The crucial element here is that the recommended curriculum modifications and instructional strategies would require that the traditional model of legal education make room for legal research and writing well beyond 1L LRW and ALR courses. While the authors frame this in the context of Gen-Ys because they are today's law school students, their recommendations are not all that era-sensitive. Similar recommendations have been proposed but ignored by the legal academy. However, in the past, legal employers were willing to hire recent law grads, knowing well, that they would be paying them during the first couple of years for what would amount to be a "post-grad" education. Even BigLaw is pushing back on that now.

While there are many other areas in the law school curriculum that need to be reformed to produce "practice ready" graduates, in the context of integrating legal research and writing skills, the authors offer recommendations that move beyond the "exposure" component. Assuming law profs are willing -- remember the whole "academic freedom" argument in the context of teaching -- the authors recommend that skilled legal writing and research instructors need to be incorporated in the teaching model across the curriculum. I think this would take something more than the assignment of law profs' teaching or research assistants since they don't qualify as being "skilled." Staff already on hand, particularly academic law librarians, would be a logic choice. This does assume that staff are skilled in practitioner research using the tools practitioners use (eg beyond WEXIS) and have sufficient expertise in both the topical resources and the subject matter of the course. The days of academic law librarian-scholars in a particular area, even in foreign and international law, are waning at all but the largest academic law libraries. Understanding of a topic area need not be mean having substantial expertise to be on the same page as the law prof teaching a doctrinal course in, for example, CivPro, but it be something more than an basic understanding of black letter law on the topic.

A critical component of the authors recommendation in this regard, one not explicitly stated but certainly essential, is the replication of what law school students may find the real world "reference interview" and its follow-up, actually helping perform the research side-by-side the attorney or student. Assignments are given, the young associate or in this case the law school student runs to the library for help. It certainly would help in the context of legal education if the team approach suggested by the authors sent the law school students running to a legal research expert who was closely working with the law prof from the day both were crafting the course through completation of the course.

There would be no small amount of additional work load added to academic law librarians. Not only must they have some measure of subject expertise to assist law students based on subject matter legal expertise but also some measure of expertise in using pratitioner-used legal resources. Most law school libraries do not have the depth of subject expertise on staff. But they could. If the authors recommendation for reforming the curriculum by executing instructional strategies to produce law grads who think and practice like lawyers were to be implemented, it would require hiring additional law librarians. (Good thing #1) It would also require that academic law librarians be recognized an integral member of the the teaching faculty, not just nominal members. While legal skills profs, primarily LRW profs teaching 1L courses and clinical profs are striving to gain pecking order recognition to influence the decision-making process at their local law schools, imagine the input academic law librarians could have if they were also voting on curricular matters in faculty meeting (Good thing #2).

And what is "Good thing #3'?  Private (and public) sector law librarians would spend less time teaching practice-not-ready recent law school grads how to conduct real world research and more time assisting them in performing legal research for the issue at hand. A matter like this one extends well beyond academic law librarianship... . [JH]

Legal Research Fantasies: Finding the Answer and Saving Time

On 3 Geeks, Greg Lambert takes exception to what Rich King, TR CTO, says at the 42 second mark in the below video. King states

One page from one jurisdiction may be exactly what that attorney is looking for that allows them to win the case. And, if they can't find it, you haven't done your job.

Greg was not criticizing TR Legal's feel-good video per se. It is a more general critique about legal research expections. He writes in There's One Page, In One Jurisdiction, That Will Win Your Case – Go Find It or Lose!:

the thing that does worry me about fact-based inquiries and processes is the idea that if you don't find that "one page" in that "one jurisdiction" then the idea is that "you haven't done your job." That just seems like a very high-bar to hold up for the legal profession. If we are teaching our law students, Summer and Fall Associates, our Paralegals, and our Legal Research Professionals the idea that they must find that one case in that one jurisdiction, then we are setting them up for failure.

The Fantasy of Finding "The Answer." I certainly agree. It is pretty damn rare that one will find something completely on point in one document, be it primary or secondary literature in law. Legal research is a cumlative process of knitting together authorities to craft a legal argument. This one "page-one jurisdiction-one answer" notion makes me think the TR Legal's feel-good video presents a fantasy that legal research and analysis is akin to hard science research in physics, chemistry and biology.  Of course, it comes no where close to that. Perhaps TR CTO Rich King needs to read E. Scott Fruehwald's Legal Skills Prof Blog post, Five Methods of Legal Reasoning. The thought processes identified are part of the research process. 

The Fantasy of Advertised Time-Savings. There is no getting around the reality that legal research is a time-consuming process even in the context of online legal search. "You can spend less time researching and more time lawyering" if you believe WLN's marketing pablum. A lot less time researching if you believe TR Legal's advertized survey findings. Check out the huge percentage difference. 

But take note, TR Legal's in-house produced survey finding is refering to single digit minutes saved for very short (as in 10 minute or so) research sessions. When was the last time, any professional, lawyer or librarian, spend so little time performing legal research in the real world?

Was TR Legal's surveyed research issues intentionally designed to find that "one page from one jurisdiction" for the always hoped for but almost never found citation to something that hit all four corners of the square needed to produce a definitive answer to the research question by citing just one source? What utter nonsense. [JH]