Wednesday, February 13, 2013

Jim Moliterno Answers Questions on W&L's 3L Program; Supplies Additional Data on W&L

My previous post on Washington & Lee's 3L Program stirred a lot of interest and commentary, including some disbeleiving critics.  Fortunately, Professor Jim Moliterno agreed to write a reply essay, below, that completes the cycle. [Bill Henderson]

  MoliternoJim Moliterno Replies  [This is a long reply, so a PDF version online here]

A number of comments to Bill’s January 28 post and posts regarding it on other blogs cause me to enter this conversation.

Are students really coming to W&L because of the new curriculum?  Yes, to a significant extent. How do we know?  Because the entering students say so. As do many law schools, we administer a questionnaire to our enrolling students. Among the questions asked is the obvious one: why are you here?  

In the most recent such survey the students were asked to rank the strengths of the law school. Here are the top ten, in order, according to the entering students:

  1. Third Year Curriculum
  2. Ranking / Prestige
  3. Quality of Life
  4. National Reputation
  5. Job Placement
  6. General Cirriculum
  7. Clinical Program
  8. Faculty
  9. Financial Aid Award
  10. Size of Lexington

The curriculum reform was first.  Financial aid awards were 9th, just ahead of the “size of Lexington.” The data does not support the unsubstantiated claims of some bloggers that students are choosing W&L because of the generosity of financial aid awards. 

The curriculum reform has steadily moved higher on the “strength” rankings given by enrolled students since 2009. The 2011 and 2012 surveys are nearly identical, and the written comments of students about their reasons for coming to W&L (none reprinted here), are more striking than the numbers themselves.

I don’t know of any better data on this proposition but the statements of those whose reasons are under study. If that data is unsatisfying to some, then they will continue to be unsatisfied.

Are there other reasons students come to W&L? Of course. W&L has a highly productive, highly visible  faculty engaged in scholarship and projects at the highest levels. Some students undoubtedly value W&L’s faculty prowess.  W&L is highly ranked. Some students undoubtedly are affected by a top 25 ranking. It has an excellent reputation as a small, closely-knit academic community.  Some students select for the sense of community and size. No reason will ever be the only reason for prospective students to choose a law school.  Changes made by law schools will affect student choices for or against a particular law school. The W&L curriculum reform is positively affecting a significant number of students’ calculus about choosing W&L.   

And some do come because of the financial aid package they were offered.  But the financial aid reason is unlikely to explain the increase in applications since 2008. Some students, the recipients of aid, undoubtedly come in part because of the aid.  That is no different than the students who choose [insert name of any school] because of the financial aid they were awarded.   In 2012, about the same number of offers of admission were made as in previous years, but instead of the usual 130 or 135 admittees choosing to attend, more than 260 made deposits. Some were asked to defer their attendance until 2013 and once the dust settled we had a class of 187 instead of the usual 130 to 135. This same class entering in 2012 listed the curriculum reform first and financial aid ninth as strengths of the law school.

What else was happening in 2008 and 09 when the applications increased by nearly 33% per year? 

In 2009 and 10, while W&L applications were on the rise, the US News ranking fell from 25-34 (while its reputation rank among academics stayed steady).   It has now recovered to 24.  If anything, that should have led to a drop in applications during 2008-2011 rather than the sharp increases that actually occurred.

Can we exclude all other possible explanations than those previously mentioned?  Of course not.  It could be that being in a small, beautiful mountain town is all the rage among young adults and 33% more students want that now than wanted it in 2007. I know of no data to prove or disprove that proposition, so it remains one that could be true. The reality is that the students who have come in recent years rate the curriculum reform among the top reasons (often the most important reason) for their attendance at W&L. That matters.

There is empirical evidence that the W&L curriculum reform is engaging students more than in the traditional “no plan” third year curriculum. Is it perfect evidence? Of course not.  Is it definitive evidence that has no flaw? Of course not. Is anything ever supported by perfect, definite evidence that has no flaw? Not to my knowledge.  We make all of our most important decisions in life based on the best available evidence. As long as the evidence is empirically sound and statistically significant, it is worthy of respect. The evidence of W&L 3L engagement increases is sound and statistically significant and marks a path toward further research and verification.

One commenter suggested that the data is suspect because the peer schools have not been identified. Their data belongs to them, not W&L.  LSSSE does not make specific school data available to other schools.  So W&L has only a composite score for those peer schools. And it would be unseemly for W&L to reveal the specific schools.  I will not do so here. But to be sure, W&L asked LSSSE to calculate the data from a list of schools because they are the schools with whom W&L competes for students and competes in the rankings. It would not have served W&L’s research interests to learn how it compares with a list of schools that it does not compete with in the marketplace. No one at W&L has the data for any specific school. 

Nonetheless, do not be mistaken, the schools with whom W&L is compared in LSSSE data are the schools anyone would expect them to be: schools that by their geography, rank and quality compete with W&L in the relevant markets for students and placement.

One observation: in the legal profession and legal education in particular, the status quo never seems to need empirical justification.  Only change is suspect and wrong until proven definitively to be otherwise. Is there any empirical evidence that the status quo third year is the best possible third year except that it has been done that way for a long time? None that I know of. The old adage, “if it ain’t broke don’t fix it” does not apply here.  The third year of legal education is “broke”.

Amid calls for its abandonment by some, dating back at least to the early 1970s report by Paul Carrington, the third year is widely acknowledged to be of the least value among the three years. (See below on W&L’s largely unchanged  approach to years 1 and 2.) The Roman Legions (and more than a few other military powers) have found out that the mere fact that something has been successfully done before is not sufficient evidence that it will prevail in the present or future. Arguing in favor of the status quo based on no empirical evidence, . . . based only on instinct and the argument that it is the way things are currently done, is an approach doomed to failure.  Just ask Kodak. (And see my forthcoming book: “The American Legal Profession In Crisis,” Oxford, March 2013.)

How about the claim that “[W&L’s LSAT has] gone down every year since [the new curriculum was announced], while its GPA rank has, after a plunge, more or less returned to where it was.” The blogger made that claim, once again without any data, let alone empirically credible data.  Actually the W&L median LSAT was steady at 166 from 2005-2010, dropped 2 points to 164 in 2011 and stayed at 164 for 2012. It has not “gone down every year since [the new curriculum was announced in 2008].” Meanwhile, the GPA of entering classes, which was in the 3.5 and 3.4 range in 2008-2010, has gone up to the 3.6 range (3.65 and 3.62) in 2011 and 2012. The two modest changes in LSAT and GPA have essentially off-set one another in US News points. Hardly the reason for pause suggested by the blogger.

It seems that as long as someone is arguing against change, no rules apply to the arguments’ underpinnings.

Here is what the empirical evidence from the LSSSE surveys shows and what it does not show: students are more engaged in their work and their work includes more writing, more collaboration and more problem solving. Here are a few charts even more striking than those Bill used in his post. Together they say that significantly more than their peers or their predecessors at W&L, current third year students are working more, writing more, collaborating more, applying law to real world problems more, and preparing for class more often. Overall, they describe a harder-working, more engaged student body. And they are working harder at acquire the skills that matter to success as a lawyer.

Wlone

Wltwo
Wlthree
Wlfour
Wlfive
Wlsix
Wlseven
Wleight
Wlnine
Together with the entering student survey numbers, here is what the application increases at W&L show and do not show: prospective students are choosing to apply and then enroll in a law school with a third year curriculum that engages them more in the work of lawyers. The data do not show what employers think or do not think about the curriculum reform.

It is too early for employment data. One full class has graduated from the new curriculum, in May 2012, and that in a time of such incredibly reduced employment of new lawyers.  No innovation, no matter how much it might improve graduates’ abilities to perform, will change employment data until employers become convinced. That time is not yet come. It is highly unlikely that employers will break out of established patterns in times like the present when they are hiring a scant few new lawyers.

We live in a world of enormous pedigree influence. So no matter how successful our curriculum is for students, I do not expect that we will make employment gains vis a vis the top five or ten schools in the rankings. Instead, I do expect that over a five to ten year period, we will make gains vis a vis schools that are in our peer group, defined roughly as everyone between 20 and 40. Likewise, if other schools institute well-designed and substantively effective  experiential education curriculum reforms, I would not expect that they will suddenly make dramatic gains against much more highly ranked schools.  For example, if a school ranked number 150 institutes excellent reforms, I would not expect them to make gains versus schools in the top 50. But if they do the reform well AND if they make it known AND if they preserve what is already effective in the rest of their curriculum AND if they have an effective career services shop, they will make gains against the schools within their peer group.

Asking for data to show that employers have broken down the doors of a law school to hire graduates after one year is actually a bit silly. No one who seriously respects data and the market would expect such data to exist. That is a five to ten year project and is limited even then by the pedigree factor mentioned in this paragraph.

Curriculum merits. The curriculum relies on clinics, externships and practicum courses.  Readers of this blog know about the first two and less about the third. The new “practicum” courses are not skills courses.  Instead, they are courses about the lawyers’ work in various practice settings. Rather than rely exclusively on courses in trial ad or negotiation or interviewing, the W&L curricular reform relies primarily on courses like The Lawyer for Failing Businesses and Mergers and Acquisitions Practicum and Corporate Counsel and Poverty Law Litigation and The Litigation Department Lawyer. In these courses, students learn the relevant substantive law, but they learn it the way lawyers do rather than the way students do.  They learn law to solve a client’s problem. This alone is an activity that adjusts students’ mental pathways from student to lawyer.

At W&L, students are purely students in years 1 and 2. They read cases from casebooks and attend class and take final exams. None of that focus of lost. But, they spend their third year learning law as lawyers do, with a client’s service at the center rather than an exam at the center. For example, in a course called The Lawyer For Failing Businesses, the students are placed in the role of a lawyer representing a failing business.  They counsel the client about bankruptcy options. They draft the documents necessary to start a bankruptcy proceeding. They negotiate with creditors. They draft financing documents. They deal with ancillary litigation. In doing so, they learn bankruptcy law, but not for the mere transient purpose of passing a three hour exam. They learn it in the context of its immediate use for a client.

Is it quite as wide as a traditional course in Bankruptcy? No. But it does convey to students the theory of bankruptcy law and its use. In practice, lawyers do not answer clients’ questions by saying, “Yes, I learned about that on the Tuesday of the third week of my Torts course.” They use the essence of a topic to research and discover the best answer to the client’s very specific set of circumstances. 

Nothing of consequence is lost by missing a topic in a course.  Studies show that students retain about 10% of what we tell them. Coverage-need is passé.  It is what faculty members argued (sometimes disingenuously) when their Property course was being reduced from 6 to 4 credits. It is old news. No one can claim that students can be exposed to every law topic that might be beneficial to them. Students need the essence of a topic for their use in practice, not the detail, likely forgotten in any event. We unduly glorify ourselves to think that students remember everything we say in class or assign them to read. They retain the core. We hope.

Many of us have had students say, “I never learned anything about Contracts [insert whatever course you like] until I used it in my clinic [my summer job, my externship, my practice].” Of course students are wrong to say this. They acquired cognitive knowledge during their course; they realized the gain when they used the knowledge. But their comments do have meaning: to fully grasp and understand, students must not only acquire knowledge but they must also use it.

First and second year, a three year curriculum not a third year curriculum. The reformed third year curriculum follows from the first and second years. It does not stand alone and is no rejection of the good that exists in traditional legal education.

The first year has long served a valuable purpose.  In it, students’ thinking is transformed to that of a legal analyst and the skill-peak of academics is on display most prominently. The comparative advantage in teaching by academics is most pronounced in the first year. We shine and are perhaps irreplaceable here.  Students must have the critical thinking skills that we provide to them through our first year teaching techniques.  W&L instruction in the first year is largely unchanged from the past. We have added courses in international law and administrative law to the usual stable of 1L courses, and we now teach professional responsibility to second semester, first year students. (Some schools have done the same, but we are in the minority in including these three courses in the 1L year.)  But the mission of the first year is the same as it ever was. It succeeds.

The second year is also largely unchanged from the past.  Our second year students predominantly enroll in the core subjects that are not covered in the first year: evidence, corporations, basic tax, constitutional law (a first year subject at many schools), criminal procedure, trusts and estates (though to a lesser extent than in the past), etc. Many students engage in the law journal activities, the moot court competitions and the newer negotiation, mediation, client counseling and transactional skills competitions. 

I won’t repeat here everything about how the third year works, but it requires a full credit load (24 credits) of experiential education, including clinics, externships, immersion courses (litigation and transactional), practicum courses (elaborate simulations of practice settings), and a service requirement.  But within the student’s third year there is space for a traditional course in each semester if the student chooses.  So the student who lands a clerkship and has not yet taken Fed Courts can do so in the third year, for example, without being in an overload.

Ours remains a three year curriculum, with the first two years attending well to the traditional missions of the law school experience. The third year is being made more valuable; the first two are not being slighted or cast aside.

Bar exam? So far we have not seen statistically significant bar exam results.  In one year, the pass rate was up and the next year down, but neither to statistically significant levels.  We are paying attention to this possible issue and so far see no cause for concern.  We will continue to monitor. I would say that the current, traditional bar exam is itself an impediment to legal education reform.  With some states testing 28 subjects and students typically taking fewer than that number in the entire three years, room for courses that include among their teaching goals problem solving, team work, writing, business sense, etc., are a luxury that insecure students and law schools cannot afford. All schools have some students who are bar-exam-at-risk.  Some schools have a majority of such students. In general, the more insecure the students and law school, the less able they are to reform their curriculum to reflect the actual needs of students to succeed as lawyers. The bar exam has always been touted as a “gatekeeper.”  But as the subjects tested have proliferated and the practice has become more sophisticated and less reliant on rote memorization of knowledge, the gatekeeper bears less and less relationship to what is on the other side of the gate. A macramé test would also keep the gate secure, but it would say nothing about the qualities of the passing takers to excel on the other side of the gate.  The traditional bar exam becomes less and less relevant to the practice of law every year.

There is nothing anti-academic about studying the work of lawyers. To say so betrays a false elitism more likely borne of insecurity than of truth. Many legal academics could not do what lawyers do: solve real clients’ problems that involve extra-legal attributes. The work of lawyers is sophisticated. It partakes of some of the rigor of law school teaching and scholarship, but it also relies on sophisticated problem-solving  and  a multiplicity of other  talents.  Some who claim that lawyer work is mundane and uninteresting fail to understand the nature of that work in the first instance. Some who make the claim seek cover from their own lack of capacity to do such work. Describing it as uninteresting allows the speaker to hide his or her inadequacy. The study of effective lawyers is a sophisticated inquiry. The work of excellent lawyers is not mundane. And the mundane tasks undertaken by beginning lawyers in the past are becoming commoditized and outsourced.

The current system of legal education fails to account for a simple truth: the skill-set of legal academics is not a perfect overlap with that of the role to which the vast majority of our students aspire. The 19th Century redesign of legal education was based on the premise that law school’s primary mission was not to create lawyers but rather to create law professors. (This conclusion is documented in the correspondence of the main contemporary actors involved in the reform.)  Many adjustments have been made over the subsequent century and a quarter, but the remnants of those 19th Century decisions persist today.

Generally speaking, legal academics are excellent law analyzers and theorists. We are critical thinkers and precise analysts of law and its theoretical underpinnings. Students need this same talent and we are best at conveying it, especially in the traditional first-year courses and teaching modes. But to be successful lawyers, students need more than that foundational thinking skill.  They need to learn how to problem solve when some of the factors are not strictly law-related; they need to learn to work in teams and to manage projects; they need to acquire a measure of business sense whether they serve as business counsel or manage their own law shop; they need to learn how to manage risk and assess the risk adversity level of clients; they need to communicate the law and its constraints to non-lawyers; they need to acquire bedside manner.  In short, there is a multitude of talents and skills and attributes that students need to acquire that are not the skill-domain of academics (with many academics being an exception to this rule).

One blogger said that the 3L curriculum at W&L “focuses on practical lawyer skills.” This sort of statement sells the new curriculum far short of its reality. It actually focuses on the attributes, skills and mental habits of successful lawyers, all while providing students with substantive law and theoretical learning as well. A broad view of lawyer skills would include the mental development fostered in the first year as well.  It is time to stop pretending that legal analysis is not a practical lawyer skill.  It is—and it is both critical and fundamental—but it is not the only skill/attribute/talent that lawyers need to be successful.

Successful lawyers can and should be our partners in providing this education.  They know better than we do about many of these skills and attributes.  Indeed, some of this learning will inevitably continue to take place after law school.  But the economic realities of today’s legal market dictate that less teaching is being done after law school. Law firms teach less than they once did and more graduates are having to find their own way as solos. Demands from all quarters are that legal education provide at least a head start on the development of students in these realms. Prospective students, the practicing branch, and paying clients are all making such demands, and we ignore them at our peril. Some law schools that ignore this market demand will fail.

Legal education and the legal profession are at a crossroads. Applications are strikingly down for a reason. Schools can stand pat if they choose, and some have the market power to do so for a significant time after change would be prudent and effective. All others do so at their peril. Change is not good merely for change’s sake. But it is not prudent to stay the same when the world has changed. The practicing branch has changed; client needs and demands have changed; the society that the legal profession claims to serve has changed. Only legal education (and the organized bar) now remain stubbornly tied to anachronistic ways. The legal profession itself and legal education in particular, live as if they had eyes on the back of their head, but none on their face. Only what is past seems to be valued-- Even when what has past has no empirical basis and the conditions in which it exists have dramatically changed.

Bill Henderson based his opinion on good data.  Not perfect data but good data. Data sufficient to guide decisions in most realms of life and work.  The responses to Bill’s post to date have been based on virtualy no data, but rather on surmise and rumor and vague impressions of W&L’s. My fondest hope would be that many thoughtful, careful innovators pursue their projects and produce as much data as the legal education project allows.  This is not a one-size-fits-all enterprise. But the W&L reform is one that preserves the best of a traditional legal education while enhancing what can be improved about traditional legal education. It does not deny the value of academic work. It does not deny the value of traditional teaching methods. It adds to them third year experiences that the best data available shows are having positive effects. 

[posted by Jim Moliterno]

https://lawprofessors.typepad.com/legalwhiteboard/2013/02/jim-moliterno-answers-questions-on-wls-3l-program-supplies-additional-data-on-wl.html

Blog posts worth reading, Current events, Data on legal education, Innovations in law, Innovations in legal education, New and Noteworthy, Scholarship on legal education, Structural change | Permalink

Comments

You note "One full class has graduated from the new curriculum, in May 2012," and that the bar exam has not yielded statistically significant results. But, the first-time Virginia bar passage rate in 2011 was 96%, but in 2012 was 73%. Could you elaborate as to why this was not statistically significant? Genuinely curious.

Posted by: anon | Feb 13, 2013 9:20:00 PM

I would like to suggest another approach to producing practice ready attorneys. A six year program comprised of two years of traditional undergraduate study followed by four years of legal study. After two years of law school a bachelors degree would be granted qualifying the graduate to work as a paralegal. After completing the full four year program a graduate should be fully able to practice law. After the first year of law school the ciriculum would be a mix of course work and supervised practice. This approach shortens the time to become an attorney, provides a career for students who do not want complete the full program and may help meet the unmet need for legal services.

Posted by: Bernie Schaeffer | Feb 14, 2013 1:07:26 PM

Anon, the collective out-of-state bar pass rate in 2012 was high, so the overall rate was within our normal range. A significant portion of our graduates take the exam in other states, so in some years, the in-state rate is misleadingly high or low. In 2010 and 2011, when the curriculum was optional, about 70% of students opted in. We did an analysis of the bar pass rate of those who opted in and those who opted out and there was no statistically significant difference between the two goups.

Posted by: Jim Moliterno | Feb 14, 2013 7:34:20 PM

The most powerful empirical data point in this post (and the last) appears in the comments and is the 23% drop (!) in bar passage rate in the first full year of the program in the primary bar jurisdiction. There may be some explanation, although your curt response to anon's comment would not satisfy an astute lawyer, and I imagine this has to be a subject of intense discussion at W&L. It sounds like part of the response is that the bar exam is "less and less relevant to practice every year" and that might be true, but passing the bar exam is still required for the practice of law.

Posted by: late-to-thread | Feb 16, 2013 7:17:32 AM

Jim,

You said, "Nothing of consequence is lost by missing a topic in a course. Studies show that students retain about 10% of what we tell them." Education scholars have demonstrated that students retain more knowledge when they apply it. So, students should retain more in an experiential course than they do in a class that lacks this element. In other words, students' retention of knowledge should be significantly higher in your approach, which is another advantage of your approach.

Posted by: Scott Fruehwald | Feb 16, 2013 2:48:43 PM

Quite true, Scott. The material covered in a practicum course is undoubtedly better-retained than the material taught in more traditional formats.

Posted by: Jim Moliterno | Feb 21, 2013 9:40:47 AM

Post a comment