Tuesday, February 4, 2014
I think the answer is yes. But, unfortunately, in virtually all of the debate surrounding legal education, there is a tremendous lack of clarity and precision about how we assess improvements in quality. And equally relevant, if a gain is real, was it worth the cost?
The purpose of this essay is to chip away at this serious conceptual gap. Until this gap is filled, experiential education will fall significantly short of its potential.
Is Experiential Legal Education Better? And if so, at What Cost?
Many legal educators believe that if we had more clinics, externships, and skills courses in law school, legal education would be better. Why? Because this more diversified curriculum would become more "experiential."
Inside the legal education echo chamber, we often accept this claim as self-evident. The logic runs something like this. A competent lawyer needs domain knowledge + practical skills + a fiduciary disposition (i.e., the lawyer’s needs are subservient to the needs of clients and the rule of law). Since practical skills—and some would argue, a fiduciary disposition—cannot be effectively acquired through traditional Socratic or lecture teaching methods, the ostensible logic is that schools become better by embracing the "learning-by-doing" experiential approach.
That may be true. I would bet on it. But the per-unit cost of legal education is also probably going up as well. So, have we really created a viable and sustainable long-term improvement to legal education?
In my mind, the questions we should be asking instead are the following: (1) Among experiential teaching methods, which ones are the most effective at accelerating professional development? And (2) among these options, how much does each cost to operate? Quality and cost must be assessed simultaneously. After they are evaluated, then we will be able to make choices and tradeoffs.
Let's start with quality, which I define as moving lawyers toward their peak effectiveness potential as rapidly and cost-effectively as possible. This is an education design problem, as we are trying to find the right combination of education (building domain knowledge) and experience (acquiring and honing skills through practice). There is also likely to be an optimal way to sequence the various educational and experiential steps.
Creating Compelling Evidence of Educational Quality
We legal educators have many ideas on how to improve educational quality, but we make no real progress if employers and students remain unconvinced. Can it be shown that because of a specific type of experiential curriculum at School X, its graduates are, during the first few years of practice, more capable lawyers than graduates of School Y?
[Side bar: If you are skeptical of this market test, it is worth noting that it was the preferences of law firm employers who gave rise to the existing national law school hierarchy. It happened about 100 years ago when a handful of law schools adopted the case method, required undergraduate education as a prerequisite to admission, and hired scholars as teachers. As a general matter, this was a far better education than a practitioner reading lecture notes at the local YMCA. See William Henderson, "Successful Lawyer Skills and Behaviors," in Essential Qualities of the Professional Lawyer ch 5 (P. Haskins ed., 2013).]
If a law school can produce, on balance, a better caliber of graduates than its competitors, then we are getting somewhere. As this information diffuses, employers (who want lawyers who make their lives easier) will preference law schools with the better graduates, and law students (who want more and better career options) will follow suit. Until we have this level of conceptual and empirical clarity, we might as well be debating art or literature.
If students and employers are responding to particular curricula, it is reasonable to assume they are responding to perceived value (i.e., quality as a function of price). I believe there are three steps needed to create a legal education curriculum that truly moves the market.
1. Clarity on Goals. We need to understand the knowledge, skills, and behaviors that are highly prized by legal and non-legal employers. Truth be told, this is tacit knowledge in most workplaces. It is hard intellectual work to translate tacit knowledge into something explicit that can be communicated and taught. But we are educators -- that is our job! If we think employers are missing something essential, we can add in additional factors. That's our job, too.
2. Designing and Building the Program. Working backwards from our goals, let's design and build curricula that will, overall, accelerate development toward those goals. This is harder and more rigorous than lesson planning from a casebook.
3. Communicating Value to the Market. If our program is indeed better, employers and students need to know it. This also requires a crisp, accurate message and a receptive audience. This requires planning and effort. That said, if our program truly is producing more effective lawyers, it logically follows that our graduates (i.e., the more effective lawyers) will be the most effective way to communicate that message.
Regarding point #3, in simple, practical terms, how would this work?
During the 1L year, we show our law students the roadmap we have developed (step #2) and spend the next two years filling in the knowledge, skills, and behaviors needed to achieve their career goals. This professional development process would be documented through a portfolio of work. This would enable students to communicate specific examples of initiative, collaborative learning, problem-solving, or a fiduciary disposition, etc., developed during law school. Students would also know their weaknesses, and have a clear plan for their future professional development. In a word, they'd stand out from other law graduates because, as a group, they would be much more intentional and self-directed (i.e., they'd know where they are going and how to get there).
With such a curriculum in place, our law school would collaborate with employers assess the performance of our graduates. By implication, the reference point for assessing quality would be graduates from other law schools. When our graduates fare better, future graduates will be more heavily recruited. Why? Because when an employer hires from our school, they would be more likely to get a lawyer who helps peers and clients while adding immediate enterprise value.
I suspect that many of my legal academic colleagues would argue the best law schools are not trade schools -- I 100% agree. But I am not talking about a trade school model. Rather, a world-class law school creates skilled problem-solvers who combine theory with practice and a fiduciary disposition. Graduates of a world-class law school would be reliably smart, competent, and trustworthy. This is a very difficult endeavor. It takes time, planning, collaboration, creativity and hard work. But the benefits are personal, organizational, and societal.
At a practical level, I think few law schools have targeted this goal with a full, unbridled institutional commitment. But the opportunity exists.
When I got tenure in 2009, I decided that I was going to spend the next several years doing applied research. I am a fact guy. Rather than argue that something is, or is not, better, I prefer to spend my time and effort gathering evidence and following the data. I am also a practical guy. The world is headed in this direction, thanks to the ubiquity of data in the digital age. And, on balance, that is a good thing because it has the potential to reduce conflict.
I have pursued applied work in two ways: (1) building stuff (curricula, selection systems, lawyer development tools, datasets for making strategic decisions, etc.) and assessing how well it works, and (2) observing and measuring the work of others.
A Law School Curriculum Worth Measuring
A couple of years ago, a really unique applied research opportunity fell onto my lap. I had a series of lengthy discussions on the future of legal education with Emily Spieler, who was then serving as dean of Northeastern University School of Law in Boston, a position she held for over a decade. One of the raps on legal education is that it is more alike than it is different. In fact, this very point was just made by the ABA Taskforce on Legal Education. See ABA Task Force On The Future Of Legal Education, Report And Recommendations (Jan. 2014) at 2.
Emily, in contrast, said her school was unique -- that the curriculum better prepared students for practice and enabled them to make better career planning decisions. Also, Emily stated that Northeastern students were more sensitized to the needs of clients and the privilege and burden of being a lawyer--specifically, that Northeastern grads become aware, before graduation, that their own lack of competency and diligence has real-world consequences for real-world people. And that reality weighed on students' minds.
Tall claims. But if Northeastern coulddeliver those outcomes more effectively than the traditional unstructured law school curriculum, I wanted to know about it.
On a purely structural level, Northeastern Law is definitely unique. Most law schools are organized on either quarters (University of Chicago, my alma mater) or semesters (Indiana University, where I teach). Northeastern, however, has both. The 1L year curriculum at Northeastern is the traditional two semester model. But after that, the school flips to quarters -- one quarter in law school, and one quarter in a cooperative placement with a legal employer, such as a judge, prosecutor’s office, a law firm, a corporate legal department, or a public interest organization.
This classroom/coop sequence occurs four times over eight quarters. Because the cooperative placement is not viewed as part of Northeastern's ABA-required course work -- all the contact hours are packed into two 1L semesters and four 2L/3L quarters -- students can be paid during cooperative placements. And in any given semester, roughly 30 to 40% are getting paid.
This system has been up and running for 45 years--over 5,000 students have become lawyers through this program. What an amazing research opportunity!
Now imagine the faculty meeting where the law professors get together to discuss and deliberate over whether to adopt the Northeastern model. At Northeastern, "summer" means summer quarter, not summer vacation.
How did this unique curricular structure come into being? That is quite an interesting story. During the 1950s, the law school at Northeastern was shuttered. Yet, reflecting the zeitgeist of the times, a group of Northeastern law alumni and young lawyers who were skeptical of their own legal education (at elite national law schools) petitioned Northeastern to reopen the law school and feature a more progressive, forward-looking curriculum. The university administration agreed to reopen the law school on the condition that the school adopt the signature cooperative education model. So this crucial decision was essentially made at the birth of the law school over four decades ago. Once up and running, Northeastern Law implemented other innovations, such as the narrative grading policy--i.e., no letter grades and no GPA. This was done in order to mitigate competition and encourage a focus on collaboration and skills development.
The Outcomes Assessment Project
Back in 2011, my conversations with Emily Spieler eventually led me to make a two-day pilgrimage to Boston to talk with Northeastern Law faculty, students, administrators, and coop employers. Suffice it to say, I was surprised by what I witnessed --a truly differentiated legal education with a substantial alumni/ae base spanning 45 years.
That pilgrimage eventually led to my involvement in Northeastern Law's Outcomes Assessment Project (OAP), which is something akin to The After the JD Project, but limited in scope to Northeastern -- although Northeastern will provide all of the project tools and templates to other law schools interested in studying their own alumni. From the outset, the OAP has been set up to scale to other law schools.
There are lots of tricky methodological issues with Northeastern. For example,
- It has a longstanding public interest tradition; Northeastern Law is overrepresented in government service, public interest, and non-profit sectors (including a sizeable contingent of law professors and legal clinicians). See Research Bulletin No 1.
- Its student body was over 50% female almost from the outset, nearly 20 years before legal education as a whole.
- Because of its progressive roots, GLBT law students have long been drawn to Northeastern Law -- again, nearly two decades before it was deemed safe to be out.
Because of this distinctive profile, we have to worry that any differences in graduates are primarily due to a selection effect (who applied and enrolled) versus a treatment effect (they got a different type of education). That said, the admissions data show that Northeastern Law students are, like other law students, strongly influenced by the US News rankings. If a student gets admitted to Northeastern Law and BC, BU, or Harvard Law, Northeastern seldom wins.
Over the coming months, I am going to use OAP data to attempt to develop some analytical and empirical clarity to some of the questions surrounding experiential education. Preliminary data from our Research Bulletin No 3 suggest that the coop program does remarkably well in developing the three apprenticeships identified by the Carnegie Report. More on that later.
Print version of this essay at JD Supra.
Wednesday, November 27, 2013
If you have the courage and curiosity to understand the breadth and depth of the changes taking shape in the legal market, then I would encourage you to use some of your Thanksgiving break to read "Recalculate the Future of Law," which is Insight Lab's interview with MSU Law Professor Dan Katz.
It is all-too-easy to believe that innovation occurs in the wake of a great idea, but that is not quite right. Innovation is also about timing and understanding how human institutions are held together and change and evolve. If the innovator has the benefit of timing and understands how human institutions actually work, an effective adoption strategy is possible.
Fortunately, for Dan Katz, all of these factors appear to be in alignment. Katz is acutely aware of his timing and the myriad of factors that enable innovation to take hold. He is also young (35 years old) and has the courage to place very large bets -- the largest bet being that he is not waiting to get tenure before starting his life's work. He is doing it now in his third year of teaching.
But to mind, there is some additional secret sauce. What makes Katz so disruptive is his 100% personal commitment to the growth and potential of his students. He is awaking the sleeping giant -- hope and a sense of purpose for young people. Specificially law students. If you are in his ReInvent Law Labratory, you see a different legal landscape with a whole lot more options. But to tap into that hope, Dan makes you do the work. You have to challenge yourself. And you have to shed the bullshit phobia over basic math. He is building a community of interest that has the potential to morph into a movement driven by young lawyers and law graduates. For more on Katz's unusual bio, see "This is just an education design problem," LWB, Sept 23, 2013.
The interviewer over at Insight Labs got pretty close to the full, uneditted Dan. If you want to learn about the underinvestment problem that is undermining BigLaw, the crucial role of start-ups in the emerging field of legal R&D, how the next generation of law students can do well and do good, or the real hazards of the $1 Million JD debate, give it a full read.
November 27, 2013 in Blog posts worth reading, Current events, Fun and Learning in the classroom, Innovations in law, Innovations in legal education, New and Noteworthy, Structural change | Permalink | Comments (2)
Saturday, November 23, 2013
Is it important to help law students understand the disruptions that are now occurring in the legal industry? Well, let me ask a more fundamental question. How can a law professor efficiently obtain better information on these complex and diffuse changes? None of us legal academics are experts in this area, and that's a problem in and of itself.
In the process of struggling with these questions, I decided to carve out 15% of the grade in my Corporations class for team-based profiles of NewLaw companies. Here is how I described the conundrum in my syllabus:
The legal industry is changing in dramatic ways, including the creation of new legal businesses that rely upon technology and process design to solve legal problems that have traditionally been handled by lawyers. These businesses are often financed and managed by nonlawyers, which some of you may find surprising. ...
Remarkably, very few practicing lawyers grasp the type of industry context described above ... Yet, the influx of financiers and technologists is likely going to have a dramatic effect on your future legal careers. These changes are extremely foreign to the substance of traditional legal education – we (the legal professoriate) just don’t understand the breadth and depth of the changes that are now occurring. Rather than sweep this uncomfortable fact under the rug, let’s do what great lawyers do with their clients. Let’s learn about the business and the industry so that we understand the context. Armed with this information, we can make better decisions with regard to our own careers.
Two months ago, I circulated the full assignment to the class, divided the class into teams, and gave students two weeks to select a company. The only restrictions were no duplicates, so first-come first-serve, and the company had to be a non-law firm business operating, partially or entirely, in the legal industry. (BTW, JB Ruhl's Law Practice 2050 course at Vanderbilt Law tackles this topic head-on.)
Students made their presentations this past Monday evening (Nov. 18) in Indiana Law's Moot Courtroom. It was a marathon session that ran nearly four hours. Because of the novel content, several practicing lawyers showed up to see the presentations. The following companies where profiled:
- AdvanceLaw. Privately held company that operates a closed community of legal departments who share information on law firms and individual lawyers in order obtain better quality at a lower cost. Discussed on the LWB here.
- Axiom Law. Venture and private equity-based company that helps legal departments more efficiently manage and source their legal needs. Discussed on the LWB here.
- Black Hills IP. Privately held onshoring company that does highly specialized IP-related paralegal work -- their internal motto is "innovate and automate." Founders were involved in an earlier LPO that sold to CPA Global a few year ago. Discussed on the LWB here.
- Datacert. An e-billing platform for legal departments that has added on a large overlay of data analytics so legal departments can more aggressively benchmark and monitor their expenses to outside counsel.
- Ernst & Young. Big Four accounting firm that hires an enormous number of law grads each year for its tax and consulting practices. Very much set up for the tastes and preferences of Millenial professionals including training, work space, and work-life balance.
- Exemplify. Start-up company founded by Professor Robert Anderson at Pepperdine Law and his student. Used super computer technology and inductive computational linguistics to identify the market standard language in a myriad of forms found in the SEC Edgar database. Will speed up negotiations on what is "market"; setting stage for eventual market convergence on standards.
- Huron Consulting. Publicly held consulting firm that formed out of the ashes of Arthur Anderson's post-Enron collapse. Although a business consulting organization, a surprisingly large part of their business is e-discovery through attorneys in U.S. and India. This group trudged through the company's 10Ks, which was a great educational experiemce for them. Discussed on the LWB here.
- Integreon. Venture- and private equity-based LPO that has tried to distinguish itself with its global platform and language capabilities. The company recently cut a deal with Microsoft to handle a large tranche of their patent portfolio work.
- KM Standards. Privately held legal knowledge management company that is trying to deconstruct the logic of contracts into standardized terms to enable autonmation and reduce ambiguity (and thus litigation). Potentially very disruptive.
- LegalForce. Privately held company hoping to recapture the lost consumer and start-up market through a novel storefront strategy. Financed at least initially through LegalForce's enormously successful online trademark practice run by the company's founder, Raj Abhjanker. More trademarks granted by PTO than any other law firm.
- Manzama. Privately held company in Bend, Oregon that scrapes the Internet with machine learning technology to filter business intelligence for law firms and other professional service firms track. Enormously scalable. Daily results presented through a dashboard technology.
- Modria. Online dispute resolution system that enables businesses and governments (mostly municipalities) to avoid costly, in-person legal proceedings to resolve a steady stream of similar disputes that are part of running a business or government. Discussed on the LWB here.
- Neota Logic. Privately held company founded by former Davis Polk partner and CIO Michael Mills. The company specializes in the creation of expert systems that can improve the quality and efficiency of many transactional and compliance related activities.
- Pangea3. LPO with substantial operations in India. Initially back by venture capital in 2004 but subsequently sold to Thomas Reuters in 2010. Employs roughly 1,000 lawyers in the US and India. Discussed on the LWB here.
- Recommind. Privately held company that specialized in predictive coding for use in document review and e-discovery. Founders were graduate students in Artificial Intelligence programs at Stanford and UC Berkeley in early 1990s. Discussed on the LWB here.
- Stewart Richardson. A privately held Indianapolis-based deposition services company that has gradually and successfully expanded into a broader array of law firm support services. Very focused on technology to make the job of clients easier.
The assignment was an experiment, albeit one that worked very well. Both students and the visiting lawyers reported surprise at the depth and breadth of the innovations taking holding the legal market.
Although some of the innovations where clearly eroding the need for traditional legal service jobs, the profiles also revealed the tremendous opportunities for those willing to stretch into the law and technology space. Many students commented that the evening drove home the point that they need to proactively obtain new skills and knowledge. Why? Because the emerging market has no secure place for the complacent or mediocre. Better for them to discover it in the course of an assignment than for me to say and have it fall on deaf ears.
Many thanks to the profiled company, who exhibited enormous generosity in helping my students complete this assignment. Remarkably, most groups had the benefit of a lengthy conference call with senior leadership. My only regret is that more practicing lawyers did not attend. My students, who have have 1L team and presentation experience, brought their "A" game. I will fix that in the next class, as there is no shortage of NewLaw companies to be profiled.
Monday, September 23, 2013
A few years ago I had the good fortune of teaming up with Rachel Zahorsky for a series of feature stories in the ABA Journal, including "Paradigm Shift" (July 2011), "The Law School Bubble" (Jan 2012), and "The Pedigree Problem" (July 2012). The fourth article, "Who's Eating Law Firms Lunch," is now online; and without a doubt it is my favorite.
Why? Because of the final vignette in the story, which features Dan Katz of ReInvent Law fame. We were sitting at the bar at the January 2013 AALS Conference in New Orleans when Dan told me this story. My jaw just dropped. Dan has faith in his students, just like Bellotti had faith in him. Dan believes, so Dan just does. No fear. No bullshit. It was, suffice to say, quite refreshing.
I am reposting the whole vignette in the hope that a few more academics, lawyers, and law students will read it. The title of the post is the last line in the story. To my mind, that Dan Katz line sums up the next ten years of innovation in legal education. Please keep reading until you get to that final line. The insight is worth the effort.
For the past two years, MSU’s Katz was the only full-time law professor who spoke at the LegalTech conference. Katz and Knake are creating a curriculum relevant to the emerging law and technology sector, albeit primarily for companies like Novus Law and Recommind, whose competitive advantage is rooted in process and technology.
Within the legal academy, Katz is an anomaly. Aside from his JD, he has a PhD in political science and public policy from the University of Michigan. However, he focused almost all of his graduate study on complex systems. It’s a relatively new scientific field that uses mathematical modeling to understand how a multitude of human and nonhuman factors interact and influence one another. Human society and the human brain are two examples of complex systems. Neither can be effectively modeled by conventional math or statistics.
The late Larry Ribstein at the University of Illinois was one of Katz’s early mentors. When he went into teaching a few years ago, Katz says, Ribstein told him: “I bet you must feel like an alien. I greatly admire your work. You are definitely on the right track. But the rest of the legal academy is just not ready for you.”
In June 2011, Katz joined the faculty at MSU Law. Michigan State partnered with the Detroit College of Law in 1995 and moved the law college into a building in East Lansing two years later. Though the school’s rebranding efforts did raise its profile, to most of the profession, MSU Law remains a nonprestigious regional law school located in the heart of the Rust Belt.
None of this dissuades Katz from his sincere belief that it is possible to turn any institution into the preferred recruiting grounds for the nation’s emerging law-and-tech industry.
“When I was 18 years old,” explains Katz, “I had the privilege of joining a transformative organization”— as a kicker for the University of Oregon football team, the Ducks. “We were in the Pac-10, but it’s in Eugene, Ore., where it is often cloudy and raining. We had no shot at all with the top recruits from Southern California. So coach Mike Bellotti had to figure out ways to stretch and optimize what some might call second-tier talent.
“Oregon is now a national powerhouse, but the seeds of that success were sown much earlier. It was difficult to be bigger or faster than USC or UCLA. So Coach Bellotti decided we would be better on the details of the game. We would be better conditioned and we would pay significant attention to special teams. Our emphasis on special teams got us better field position. And by the third or fourth quarters, our opponents would have their hands on their burning legs. But because of our conditioning regimen, we had more stamina. Our success became contagious. Over time, we were able to get prized recruits. It was a culture of innovation.”
During Bellotti’s tenure at Oregon, from 1995 to 2008, the Ducks had only one losing season, blotting out decades of mediocre performance. The year that Katz graduated, the Ducks were co-champions of the Pac-10, a feat that makes him beam with pride.
Katz’s “secret sauce” for ReInvent Law is arguably much more important than a degree in complex systems. He looks at the 25 students entering the ReInvent Law Laboratory as raw human potential. Katz also actively recruits potential law school applicants to his program, though he declines to discuss his strategy.
Katz understands that the most attractive candidates for the law and technology sector are those with special skills that are often obtained through prelaw work experience. “But there is no reason why some of those key skills and experiences cannot be learned and obtained right here,” Katz says of the MSU program.
He notes that virtually all law students have high cognitive ability. He feels the key to their future success is mastery of domain-specific knowledge—often in areas that are complementary to law—and the ability to collaborate across disciplines. This requires engagement and an immense amount of time spent on the task. So how does one develop the educational program that will prepare the law student for legal-tech jobs—some that may not yet exist?
“This,” Katz says, “is just an education design problem."
Saturday, September 21, 2013
Lawyers can successfully adapt to the disruption of the Information Age just like we adapted to the legal challenges of the industrial era -- build a system to create the human capital that is in short supply. This was original logic of the Cravath System, which created teams of specialized business lawyers who could handle the legal needs of rapidly growing industrial and financial clients in the early 20th century. This Clockworks approach still works, but the specifications of the system need to be updated. At the end of this presentation, I offer a prototype of what we might include in a 21st century Clockworks approach to lawyer development.
Presented at the "Innovations in the Law: Science and Technology" Conference, Oregon District of the Federal Bar Association (Sept 20, 2013)
Sunday, June 30, 2013
As noted in Part I of this post, the competitive dynamics among law schools are about to change due to a combination of two factors: (1) the ABA's collection and publication more granular data on school-level employment outcomes, and (2) the decision by U.S. News to make JD Bar Passage Required and JD Advantaged the primary measures for the employed-at-9-months input to its rankngs formula.
The histogram below reveals a near perfect bell curve for this revamped US News
input [click on to enlarge]. This is a huge change from prior years
when schools were all bunched at the 95% level because employment of any
kind was all that mattered. Under the old methodology, any law school that
limited itself to full-time, professional law-related jobs would have
plummeted in the rankings 10 to 50 spots.
Because spring 2013 was the first year with the new methodology, the impact of the change is not well understood. The most stark fact of the new environment is that the full-time, professional law-related jobs are in short supply. Among the class of 2011 (the stats used for the 2013 rankings), this desirable outcome was achieved by only 63.0% of graduates. When we subtract out full-time, long-term law-related professional jobs funded by law schools -- a luxury that only a small number of mostly first-tier law schools can afford -- the total drops to 61.9%.
Digging deeper, some other significant patterns emerge.
The vast majority of law schools feed into the regional labor markets where they are located. In places like California, those markets are saturated.
Among the ABA-accredited law schools in California, 46.5% of the class of 2011 obtained full-time JD Bar Passage Required jobs. The comparable figure for the remaining ABA-accredited law schools was 56.0%. Likewise, there is also a disparity for JD Advantage jobs: 6.2% in California versus 8.3% for schools in all other states. In fact, among the 19 ranked California law schools, only four -- Stanford, UC Berkeley, USC, UCLA -- are above the 63.0% average for full-time, professional law-related jobs.
Based on these data, it should come as no suprise that no law school located in California went up in the 2013 U.S. News rankings. Stanford, USC, and Santa Clara hung onto their ranking, but 11 California law schools dropped, with an average decline of 11 spots. Five other Calfornia schools remained in the unranked fourth-tier category.
In contrast, some of the biggest winners in the methodology change were flagship public law schools that are relatively big fish in smaller regional markets. Students at these schools tend to stay in-state and get JD Bar Passage Required jobs at rates far higher than the 54.9% average for the class of 2011 average.
Below are the top 15 non-national public law schools based on the proportion of FT Bar Passage Required jobs.
Between 2012 and 2013, the average rankings gain for the above schools was +9 spots. Among this group, the only school to go down in the rankings was ASU Law (-3). And that decline was largely due to the fact that ASU reported a 98% employed-at-nine-months figure for the class of 2010--a figure that drew suggestions of aggressive gaming. See Brian Tamanaha, When True Numbers Mislead, Balkanization, April 2, 2012.
The heavier weighting for JD Bar Passage Required jobs also benefits a handful of lower-ranked private law schools that are practice-oriented and tend to feed smaller firms within their regional areas.
- Campbell (71.4% FT bar passage jobs) went from unranked to #126.
- South Texas (64.4% FT bar passage jobs) went from unranked to #144
- St. Mary's (78.3% FT bar passage jobs) went from unranked to #140.
Part-Time Law Schools Dominate JD Advantaged Jobs
JD Advantaged Jobs count the same as JD Bar Passage Required Jobs. But what, exactly, is included in this category? According to the ABA,
A position in this category is one for which the employer sought an individual with a J.D., and perhaps even required a J.D., or for which the J.D. provided a demonstrable advantage in obtaining or performing the job, but which does not itself require bar passage or an active law license or involve practicing law.
See ABA Class of 2012 (definitions). Many professionals enroll in law school on a part-time basis to improve their career prospects. It should be no surprise, then, that schools with part-time programs tend to be the largest producers of graduates with full-time JD Advantage jobs. In many cases, it is the full-time job that the student held during law school -- and presumably retains upon graduation -- that confers the advantage.
Of the top 10 schools based on the percentage of JD Advantage law school jobs, eight had part-time programs and the other two were located in a state capital, which tends to increase the number of opportunities related to government and public policy.
The schools listed above gained an average of 3.5 spots in the rankings, albeit the average is pulled down by the inclusion of Southwestern, which had to weather the brutal California legal market.
It is worth noting that the percentage of JD Advantage jobs is negatively correlated with the percentage of JD Bar Passage Required Jobs (-.33) .The table below summarizes the differences between schools with Part-time versus Full-Time only programs.
The higher percentage of JD Advantage jobs (10.1% versus 6.9%) for schools with part-time programs is unlikely the results of chance, as the differences in means are statistically signficant at p < .001. But what does this inverse relationship mean?
programs tend to be affiliated with lower ranked law schools, which in turn would produce a lower average percentage of JD Bar
Passage Required jobs. Yet, part-time programs are also in larger,
urban locations. Thus, in addition to the continued employment of
part-time students with their current employers, the sheer proximity to
large, specialized regional economies probably increases the proportion
of JD Advantage jobs. Indeed, any school in an large metro area would
be foolish to ignore the human capital needs of non-legal employers, as
knowledge of the law is very helpful in navigating through an ever more
complex, regulated, and interconnected world.
What is the Best Strategy for Maximizing Full-Time, Professional Law-Related Jobs?
Largely through happenstance, the ABA and U.S. News have created an environment where law schools have to ask this basic but very important question. Part-time jobs will no longer cut it. And few law schools have the cash to hire their own grads full-time for a year past graduation -- and if they do, there are probably better uses for the millions of dollars needed annually to prop up a school's ranking.
The new gold standard employment outcome is full-time, long-term professional law-related jobs. The issue of how to maximize this outcome is so pressing and intricate that it may warrant trade-offs in the admissions process, favoring students will lower credentials but more rock-solid employment prospects on the backend at graduation. This is the topic I will take up in Part III.
[posted by Bill Henderson]
Friday, June 28, 2013
NALP recently released the employment outcome data for the class of 2012. The good news is that the absolute number of JD Bar Passage Required jobs went up from the prior year. The bad news is that a significantly larger class of entry-level lawyers were competing for those jobs. The class of 2011 totaled 41,623, versus 44,339 in 2012 (+2,716, or +6.5%). And note, the class of 2013 is likely to be even bigger -- roughly +1.6% based on the size of the entering 1L classes in the fall of 2010 (see ABA enrollment data).
Setting aside the year-over-year flucuations, the trendlines suggest a relatively large and persistent shortfall in the number of full-time, professional law-related jobs. I assembled the graph below from NALP data [click on to enlarge].
[Methodological notes: NALP used the JD-Preferred category until the class of 2011, when NALP and the ABA collaborated on the creation of the JD Advantage category. According to NALP, the jobs in the two categories are "largely the same." See NALP, Detailed Analysis of JD Advantage Jobs (April 2013). The figures for 2012 are estimates of full-time employment calculated from (a) NALP's just released figures for 2012 class size and the percentage breakdowns by job category, and (b) the percentage breakdowns of full-time versus part-time from the prior year, which also relied on the new JD Advantage definition. In short, basic algebra.]
A reasonable expectation of a 3-year, $100,000+ financial commitment is that nine months after graduation, the entry-level lawyer has secured a full-time professional job. See Legal Whiteboard, June 26, 2007. Those outcomes are reflected in the blue-red-green bars above. Since 2007 (the first year that NALP collected data on full-time versus part-time employment), the percentage of jobs fitting these criteria has fallen from 85.0% to 73.9%. So the overall size of the purple bar -- part-time jobs, nonprofessional, unemployment, etc. -- has grown from 15% to 26.1%.
Unfortunately, the pain does not end there. With a limited pool of full-time professional jobs and the number of graduates trending upward, the law of supply and demand kicks in. Consider this arc of median entry-level salaries of employed graduates: $65,748 for class of 2007, $72,000 for 2008, $72,000 for 2009, $63,000 for 2010, $60,000 for 2011, $61,245 for $2008. So, in short, the odds of landing a full-time professional job have gone down, and so has the starting pay. Yet, tuition and student debt continue to edge up. These unsustainable trends have made law schools fair game for criticism by the media and law student bloggers.
That said, a market correction is clearly underway. A considerable number of prospective law students are deciding (rationally) not to apply to law school -- from 98,700 when the class of 2007 enrolled in the fall of 2004 to an estimated 58,424 for the fall of 2013. Likewise, law schools, to the extent they can afford it, are enrolling fewer students. From the high water mark in the fall of 2010 (49,700), law schools only enrolled 41,400 1Ls in the fall of 2012, and the numbers are sure to be even lower this fall. See Jerry Organ's estimates, Legal Whiteboard, May 20, 2013. To weather this storm, law schools are running significant deficits or drawing down their endowments.
So, can we conclude that the market correction will be complete when the relatively small class of 2017 enters the job market four years from now? I certainly think the smaller number of graduates will help. But I would argue that two things have fundamentally changed:
1. Revenues versus credentials. Law schools are struggling with the need to balance their desire to hang onto respectable LSAT/UGPA medians with a need to generate sufficient revenue to cover their operating costs. If a law school favors revenues this year, its US News rankings could drop, affecting its applicant pool in future years. On the other hand, the combination of shrinking 1L classes and lavish scholarships -- a strategy being pursued by dozens of law schools -- is unsustainable over the medium to long term. A decision to enroll fewer students this year is a three-year commitment to lower revenue. If the smaller entering class is repeated next fall, the budget pain doubles. Do it three years running, and the revenue shortfall triples. Many law schools are not trying to outrun the bear; they are trying to outrun other law schools in their regional market. Some law schools may not make it out of this trough.
2. Competition over full-time, professional law-related jobs. If there is one silver lining that has emerged from this troubled period in U.S. legal education, it is the willingness of the ABA to collect and publish more granular employment outcome data at the law school level. In turn, U.S. News has incorporated these data into its rankings formula. Instead of propping up our rankings by hiring our own students or benefiting when they got jobs nine months out working as a retail manager or a cab driver, under the new 2013 U.S. News rankings formula, only full-time, long-term jobs that are JD Bar Passage Required or JD Advantaged are given "full weight."
It is this second point that is going to push change in how law schools do business--we now have an employment outcome in which the ranking payoff is now fully in allignment with what law students want--full-time, professional law-related jobs.
Specifically, the employed-at-nine-months input to the U.S. News rankings formula is currently given 14% weight. According to the U.S. News law school rankings methodology, the magazine is weighting 22 of the 35 employment outcomes collected and published by the ABA. Among these 22 factors, we don't know the internal weighting. What we do know based on the "full weight" given to JD Bar Passage Required and JD Advantage jobs, is that the highest employed-at-nine-month scores will go to law schools with the highest percentages in these two categories. This is a completely new world for law schools -- one that incentivizes what law students care about when they make the decision to enroll.
Part II to follow ...
[Posted by Bill Henderson]
Wednesday, June 5, 2013
For those trying to better understand how legal education can better prepare law students for the world that awaits them, I would encourage you to take a look at the draft article my colleague, Neil Hamilton, Director of the Holloran Center for Ethical Leadership in the Professions at the University of St. Thomas School of Law, recently posted on SSRN. The article is entitled Law-Firm Competency Models and Student Professional Success: Building on a Foundation of Professional Formation/Professionalism. Here is some of the description from the abstract:
A law student who understands legal employer competency models can differentiate him or herself from other graduates by using the three years of law school to develop (and to create supporting evidence to demonstrate) specific competencies beyond just knowledge of doctrinal law, legal analysis, and some written and oral communication skills. . . .
In Part I below, this essay analyzes all available empirical research on the values, virtues, capacities and skills in law firm competency models that define the competencies of the most effective and successful lawyers. Part II examines empirical evidence on the competencies that clients evaluate. Part III evaluates the competencies that make the most difference in fast-track associate and partnership promotions. These data and analyses lead to several bold propositions developed in Part IV:
1. Law students and legal educators should identify and understand the values, virtues, capacities and skills (the competencies) of highly effective and successful lawyers in different types of practice (one major example is law firm competency models analyzed below in Part I);
2. Each student should use all three years of experiences both inside and outside of law school (including the required and elective curriculum, extracurricular activities, and paid or pro bono work experiences) to develop and be able to demonstrate evidence of the competencies that legal employers and clients want in the student’s area of employment interest;
3. Law schools should develop a competency-based curriculum that helps each student develop and be able to demonstrate the competencies that legal employers and clients want; and
4. Both law students and law schools should understand that the values, virtues, capacities and skills of professional formation (professionalism) are the foundation for excellence at all of the competencies of an effective and successful lawyer.
The article presents far more useful information than can be summarized here, and different readers may be struck by different things discussed in the article. One of the most significant takeaways for me, however, is the convergence around an array of competencies frequently not taught in law school. The article analyzes competency models used to assess associate development at 14 medium to large law firms in the Twin Cities and compares that with some other literature on competencies clients look for in attorneys. The analysis demonstrates that in addition to traditionally understood technical skills – legal analysis, oral and written communication, and knowledge of the law – there is significant convergence around several competencies frequently not taught in law school – 1) Ability to initiate and maintain strong work and team relationships; 2) Good judgment/common sense/problem-solving; 3) Business development/marketing/client retention; 4) Project management including high quality, efficiency, and timeliness; 5) Dedication to client service/responsive to client; and 6) Initiative/ambition/drive/strong work ethic.
Whether law schools are going to be able to find efficient ways to offer students opportunities to develop these competencies, it is imperative that we make our students aware that they need to be developing these competencies to give themselves the greatest likelihood of professional success.
[posted by Jerry Organ]
June 5, 2013 in Data on legal education, Data on the profession, Important research, Innovations in legal education, Law Firms, Scholarship on legal education, Scholarship on the legal profession | Permalink | Comments (0)
Wednesday, April 17, 2013
Last month, The National Jurist published an article I wrote that was a tribute to Leonard ("Len") Fromm, Dean of Students at Indiana Law from 1982 to 2012. Len passed away in February. The editors at The National Jurist supplied the official title, which I thought was spot on: "What Every Law Student Needs to Excel as an Attorney: Introducing the Fromm Six." [original PDF] I am republishing the essay here because I want as many people as possible to know the story and contribution of this truly great man. [posted by Bill Henderson]
Introducing the Fromm Six, National Jurist (March 2013).
One of the greatest people in legal education that you have never heard of is a man named Leonard Fromm. Fromm served as Dean of Students at Indiana University Maurer School of Law from 1982 to 2012. On February 2, 2013, Dean Fromm passed away after a relatively short battle with cancer.
I want to discuss an innovation that Dean Fromm contributed to legal education—a contribution that, I predict, will only grow over time. This innovation is a competency model for law students called the Fromm Six. But first, let me supply the essential background.
After several years in counseling and adult education, Dean Fromm joined the law school in 1982 to preside over matters of student affairs. Over the course of three decades he quietly became the heart and soul of the Maurer School of Law. Dean Fromm was typically the first person that new students met during orientation—the law school administrator who completed character and fitness applications for state bar authorities and the voice that called out their names at commencement (with an amazing, booming tenor). During the three years in between, Dean Fromm counseled students through virtually every human problem imaginable. His most difficult work was done in his office with his door closed and all his electronic devices turned off. It was private work that was not likely to produce much fanfare.
During his tenure at Indiana Law, Dean Fromm’s title was expanded to include Alumni Affairs. The change did not expand his duties in any significant way—Len was already working 70 hours a week in a job he loved. Rather, the change reflected the fact that Indiana Law alumni associated (and often credited) Dean Fromm with the deepest and most abiding lessons of law school—overcoming self-doubt; confronting self-destructive behavior; recognizing the importance of relationships; finding the courage to try something again after disappointing failure; or discovering the ability to see the world through the eyes of one’s adversary or opponent.
One of the cumulative benefits of Dean Fromm’s job was the ability to track the full arc of lawyers’ careers, from the tentative awkwardness of the 1L year, to involvement in the school’s extracurricular events and social scene, to coping strategies for students not at the top of their class, and the myriad, unexpected turns in our graduates’ professional careers. During his tenure he interacted with nearly 6,000 students and stayed in contact with a staggering number of them after graduation. Invariably, he saw the connection between law school and a student’s subsequent success and happiness later in life (noting, in his wise way, that professional success and happiness are not necessarily the same thing).
In 2008, I started collaborating with Len on a project to construct a law school competency model. Our first iteration was a list of 23 success factors which we constructed with the help of industrial & organizational (IO) psychologists. Although valid as a matter of social science, the list was too long and complex to gain traction with students. In 2010, the faculty who taught Indiana Law’s 1L Legal Professions class got together and reduced the list of competencies to 15. Once again, we found it was too long and complex to execute in the classroom.
During the summer of 2011, as we were debriefing the challenges of another year in our competency-based 1L Legal Professions course, Dean Fromm said, “I have an idea.” A short time later, he circulated a list of six competencies that were appropriate to 1Ls and foundational to their future growth as professionals. Finally (or At last), we now had a working tool! Moreover, none of the professors teaching the Legal Professions course, including me, wanted to revise a single word—a veritable miracle in legal academia.
Upon reviewing the list I kidded Len that the new IU competency model should be called “The Fromm Six”, which was a play on the famous “Big Five” personality model that forms the bedrock of scientific personality testing. (Len had a Masters degree in Counseling Psychology as well as a law degree.) He just laughed. But the “Fromm Six” had a lot of resonance with the rest of us so the label stuck.
In May 2012, Dean Fromm retired from his position as Dean of Students and Alumni Affairs. At age 70 he was preparing to join us in teaching the 1L Legal Professions course. This was to be in addition to his usual Negotiations class, where he was a master. Instead, within a few weeks of retirement, Len was diagnosed with a virulent cancer that never let go.
None of us can make sense of Len’s death as it abruptly ended
a life of complete, unselfish service to a large community of students, faculty
and graduates. But, as best I can, I am
inclined to pay tribute to his life. And
to my mind, there is no greater tribute than to publish and publicize the Fromm
Six so that another generation of lawyers can benefit from his wisdom, grace and
Self-Awareness – Having a highly developed sense of self. Being self‐aware means knowing your values, goals, likes, dislikes, needs, drives, strengths and weaknesses, and their effect on your behavior. Possessing this competence means knowing accurately which emotions you are feeling and how to manage them toward effective performance and a healthy balance in your life. If self‐aware, you also will have a sense of perspective about yourself, seeking and learning from feedback and constructive criticism from others.
Active Listening – The ability to fully comprehend information presented by others through careful monitoring of words spoken, voice inflections, para‐linguistic statements, and non‐verbal cues. Although that seems obvious , the number of lawyers and law students who are poor listeners suggests the need for better development of this skill. It requires intense concentration and discipline. Smart technology devices have developed a very quick mode of “listening” to others. Preoccupation with those devices makes it very challenging to give proper weight and attention to face‐to‐face interactions. Exhibiting weak listening skills with your colleagues/classmates/clients might also mean that they will not get to the point of telling you what they really want to say. Thus, you miss the whole import of what the message was to be.
Questioning – The art and skill of knowing when and how to ask for information. Questions can be of various types, each type having different goals. Inquiries can be broad or narrow, non‐leading to leading. They can follow a direct funnel or an inverted funnel approach. A questioner can probe to follow up primary questions and to remedy inadequate responses. Probes can range from encouraging more discussion, to asking for elaboration on a point, to even being silent. Developing this skill also requires controlling one’s own need to talk and control the conversation.
Empathy– Sensing and perceiving what others are feeling, being able to see their perspective, and cultivating a rapport and connection. To do the latter effectively, you must communicate that understanding back to the other person by articulating accurately their feelings. They then will know that you have listened accurately, that you understand, and that you care. Basic trust and respect can then ensue.
Communicating/Presenting –The ability to assertively present compelling arguments respectfully and sell one’s ideas to others. It also means knowing how to speak clearly and with a style that promotes accurate and complete listening. As a professional, communicating means persuading and influencing effectively in a situation without damaging the potential relationship. Being able to express strong feelings and emotions appropriately in a manner that does not derail the communication is also important.
Resilience –The ability to deal with difficult situations calmly and cope effectively with stress; to be capable of bouncing back from or adjusting to challenges and change; to be able to learn from your failures, rejections, feedback and criticism, as well as disappointments beyond your control. Being resilient and stress hardy also implies an optimistic and positive outlook, one that enables you to absorb the impact of the event, recover within a reasonable amount of time, and to incorporate relevant lessons from the event.
Sunday, March 24, 2013
Each year, the instructors in Indiana Law's 1L Legal Professions class coordinate with Indiana Law's Office on Career and Professional Development (OCPD) to run the Career Choices Speakers Series -- 16 lunchtime forums on Thursdays and Fridays throughout the second semester. It has been an enormous hit with students. Although our 1Ls are required to attend at least three, a huge proportion of the 1Ls attend over ten.
Below is a photo of this Thursday's pizza run for the session on Direct Service Public Interest Lawyers -- 22 pizzas and the laptop/scanner used for attendance. Over the course of semester, we will purchase well over 300 pizzas. Who pays for all of this food and equipment (plus about a dozen dinners for students and alums that occur before and after these events)? An Indiana Law alumni who profoundly believes in the role of ethics and integrity to achieve personal and professional success in life. And he has done so quietly, behind the scenes, every year for the last five.
I thought our alum would enjoy seeing the pizza gurney. Thank you! You are opening students' eyes and helping them make better decisions, all through relationships with other lawyers.
[photo credit, 1L Dakota Scheu, via iPhone]. For additional information on this highly effective program, see my prior post, A New Tool for Lawyer Professional Development.
[posted by Bill Henderson]
Sunday, February 17, 2013
On March 8, 2013 - The ReInventLaw Laboratory - Founded by Daniel Katz and Renee Knake from Michigan State will host ReInventLaw Silicon Valley 2013 @ The Computer History Museum in Mountain View, CA.
Topics to be covered include:
LegalTechStartUp, Lawyer Regulation, Quantitative Legal Prediction, Legal Supply Chain, Project Management, Technology Aided Access to Justice, Design, 3D-Printing, Driverless Cars, Business of Law, Legal Education, Legal Information Engineering, New Business Models for Law, Lean Lawyering, Augmented Reality, Legal Process Outsourcing, Big Data, New Markets for Law, Virtual Law Practice, E-Discovery, Information Visualization, E-Discovery, Legal Entrepreneurship, Legal Automation … and much more.
What do I need to know?
- At all price points, the legal services market is rapidly changing and this disruption represents peril & possibility. This meeting is about the possibility ... about some of the game changers who are already building the future of this industry.
- This is a 1 day event featuring 40 speakers in a high energy format with specific emphasis on technology, innovation and entrepreneurship.
- It will highlight the new and growing portion of the legal services industry. It will not be boring.
- For more on our lab and related events please see: http://reinventlaw.com/
How Much Does it Cost?
This event is generously sponsored in part by the Ewing M. Kauffman Foundation, Michigan State University College of Law and the ReInvent Law Laboratory.
Thus, tickets are FREE but limited.
There will only be 400 tickets for this free event. Many of them are already taken and when they are gone, they are gone. Thus, if you or your friends/colleagues/students would be interested in attending -please sign up today.
Final Thoughts …
As I mentioned to Bill Henderson the other day … the old internet adage applies with equal vigor in the legal services industry "the future is here … it is just not evenly distributed."
Come join the future already in progress at #ReInventLaw Silicon Valley March 8th, 2013 (and at our other free public events in London and New York later in 2013).
February 17, 2013 in Current events, Fun and Learning in the classroom, Important research, Innovations in law, Innovations in legal education, New and Noteworthy, Structural change | Permalink | Comments (0)
Wednesday, February 13, 2013
My previous post on Washington & Lee's 3L Program stirred a lot of interest and commentary, including some disbeleiving critics. Fortunately, Professor Jim Moliterno agreed to write a reply essay, below, that completes the cycle. [Bill Henderson]
Jim Moliterno Replies [This is a long reply, so a PDF version online here]
A number of comments to Bill’s January 28 post and posts regarding it on other blogs cause me to enter this conversation.
Are students really coming to W&L because of the new curriculum? Yes, to a significant extent. How do we know? Because the entering students say so. As do many law schools, we administer a questionnaire to our enrolling students. Among the questions asked is the obvious one: why are you here?
In the most recent such survey the students were asked to rank the strengths of the law school. Here are the top ten, in order, according to the entering students:
- Third Year Curriculum
- Ranking / Prestige
- Quality of Life
- National Reputation
- Job Placement
- General Cirriculum
- Clinical Program
- Financial Aid Award
- Size of Lexington
The curriculum reform was first. Financial aid awards were 9th, just ahead of the “size of Lexington.” The data does not support the unsubstantiated claims of some bloggers that students are choosing W&L because of the generosity of financial aid awards.
The curriculum reform has steadily moved higher on the “strength” rankings given by enrolled students since 2009. The 2011 and 2012 surveys are nearly identical, and the written comments of students about their reasons for coming to W&L (none reprinted here), are more striking than the numbers themselves.
I don’t know of any better data on this proposition but the statements of those whose reasons are under study. If that data is unsatisfying to some, then they will continue to be unsatisfied.
Are there other reasons students come to W&L? Of course. W&L has a highly productive, highly visible faculty engaged in scholarship and projects at the highest levels. Some students undoubtedly value W&L’s faculty prowess. W&L is highly ranked. Some students undoubtedly are affected by a top 25 ranking. It has an excellent reputation as a small, closely-knit academic community. Some students select for the sense of community and size. No reason will ever be the only reason for prospective students to choose a law school. Changes made by law schools will affect student choices for or against a particular law school. The W&L curriculum reform is positively affecting a significant number of students’ calculus about choosing W&L.
And some do come because of the financial aid package they were offered. But the financial aid reason is unlikely to explain the increase in applications since 2008. Some students, the recipients of aid, undoubtedly come in part because of the aid. That is no different than the students who choose [insert name of any school] because of the financial aid they were awarded. In 2012, about the same number of offers of admission were made as in previous years, but instead of the usual 130 or 135 admittees choosing to attend, more than 260 made deposits. Some were asked to defer their attendance until 2013 and once the dust settled we had a class of 187 instead of the usual 130 to 135. This same class entering in 2012 listed the curriculum reform first and financial aid ninth as strengths of the law school.
What else was happening in 2008 and 09 when the applications increased by nearly 33% per year?
In 2009 and 10, while W&L applications were on the rise, the US News ranking fell from 25-34 (while its reputation rank among academics stayed steady). It has now recovered to 24. If anything, that should have led to a drop in applications during 2008-2011 rather than the sharp increases that actually occurred.
Can we exclude all other possible explanations than those previously mentioned? Of course not. It could be that being in a small, beautiful mountain town is all the rage among young adults and 33% more students want that now than wanted it in 2007. I know of no data to prove or disprove that proposition, so it remains one that could be true. The reality is that the students who have come in recent years rate the curriculum reform among the top reasons (often the most important reason) for their attendance at W&L. That matters.
There is empirical evidence that the W&L curriculum reform is engaging students more than in the traditional “no plan” third year curriculum. Is it perfect evidence? Of course not. Is it definitive evidence that has no flaw? Of course not. Is anything ever supported by perfect, definite evidence that has no flaw? Not to my knowledge. We make all of our most important decisions in life based on the best available evidence. As long as the evidence is empirically sound and statistically significant, it is worthy of respect. The evidence of W&L 3L engagement increases is sound and statistically significant and marks a path toward further research and verification.
One commenter suggested that the data is suspect because the peer schools have not been identified. Their data belongs to them, not W&L. LSSSE does not make specific school data available to other schools. So W&L has only a composite score for those peer schools. And it would be unseemly for W&L to reveal the specific schools. I will not do so here. But to be sure, W&L asked LSSSE to calculate the data from a list of schools because they are the schools with whom W&L competes for students and competes in the rankings. It would not have served W&L’s research interests to learn how it compares with a list of schools that it does not compete with in the marketplace. No one at W&L has the data for any specific school.
Nonetheless, do not be mistaken, the schools with whom W&L is compared in LSSSE data are the schools anyone would expect them to be: schools that by their geography, rank and quality compete with W&L in the relevant markets for students and placement.
One observation: in the legal profession and legal education in particular, the status quo never seems to need empirical justification. Only change is suspect and wrong until proven definitively to be otherwise. Is there any empirical evidence that the status quo third year is the best possible third year except that it has been done that way for a long time? None that I know of. The old adage, “if it ain’t broke don’t fix it” does not apply here. The third year of legal education is “broke”.
Amid calls for its abandonment by some, dating back at least to the early 1970s report by Paul Carrington, the third year is widely acknowledged to be of the least value among the three years. (See below on W&L’s largely unchanged approach to years 1 and 2.) The Roman Legions (and more than a few other military powers) have found out that the mere fact that something has been successfully done before is not sufficient evidence that it will prevail in the present or future. Arguing in favor of the status quo based on no empirical evidence, . . . based only on instinct and the argument that it is the way things are currently done, is an approach doomed to failure. Just ask Kodak. (And see my forthcoming book: “The American Legal Profession In Crisis,” Oxford, March 2013.)
How about the claim that “[W&L’s LSAT has] gone down every year since [the new curriculum was announced], while its GPA rank has, after a plunge, more or less returned to where it was.” The blogger made that claim, once again without any data, let alone empirically credible data. Actually the W&L median LSAT was steady at 166 from 2005-2010, dropped 2 points to 164 in 2011 and stayed at 164 for 2012. It has not “gone down every year since [the new curriculum was announced in 2008].” Meanwhile, the GPA of entering classes, which was in the 3.5 and 3.4 range in 2008-2010, has gone up to the 3.6 range (3.65 and 3.62) in 2011 and 2012. The two modest changes in LSAT and GPA have essentially off-set one another in US News points. Hardly the reason for pause suggested by the blogger.
It seems that as long as someone is arguing against change, no rules apply to the arguments’ underpinnings.
Here is what the empirical evidence from the LSSSE surveys shows and what it does not show: students are more engaged in their work and their work includes more writing, more collaboration and more problem solving. Here are a few charts even more striking than those Bill used in his post. Together they say that significantly more than their peers or their predecessors at W&L, current third year students are working more, writing more, collaborating more, applying law to real world problems more, and preparing for class more often. Overall, they describe a harder-working, more engaged student body. And they are working harder at acquire the skills that matter to success as a lawyer.
February 13, 2013 in Blog posts worth reading, Current events, Data on legal education, Innovations in law, Innovations in legal education, New and Noteworthy, Scholarship on legal education, Structural change | Permalink | Comments (6)
Sunday, February 10, 2013
We were born with a fast brain, but we need a slow one to advance civilization, among other things. I am talking about insights of behavioral economics being applied to lawyer decisonmaking and judgment, and I think the answer to my question is "yes". Indeed, I think the insights of behavior econonomics put a whole new and important gloss on the tired adage, "Thinking like a lawyer."
We cover the basics of this topic in my 1L Legal Professions class. Apparently, it resonated with one of my many attentive students, as he/she sent me this amazing science video. It boils down all of Dan Kahneman's brilliant Thinking, Fast and Slow treatise into four very engaging minutes. This is a vegetable that tastes like chocolate. (H/T to a wise anonymous 1L at Indiana Law.)
[posted by Bill Henderson]
Tuesday, January 29, 2013
Here it is in a nutshell. There is empirical evidence that Washington & Lee’s experiential 3L curriculum is delivering a significantly better education to 3L students—significantly better than prior graduating classes at W&L, and significantly better than W&L’s primary competitors. Moreover, at a time when total law school applicants are on the decline, W&L’s getting more than its historical share of applicants and getting a much higher yield. When many schools are worried about revenues to survive next year and the year after, W&L is worried about creating the bandwidth needed to educate the surplus of students who enrolled in the fall of 2012, and the backlog of applicants that the school deferred to the fall of 2013.
[This is a long essay. If you want it in PDF format, click here.]
Alas, now we know: There is a market for high quality legal education. It consists of college graduates who don’t want to cast their lot with law schools who cannot guarantee students entree to meaningful practical training. Some might argue that W&L is not objectively better-- that the 3L curriculum is a marketing ploy where the reality falls well short of promotional materials and that, regardless, prospective students can't judge quality.
Well, in fact there is substantial evidence that the W&L 3L program delivers comparative value. The evidence is based on several years' worth of data from the Law School Survey of Student Engagement (LSSSE). I received permission from Professor James Moliterno, someone who took a leadership role in building W&L’s third year program, to share some of the key results (each school controls access to its LSSSE data.) They are below.
But before getting into empirical evidence, I want to put squarely on the table the most sobering finding that likely applies to virtually all of legal education. It is this: On several key LSSSE metrics, W&L has made impressive gains vis-à-vis its own historical benchmarks and its primary rival schools. But even for this leader, there remains enormous room for improvement. More on that below.
Here is the bottom line: Traditional legal education, when it is measured, does not fare very well. Yet, as W&L shows, substantial improvement is clearly possible. We law professors can respond to this information in one of two ways:
- Don’t measure, as it may disconfirm our belief that we are delivering a great education.
- Measure—even when it hurts—and improve.
I am in the second camp. Indeed, I don’t know if improvement is possible without measurement. Are we judging art work or the acquisition of key professional skills needed for the benefit of clients and the advancement of the public good?
Moving the Market
I doubt I will ever forget Jim Moliterno’s September 2012 presentation at the Educating Tomorrow’s Lawyers (ETL) conference at the University of Denver. He presented a single graph (chart below) showing W&L actual applicant volumes since 2008 versus what would have happened at W&L if its applicant volume had followed national trends.
While law school applicants crested a few years ago, W&L enjoyed a large run-up in volume of applicants, presumably due to the launching of their new 3L program. This larger applicant pool effectively served as a buffer when applicant declines began in 2011 and 2012. Since 2008, overall law school applicants are down -19%, yet W&L is up overall +33%.
But much more significantly, after their experiential 3L year was up and running and the overall legal job market continued to stagnate, W&L yields spiked. Ordinarily they would enroll 135 students. But for the fall of 2012, they received enrollment commitments from well over 260 students. Indeed, at the ETL conference Jim Moliterno said the school had to offer financially attractive deferments to get the class to approximately 185 incoming students -- a 50 student bulge.
When Jim Moliterno showed the above graph and explained the corresponding changes in yield, my good friend Gillian Hadfield, a skeptical, toughminded, evidence-demanding economist who teaches at USC Law, leaned over and said to me, “that is the single most important takeaway from this entire conference.” I agreed. The market for a legal education with practical training is, apparently, much more inelastic than the market for traditional JD programs.
Yet, what is perhaps most remarkable is that a large proportion of incoming students at W&L were enrolling based on little more than faith. Nobody knew for sure if W&L had the ability to pull off their ambitious 3L curriculum. The program relies on a large cadre of adjunct professors, after all, and W&L is located in remote Lexington, Virginia. Many law faculty outside of W&L, and perhaps some inside, thought (or perhaps think) that the program could not live up to the hype. Well, as shown below, the program appears to have produced meaningful gains.
The only data-driven critique anyone can muster is that the gains remain significantly short of perfection. But that critique bites harder on the rest of us. To use a simple metaphor, W&L is tooling around in a Model-T while the rest of us rely on horse and buggy. What ought to be plain to all of us, however, is that, just like automobile industry circa 1910, we are entering a period of staggering transformation that will last decades. And transformation will be roughly equal parts creation and destruction. See Schumpeter.
W&L Data, Internal Historical Benchmark
LSSSE is a phenomenally rich dataset – nearly 100 questions per year on a wide variety of topics related to student classroom experience, faculty interaction, type and quantity of assessments, time allocation, and perceived gains on a variety of dimensions related to personal and professional development. The survey instrument is online here.
Aside from a host of questions related to demographics, career goals, and debt, major sections in the LSSSE include:
- Section 1, Intellectual Experience (20 questions)
- Section 2, Examinations (1 question)
- Section 3, Mental Activities (5 questions)
- Section 4, Writing (3 questions)
- Section 5, Enriching Educational Experiences (9 questions)
- Section 6, Student Satisfaction (7 questions)
- Section 7, Time Usage (11 questions)
- Section 8, Law School Environment (10 questions)
- Section 9, Quality of Relationships (3 questions)
- Section 10, Educational and Personal Growth (16 questions)
W&L deserves to be a detailed case study. But frankly, legal education can’t wait. So I will do the best I can to cover the landscape in a blog post. I hope every law faculty member who reads this post makes a strong plea to their dean to enroll in LSSSE. Why? So your school can benchmark itself against the detailed LSSSE case studies that are bound to flow out of W&L and other innovative law schools. Though they don’t get much press, there are, in fact, other innovative law schools.
Friday, January 18, 2013
Brian discusses the bleak employment prospects of law schools, but (through no fault of his own) understates the nature of the structural change that is occurring in the U.S. and global market for legal services. In Part II, I will write about some logical next steps for law schools looking to get ahead of the coming tsunami.
I tried to write Part II, but a blog post just was not up to the task. Further, I sensed that my colleagues were in no mood for half-baked solutions. There has been enormous criticism of legal education on the blogs and in the media, but very little in the way of detailed prescriptions to improve the situation. I felt an obligation to back off on the criticism and focus on solutions. So, in essence, Part II of my Tamanaha review became an article.
I just posted to SSRN an article entitled "A Blueprint for Change" forthcoming in the Pepperdine Law Review. It is both a diagnosis and a proposed solution -- a solution I am actively pursuing. Here is the abstract:
This Article discusses the financial viability of law schools in the face of massive structural changes now occurring within the legal industry. It then offers a blueprint for change – a realistic way for law schools to retool themselves in an attempt to provide our students with high quality professional employment in a rapidly changing world. Because no institution can instantaneously reinvent itself, a key element of my proposal is the “12% solution.” Approximately 12% of faculty members take the lead on building a competency-based curriculum that is designed to accelerate the development of valuable skills and behaviors prized by both legal and nonlegal employers. For a variety of practical reasons, successful implementation of the blueprint requires law schools to band together in consortia. The goal of these initiatives needs to be the creation and implementation of a world-class professional education in which our graduates consistently and measurably outperform graduates from traditional J.D. programs.
I have a large backlog of shorter articles and analyses that I have not posted because I wanted my own detailed solution in the public domain. I hope to tie all of these ideas together over the coming weeks.
Thank you, Brian Tamanaha, for writing an book that required me to think in terms of solutions.
[posted by Bill Henderson]
January 18, 2013 in Current events, Data on legal education, Data on the profession, Innovations in legal education, Scholarship on legal education, Scholarship on the legal profession, Structural change | Permalink | Comments (2)
Monday, November 19, 2012
Law schools care deeply about their academic reputation. If this were not true, my Indiana Law mailbox would not be stuffed full with glossy brochures sharing the news of faculty publications, impressive new hires, areas of concentration, and sundry distinguished speaker series, etc.
Because of the timing of these mailings – I got nearly 100 in Sept and October—I am guessing that the senders hoped to influence the annual U.S. News & World Report Academic Reputation survey. Cf. Michael Sauder & Wendy Espeland, Fear of Falling: The Effects of U.S. News & World Report Rankings on U.S. Law Schools 1 (Oct 2007) (reporting "increases in marketing expenditures aimed toward raising reputation scores in the USN survey"). But does it work? A recent study by Larry Cunningham (St. Johns Law) suggests that the effect is, at best, decimal dust.
Glossy brochures may not reliably affect Academic Reputation, but I have uncovered four factors that are associated with statistically significant increases and decreases of USN Academic Reputation. To illustrate, consider the scatterplot below, which plots the 1993 ordinal rank of USN Academic Reputation against the 2012 ordinal rank [click on to enlarge].
Four sets of dot (Red, Blue, Orange, and Green), each representing distinctive shared features of law schools, tend to be above or below the regression line. These patterns suggest that changes in USN Academic Reputation over time are probably not the result of random chance. But we will get to the significance of the Red, Blue, Orange, and Green dots soon enough.
The primary takeaway from the above scatterplot is that 2012 USN Academic Reputation is overwhelmingly a function of 1993 USN Academic Reputation. Over 88% of the variation is explained by a school's starting point 20 years earlier. Part of this lock-in effect may be lateral mobility. That is, there are perks at higher ranked schools: they tend to pay more; the teaching loads are lighter; and the prestige is greater, etc. So school-level reputations rarely change, just the work addresses of the most productive scholars. This is, perhaps, the most charitable way to explain the enormous stickiness of USN Academic Reputation.
That said, the scatterplot does not show a perfect correlation; slightly less than 12% of the variation is still in play to be explained by influences other than starting position. A small handful of schools have made progress over these 20 years (these are the schools above the regression line), and a handful have fallen backwards (those below the line).
The Red circles, Blue rectangles, Orange diamonds, and Green circles represent four law school-level attributes. The Reds have been big gainers in reputation, and so have the Blues. In contrast, the Oranges have all experienced big declines; and as as a group, so have the Greens. When the attributes of the Red, Blue, Orange, and Green Schools are factored into the regression, all four are statistically signficant (Red, p =.000; Blue, p = .001; Orange, p = .012; Green, p = .000) and the explained variation increases 4% to 92.3%. As far as linear models goes, this is quite an impressive result.
Before you look below the fold for answers, any guesses on what is driving the Red and Blue successes and Orange and Green setbacks?
Thursday, September 6, 2012
Below is my most recent column in the National Jurist [PDF version]. Although 100% targeted at law students, I think lawyers and law professors might find this topic interesting. [Bill Henderson]
Richard Susskind is a famous British lawyer and technology consultant who travels the world giving speeches on how the legal industry is on the brink of a fundamental transformation. Because his topic is change, Susskind’s ideas are quite controversial among lawyers. But as a futurist, he has a pretty good track record.
Back in 1996, in his book The Future of Law, Susskind predicted that e-mail would someday become the dominant method for lawyers and clients communicate with each other. Because the Web was still a novelty limited to universities and computer aficionados, Susskind’s comments were viewed as reckless and unprofessional—lawyers would never rely on such an insecure method to communicate with clients. Yet, 16 years later, lawyers are daily lives are comprised of an endless stream of emails coming over their desktops, laptops and smart phones.
Saturday, September 1, 2012
For the Labor Day weekend, I thought I would post this video of Henry Rollins, an American singer and artist who has continually reinvented himself since he left his job as a manager of a Hagen-Daaz ice cream store in 1981 to become the lead singer in Black Flag.
The point of posting this video is not to glorify Henry Rollins, but to consider, on its own terms, the life narrative of one interesting person. Rollin's formula of "application, discipline, focus, repetition" sounds a lot like deliberate practice. Based on my own research, I have broken this process into two steps:
- Identifying the core elements needed to be become an expert or master in a specific domain -- Jeff Lipshaw was alluding to this in his post on Donald Schon and reflective practice;
- Practicing, through thousands of hours of effort, on elements that one lacks in order to move along the continuum to mastery. Number 2 works best when the person has the benefit of feedback and coaching. Of course, they also have to be willing to do the work.
For an individual, it may not be necessary to formally break down the core elements into specific pieces. Instead, these pieces can be obtained iteratively through trial and error and reflection. I think this is what Rollins has done. It is a formula that works for one highly determined person. But can it be scaled?
As an educator, I am interested in making the components of practice mastery more explicit and transparent--this is step #1 above. To accomplish step #1, we still need to do foundational research that deconstructs the careers of outstanding lawyers into sets of specific skills, abilities, and competencies--i.e., the things to be practiced. (Notice I said "sets" -- outstanding lawyers often master different domains.) At present, the Shultz-Zedeck Effective Lawyering study is the only solid published research that is even adjacent to this topic.
Once these components of effective lawyers are identified--i.e., a law school identifies the skills, abilities and competencies it wants to develop over the course of three years--we move to step #2. This step raises complex questions of order (which competencies first, which come second, etc.) and pedagogy (best and most cost-effective methods) and measurement (how do we know we have made progress?). I think the answers would have to come iteratively, through trial and error.
Any educational institution pursuing this strategy would have to commit itself to studying and continuously improving the educational process. For law schools, this would be new. At the vast majority of law schools, we mostly teach legal knowledge, we don't articulate our intended educational outcomes, we let students pick their courses ala carte with minimal guidance, and we don't engage in serious measurement. But we could. I think this is the next great frontier--an enormous opportunity for any law school willing to think for itself, to experiment and to change. The data needed would come from one's own alumni, ideally supplemented with data sharing within a law school consortium.
[posted by Bill Henderson]
Monday, July 2, 2012
[by Bill Henderson, originally published in The National Jurist, March 2012 (PDF)]
For over a century, law schools have suffered from an inferiority complex. We have masked it well, but its consequences are finally coming home to roost. Like most psychological conditions, our lives will be much better and healthier when we deal with its root cause. Further, when law students understand this history, they will better understand the changing nature of the legal economy. They can even help law schools with the cure.
In 1918, the renowned economist Thorstein Veblen famously quipped, “the law school belongs in the modern university no more than a school of fencing or dancing.”
Veblen, like many of his academic colleagues, believed that universities should be citadels for science-based learning and the production of knowledge. Law, in contrast, was a trade. Indeed, in the early 1900s, a substantial portion of the practicing bar had obtained their skill and knowledge through office apprenticeships. When law schools did begin to appear, they were just as likely to be proprietary law schools operating out of a local YMCA than to be part of an established university.
Despite the skepticism of the academic class, there were a host of practical reasons for universities to create (or, in some cases, acquire) a law school. First, the law was the primary occupation for many elected officials, which held out the prospect of reflected glory. (Veblen recognized this motivation, which compounded his worry.) Second, a handful of law schools at prestigious universities had begun adopting the so-called case method, which purported to find objective legal rules and principles akin to a scientist working in a laboratory. The perceived rigor of the case method provided at least a veneer of science. Third, with their large lecture halls filled with tuition-paying law students, law schools made money.
Friday, June 29, 2012
Ignite rules are simple: a talk with 20 PowerPoints that advance automatically every 20 seconds. Six minutes to make your point. It you don't know your material, it's a disaster. If you are prepared and you understand how to connect with your audience, you educate and inspire -- in a word, you ignite the audience.
Below is an example of Ignite done very well, by Michael Bossone (Miami Law, co-founder of Law Without Wall). This presentation just got a rousing ovation at the Law Tech Boot Camp in London. I saw it happen live. It was awesome.
[posted by Bill Henderson]