Thursday, April 20, 2017
Two interesting items came past my desk this morning. (Actually, past my computer screen.) The first one concerned the news that Whittier Law School will be closing. (This is very sad news for the students and faculty.) The other was a study of the effectiveness of formative assessment for student learning. I think these two very different stories are related.
Law schools have changed radically over the last fifty years. More students are going to law school, and these students come from very diverse backgrounds. In addition, many of these students come from poor educational backgrounds. Furthermore, colleges seem to be dumbing down their curriculums, particularly in the areas of writing, logical thinking, and critical reasoning.
Some law schools have reacted to the new type of students; others haven't. I wonder if Whittier's low bar score isn't largely due to the fact that it failed to change in reaction to the new type of students it was admitting.
I have argued that formative assessment with prompt and frequent feedback is critical to teaching the new type of law students. (E. Scott Fruehwald, How to Help Students from Disadvantaged Backgrounds Succeed in Law School, 1 Texas A & M Law Review 83 (2013) (here).) I think that formative assessment is particularly vital in the first year when students are developing their legal reasoning and problem-solving skills.
Now, a group of scholars at Ohio State has written an important study concerning the effectiveness of formative assessment on law student learning: Deborah Jones Merritt, Ruth Colker, Ellen E. Deason, Monte Smith and Abigail B. Shoben, Formative Assessments: A Law School Case Study.
"Several empirical studies have shown that formative assessment improves student learning. We build on those studies by reporting the results of a natural experiment at The Ohio State University Moritz College of Law. Students in one of three first-year sections had the opportunity to complete a formative assessment in their spring-semester Constitutional Law course. The assessment consisted of an essay question that the professor had used on a prior exam. Students who submitted an essay answer received prompt, extensive written feedback; they also had the chance to discuss their answer with the professor.
Over the course of three years, about half of the students enrolled in the section took advantage of the formative assessment. Those students achieved significantly higher grades on the final exam even though the assessment score did not factor into their course grade. Notably, students receiving this formative feedback also secured a significantly higher GPA in their other spring-semester classes. Both of these effects persisted after controlling for LSAT score, UGPA, gender, race, and fall-semester grades. These controls helped reduce any effect of selection bias on our findings.
In addition to exploring these relationships between formative assessment and academic achievement, we discuss several race and gender effects that emerged in our analyses. Women, for example, were significantly more likely than men to complete the formative assessment. Women also received significantly higher grades than men in a spring-semester course on Legal Analysis and Writing; men, conversely, received significantly higher grades than women in a Legislation course. A race effect, meanwhile, emerged for students with LSAT scores at or above the school median: Among those students, nonwhite students who completed the formative assessment achieved significantly higher grades in Constitutional Law than white students who submitted the same exercise.
All of these relationships deserve further empirical study. In particular, our results suggest the importance of examining the transfer effects of formative feedback, gender differences in law school learning, and paths for improving the academic experience of minority students."
If law schools do nothing else to improve law student learning and, consequently, their bar passes rates, they should add frequent formative assessment to their first-year classes. The studies show that this will significantly improve student learning.
Update: "Whittier College President Sharon Herzberger said in an interview that the board 'was concerned about the student outcomes at the law school' — namely how many students were graduating, passing the bar exam and finding employment in the legal profession." (here) So, the law school is being shut down because it isn't doing an effective job of educating its students. I hope other law schools with low bar pass rates or poor placement results are paying attention.
Further update: "Whittier College President Sharon Herzberger told the Wall Street Journal bluntly that students were not being prepared well enough to pass the bar exam and secure legal jobs. Fewer than a quarter of Whittier Law school’s graduates who took the July 2016 bar exam passed the test, the worst performance among accredited law schools in the state, according to the Journal.
Out of 128 graduates in 2016, only 45 went on to find jobs that required a law degree." (here)
P.S. The study showed that a major formative assessment in one class also helped improve grades in other classes taken during that semester. What do you think would happen if all classes included frequent formative assessment?