Wednesday, March 22, 2017
Harvard Law School’s readiness to accept the GRE in place of the LSAT raises a host of important questions – most importantly, what will the effect be?
In explaining the move, Jessica Soban, assistant dean and chief admissions officer, told The New York Times that it would “encourage more students in the United States and internationally from a greater degree of disciplines to apply,” and the law school’s dean, Martha Minow, suggested that the law school would be able to admit a more diverse class as a result.
Whether such goals are realized will depend on numbers other than GRE scores. The results matter, because other law schools likely will follow the lead of Harvard and the University of Arizona, which announced the same move last year.
Many more students take the GRE each year than take the LSAT: 584,677 worldwide, including 326,957 United States citizens, in the 2015-2016 cycle, according to Educational Testing Service (ETS), which administers the GRE. In contrast, 84,771 people took the LSAT (and 56,500 applied to law school) in roughly the same period, according to the Law School Admission Council (“LSAC”), which administers the LSAT.
The larger pool of potential applicants could indeed produce greater racial and ethnic diversity at law schools. According to ETS, 55 percent of the U.S. citizens taking the GRE were white, 6 percent were Asian American, 7 percent were African American, 3 percent were Mexican American, and 4 percent were classified as “other Hispanic,” a category excluding Puerto Rican citizens, who accounted for 1 percent.
In contrast, about 61 percent of law school applicants were white, 15 percent were Asian American, 15 percent were African American, and 13 percent were Latino. (I did not find precisely analogous data, so please note that these percentages reflect two different sets of numbers: GRE test takers who are U.S. citizens, on the one hand, and law school applicants, on the other.)
Average GRE scores of different racial and ethnic groups vary across the population of U.S. citizens who took the test, according to ETS. White and Asian American test takers earned higher average scores than black, Mexican American or “other Hispanic” test takers. White test takers tended to score higher on the verbal reasoning component of the exam and Asian American test takers tended to score higher on quantitative reasoning.
As with the LSAT, on which LSAC reports similar patterns, if admissions processes at relatively more selective institutions continue to weigh test scores heavily, the greater size of the pool may have less of an effect on student diversity at law schools. The selection process, committed to a particular definition of merit, may get in the way.
Just how important is the LSAT, or GRE, for that matter, in predicting who will do well in law school? To ask that question is to raise another, which is, how good should it be in order to play the powerful role in admissions decisions that it does. According to an ETS report for the University of Arizona last year, “scores for all GRE subtests, individually or composited, are both highly reliable and valid for use in law school admissions.”
The study was based on a sample of 78 people, and took into account GRE, LSAT, undergraduate GPA and first term law school GPA. The study did not capture how students did in the second and third years of law school.
The ETS study presented the data in different ways but the most intuitive may be this: 54 percent of students in the top third of GRE composite scores had a first semester law school GPA in the top third of their class, while 23 percent were in the bottom third. Of students in the bottom third of GRE scores, 48 percent were in the bottom third of their law school class and 16 percent were in the top third. That means that 52 percent – more than half – of the students in the bottom one-third on the GRE did not land in the bottom one-third of their law school class.
To put that finding in context, the studies mentioned in the report for the University of Arizona found that the GRE verbal and quantitative sections “almost always predicted graduate [school] grade point average (GGPA) in general and first year GGPA at least as strongly as [undergraduate] GPA.”
A recent study, available here, suggests that the LSAT itself is not a great predictor of law school performance. While an LSAT score is a statistically significant predictor of law school GPA (twice as accurate in the first year than overall), the effect is not overwhelming. Each additional LSAT point predicts an increase in law school GPA of 0.016 – a bigger deal, the bigger the LSAT score difference between two applicants, but nonetheless a “modest” effect “compared to how heavily schools weight LSAT scores.”
So both tests predict law school performance, but to a limited degree.
Where should this leave us? Frankly, uncertain. The tests do not predict a lot. The ETS report for the University of Arizona does not look beyond the first semester of law school. Average performance on both LSAT and GRE appears to vary with the race of the test-taker.
Law schools have been using an imperfect tool and two are now moving toward incorporation of a complementary but also imperfect tool. While predictions are always risky, especially about the future, it seems highly likely that results of adoption of the GRE will be imperfect, too.