Does Academic Support Matter? A Brief, Preliminary Response to Blinded by Science and its Progeny, Part Two

In our previous post we discussed the Blinded by Science series’ essential thesis, that other factors at FIU Law, other than teaching the science of learning, impact its bar pass rate.  By demonstrating numerous methodological flaws in the series' analysis, we suggested that Blinded by Science's conclusions about transfers and attrition are faulty. 

In this post, however, we do something probably unexpected:  We agree with Blinded by Science.  

Here is what we mean....

-------

To investigate its central theme (other impacts on FIU Law’s bar passage), the series digs deep into the pile of ABA Form 509 records; it uses extravagant mathematical methods; it examines university records, details listserv quotes, and exposes personal emails.  All to prove what?  All to prove the unremarkable proposition that FIU Law’s higher-than-expected bar pass rate involves causal factors other than the science of learning. 

We absolutely agree. 

In fact, we agree that transfers and attrition impact bar passage.  In our forthcoming paper, we will detail the study we (Ruiz) conducted that suggests that transfers and attrition do have a weak but statistically significant impact on bar passage. (We further detail how Blinded by Science’s  methodological flaws led to an overstatement of that impact.)

But here is what Blinded by Science got wrong:  Delta.

In mathematics, “delta” means “change.”  The general idea is that if a change in a dependent variable, such as bar passage rate, is due to the influence of an independent variable, such as the number of transferred or academically dismissed students, there must be change in the independent variable to account for the change in the dependent variable.  In other words, if the transfer/ attrition theory helps explain the 2015 increase in the bar pass rate, the transfers and attrition numbers must have changed between 2014 and 2015.  If so, we would see, say, five transfers in the 2014 bar exam cohort, and twenty in the 2015 group; and we would further see five students attritted in the 2014 bar exam cohort and twenty in the 2015 cohort.  Thus, if the transfer/ attrition theory is valid, the bar pass rate increase had to coincide with a statistically significant change in the transfer and attrition numbers between those two years.

But no meaningful change occurred. 

With an exception that we will rebut, The Blinded by Science articles did not look at this.  The Blinded by Science articles’ only focus was on explaining, by looking solely at the cohorts from 2015-2018, the other factors involved in FIU Law’s pass rates during that period, after the initial increase.  But the articles do nothing to explain what happened between 2014 and 2015. 

So, why is it that the bar pass rates changed, increasing almost 11 points from 2014 to 2015, even though the number of transfers and academic dismissals did not (and despite the fact that the statewide pass average dropped four points)?  Our contention all along has been only that academic support contributed to that change. 

Our claim is that, together with the contributions noted above and a number of other initiatives designed to support student success, which we address in the article, academic support simply leveraged potential that already was there in the first place.  Although we have never claimed to be the cause of the bar pass rate increase, we unflinchingly believe that we contributed to it … and that other ASPs can and do, too. 

And that is why the Blinded by Science articles are so problematic.  By arguing so forcefully, and personally, that we FIU Law Bar Ninja/ Silver Bullet charlatans could not possibly have impacted the school’s bar pass rate, Blinded by Science unintentionally articulated an intense doubt about whether academic support can be a real factor in bar passage.  That suggestion undermines the purpose of our previous writings, which was to provide evidence that a well-resourced ASP, given latitude to do the work it needs to do, can tangibly contribute to bar passage.  Our point was not to say, as the author contends, that any ASP unable to produce these results is doing something wrong.  To the contrary, our point was to say that because ASPs in American law schools are almost never adequately resourced, empowered to use effective methods, or given adequate conditions of employment, law schools are preventing them from realizing their potential and needlessly blocking their own students’ success. 

We worry that, instead of interpreting FIU Law’s success as an indication of ASP as a worthy investment, law schools crediting the Blinded by Science series’ analysis of the transfer/ attrition theory will simply choose to fail out more students.  We worry that if law schools credit Blinded by Science’s Bar Ninja/ Silver Bullet theory, they will employ very few academic and bar support instructors, expect them to be Ninjas, and then put responsibility for bar passage solely upon these one or two untenured, unsupported, and pedagogically restricted ASP professionals.  

Our forthcoming article will demonstrate that we never claimed that the science of learning was the sole cause of our bar pass rate or to being Bar Ninja’s possessing silver bullets.  We will demonstrate that by attributing comments of others to us, citing quotes without actually citing quotes, and ignoring numerous statements we made actually denying the sole causation/ Ninja theories, Blinded by Science, et al. weaved a picture that just is not true. 

But, even more importantly, we will double-down on one contention we know to be true:  Academic support matters, and law schools should finally, after three decades, fully embrace the methods ASP faculty have championed for years.

[Louis Schulze & Raul Ruiz, FIU Law]