Do New School Report Cards Reveal Something Worth Knowing or Are They Just Statistical Magic?

Texas has released its statewide grading of public schools, using a grading scale of A to F.  The grades would appear to be a reflection of school poverty levels than anything else.   The Texas Tribune’s analysis shows that “No school district with a rate of low-income students lower than 30 percent received an overall rating of C, D or F.”  And as the level of poor students in the schools increase, the letter grades become more variable, but the clear trend is that letter grades substantially decline.  Almost the entire cluster of Cs, Ds, and Fs is with schools with poverty rates in excess of 50%.

This data poses crucial questions and possibilities.  First, is the variability in letter grades among high poverty schools an indication that the system really is distinguishing between higher and lower quality schools.  Second, do schools have any way to predict, understand, and respond to their overall poor performance?  If they don’t, this system is no better than the statically teacher evaluation systems that state leaders claimed would magically transform the teaching profession, but which were shown to be pretty much pointless, if not harmful, within just a few years. 

Third, even if schools understand these results, do those at the bottom end actually have the resources they need to make change.  As research increasingly shows, money does, in fact, matter to student outcomes and it matters a lot.  Unfortunately, Texas schools are way short on it.  Recent school finance litigation has shown how Texas has been dramatically underfunding its education system.  The largest scale snapshot shows school funding is down roughly 16% in real dollar terms in Texas since 2008.

The whole system, however, might be nothing more than voodoo magic.  States adopted their new rating systems pursuant to the Every Student Succeeds Act.  And as I emphasize in Abandoning the Federal Role in Education,

under the ESSA, states have enormous flexibility in the amount of weight they assign to particular tests and to student achievement factors overall. Not only does this flexibility permit an individual state to minimize the weight it assigns, but it also allows every state to do something different.  One state might make student proficiency tests the dominant measure of student achievement while another state uses student growth.    And regardless of the approach a state takes, states can assign significantly different weights to tests and other student achievement measures. A state might, for instance, assign test results 95 percent in their accountability metric and any number of non-test factors 5 percent or less collectively.  Another state might assign test results 60 percent in its accountability metric while assigning 40 percent to softer factors, such as student engagement, teacher engagement, and school climate.  With a number of options, states will have the ability to manipulate their accountability systems so as to produce desired outcomes. 

None of the foregoing means to suggest that testing is an effective means to promote equal education opportunity or that some optimum weight should be afforded to test results. The point here is that the ESSA maintains the NCLB’s notion that there is merit to testing and accountability, but undermines its own premise. If testing and accountability are plausible tools for achieving equality, leaving states’ testing regimes to random variability undermines equality. Rather than tracking a single proficiency standard as in the NCLB, the ESSA affords disadvantaged students educational opportunities that more closely track the approach of their home state rather than any mandate in statute. In this respect, the ESSA does little to continue the Elementary and Secondary Education Act’s historic mission to promote improvements in academic achievement for disadvantaged students."

It seems to me the overarching problem is that we don’t really know if the system is a statistical sleight of hand, a reflection of inadequate funding, or a reflection of socio-economic segregation (which is the biggest driver of student achievement).  And if we don’t know that, then these report cards cannot really tell us anything worth knowing—even if the underlying data and method is right on target.  In other words, transparency is worth its weight in gold.  Letter grades in the hands of parents and administrators who don’t know how to interpret them are more dangerous than pure ignorance.

Get the full Texas Tribune analysis below.

Texas has issued A-F grades for its school districts. Here's a look at what stands out.

By Aliyya Swaby, The Texas Tribune Aug. 24, 2018

"Texas has issued A-F grades for its school districts. Here's a look at what stands out." was first published by The Texas Tribune, a nonprofit, nonpartisan media organization that informs Texans — and engages with them — about public policy, politics, government and statewide issues.

Several days after Texas gave school districts their first official grades, school communities are still figuring out how to digest their report cards and interpret the results.

Even school superintendents, the biggest critics of the new A-F school ratings, are poring over the data — forced to balance their mistrust of the new system with a responsibility to score high by its metrics.

"I can walk and chew bubble gum at the same time," said John Kuhn, Mineral Wells ISD superintendent, whose North Texas district received an overall rating of C. "I can criticize a system that is patently unfair while also seeking to improve under that system. I do feel a great deal of pressure to move our score up."

State officials have argued that giving school districts grades instead of the previous pass/fail ratings, will give parents more information to choose the best educational options for their kids. But so far, there's a lot of disagreement about exactly what the data means.

Here are a few of the takeaways from this year's ratings:

1. The majority of Ds and Fs were given to districts with higher rates of economically disadvantaged or low-income students than the state average. No school district with a rate of low-income students lower than 30 percent received an overall rating of C, D or F. (The state calculates the percent of low-income students based on the number who receive free or reduced-price meals, which many agree is an inexact measure.)

Educators point to this data to argue that the new ratings benefit districts that already have resources and don't show how well school districts are performing despite the inherent challenges of educating students whose families are struggling.

Many argue the ratings are too heavily based on state standardized tests and therefore are unfair to schools enrolling mostly low-income students, who tend to perform lower on those tests.

"The ratings reflect a mixture of factors that are within the schools' control and that are without the schools' control," Kuhn said. "The state should not be allowed to underfund schools in a community and then slap on a failing score without acknowledging the state's role in not providing resources."

Loading...

"If you're looking at this B district and it really should have scored a C, there might be great things going on in that district," said Chris Jackson, accountability director at Cleburne ISD, a North Texas district that received an overall rating of C.

2. But it wasn't impossible for school districts enrolling higher rates of low-income students to do well. The majority of school districts in the state's South Texas region, which includes the Rio Grande Valley, received overall ratings of As and Bs, despite educating mostly students who are low-income and in need of additional resources to learn. State education officials have pointed to these districts to say that poverty has a "moderate" but not all-encompassing effect on the ratings.

Valley View ISD, at the border with Mexico, was the traditional, non-charter school district with the highest rate of low-income students to also receive an A from the state. Superintendent Rolando Ramirez attributes the high grade to the district's programs that fill in the gaps for students who don't have a lot at home: free meals during and after school, community events with free vaccines and physical exams, and free services for parents.

"We're trying to help parents and kids and alleviate, as much as we can, any expense they might have," he said.

However, in West Texas' Permian Basin region, most school districts received Cs, Ds and even one F — despite having generally lower rates of low-income students than the state average.

Wink-Loving ISD — a small rural district with two campuses and 400 students — scored that F, making it an outlier given its 30 percent poverty rate.

Loading...

"We're in the oil fields. Many of the kids who have been economically disadvantaged in the past, their parents have jobs now or have good-paying jobs," said Superintendent Scotty Carman. "In Wink, there's not much telling how many RVs we have in town. We have a lot of mobility, a lot of kids coming in from other states, from Mexico."

The district would have received a D, without a feature of the rating system critics are calling "forced failure." Any district that receives three out of 4 Fs in the subcategories that make up the overall grade automatically receives an overall grade of F — no matter its actual score.

3. Some school districts did better than others at paring down the number of low-performing schools, key to avoiding harsh state penalties. In recent years, Texas has implemented a system of strict sanctions for chronically low-performing schools, including state-mandated closure of a school or state takeover of a school district.

Fort Worth ISD and Dallas ISD both had 14 schools that got failing ratings last year. Fort Worth ISD boosted a couple of schools up to a passing rating. Dallas ISD whittled its list of low-performing schools down to 4, in part by giving stipends totaling up to $12,000 to high-quality teachers who choose to work in those struggling schools — a tactic state officials have expressed interest in replicating across the state.

Dallas ISD is struggling to fund the stipend program's successes, as more teachers turn around low-performing schools. "We're doing so great that teachers who are not getting results are moving out, the ones who are are improving, and the ones who are doing well are reaching the highest levels of pay," said Miguel Solis, board trustee.

Loading...

Administrators decided this year to scale back the program in part by decreasing the number of educators eligible for stipends.

4. There wasn't a major disparity in the results for charters and traditional school districts. Charters — public schools with more freedom from state regulations — and traditional districts often compete for students in urban and suburban areas of the state. The percentage of charters receiving overall A grades more than doubled the percentage for traditional districts. But both had a comparable level of As and Bs combined.

A larger percentage of charter districts scored overall ratings of Ds and Fs than traditional school districts. Laura Kelly, the Texas Charter Schools Association's director of quality, said that the percentage of low grades is amplified by the relatively small number of charter districts. Texas gave grades to 87 charter districts and 742 traditional school districts this year.

Unlike educators in traditional school districts, she said she doesn't see school funding issues playing a role in the distribution of this year's ratings.

"Traditional school districts struggle under limited resources the way charters struggle under limited resources," she said. "Do I think that plays out in the academic accountability system? I don't."

Read related Tribune coverage

• Texas school districts receive first official A-F grades. Look up how your district did here.
• Texas education officials promise focus on math, reading after “lagging” report card
• Harvey-affected superintendents want Texas to waive school ratings

This article originally appeared in The Texas Tribune at https://www.texastribune.org/2018/08/24/texas-school-districts-a-f-grades-takeaways/.

Texas Tribune mission statement

The Texas Tribune is a nonprofit, nonpartisan media organization that informs Texans — and engages with them — about public policy, politics, government and statewide issues.