New Study Shows Schools Led by Newark’s Largest Charter Operators Drove Significant and Lasting Gains in Student Test Scores

Students assigned to most of the charter schools in New Jersey’s largest city showed greater improvements on standardized test scores in math and English than they would if they were placed in district-run schools, according to a new study. The findings are bound to reignite debates over whether charters are a net gain for communities, which often pit teachers unions and their allies against charter operators and their defenders.

Newark has a complicated relationship with charter schools. Its current school district superintendent has publicly asked the state to close charters in the city, citing concern that they drain the district of needed funds. Today, about a third of the city’s students attend a charter, which are operated by non-district organizations. The contentious issue of school choice in the city is punctuated by a $100 million donation from Facebook’s Mark Zuckerberg that was faulted for pushing through controversial reforms without public input. Still, charters remain popular in the city, with nearly two-thirds of Newark voters holding a positive opinion of the city’s charter sector.

Marcus A. Winters, an associate professor at Boston University, produced the analysis for The Manhattan Institute, a think tank in New York City dedicated to promoting free-market ideas. Winters explained that the impact charter schools had on the students studied is “as big or bigger than 80 percent of all the other kinds of educational interventions that have ever been studied experimentally.” Unlike other studies that translate the test score gains in concrete examples such as months of extra learning, Winters’s study avoids that formulation. Doing so “requires so many assumptions that I’ve never been very comfortable with it,” he said.

The study takes advantage of a rare opportunity for social science research: being able to measure the effect a policy has on a group of people who were picked to benefit from it at random.

Newark, like several other cities, has a universal enrollment application for all of its district-run schools and most of its charter schools. Students and their families submit a ranking of their preferred schools, then an algorithm in the application uses those rankings plus a digital lottery to assign students to a school. Winters exploited that random element to match students who were demographically similar and who had virtually identical test scores. Winters analyzed the test scores of these matched students — some who went to charters and some who didn’t — to show the impact being assigned to a charter had on student performance.

Morgan Polikoff, an education professor at the University of Southern California who was not a part of this study, called the results compelling.

“This is a well-designed and -executed study using high-quality methods,” Polikoff said. “We can be confident that these are causal effects of Newark charter schools.”

He likened the effects to other studies, such as examining Boston schools and other urban districts, that showed similarly large gains for charter school students. A 2015 Stanford study also found that Boston and Newark charter schools had the largest impact on students out of 41 regions examined. (That study did translate the impact charters had on students into days of learning, finding that urban charters contributed a month or more of additional learning for students for every school year; critics called that approach “problematic.” The Stanford study also found numerous examples of students in traditional public schools outperforming those in charters.)

In Winters’s study, students assigned to the charter schools continued to yield higher test scores after two years, regardless of whether they remained in a charter school that was part of the analysis. That means students who enrolled in a charter school in the first year and entered a different type of school in the second or third year continued to perform at higher levels than they would had they initially been placed in a district-run school.

The study’s approach neutralizes some of the criticisms charter critics have lobbed at the sector, such as attracting students whose parents are more engaged. That students were assigned to a charter or not by the flip of a coin allows Winters to focus only on the impact of school placement, rather than outside variables. The study uses student data from the 2014-15 and 2015-16 school years, the first two years of the district’s common enrollment application, now called Newark Enrolls.

Not all of the district’s charter schools participated in Newark Enrolls. Winters’s data make causal claims about the 70 percent of the charters that took part in the common enrollment program. Even among those participating schools, there were variations in the impact they had on students. About half of the participating charters are operated by national charter groups KIPP and Uncommon Schools. When Winters compared the participating schools by whether they were run by those two operators, he found that on average, those that weren’t had virtually no significant impact on their students. In only one area of focus, first-year math scores, did those schools show to have a larger impact on students than district schools would.

“There’s probably some of them doing very well and some of them not doing very well,” Winters said. The data available to him limited his ability to test the individual impact of each school. Moreover, some of the schools in his sample have since shuttered.

Winters’s study only looks at whether charter schools improved student test scores — it doesn’t explain why. It also avoids other aspects of a school important to parents. The 74 analyzed federal suspension and expulsion data from the 2015-16 school year to compare all but two of the charter schools Winters examined to the schools run by the district. Although there was variation among the charter schools, overall their out-of-school suspension rate was 21 percent, compared with the 6 percent for Newark Public Schools. Twenty-two percent of the students in the KIPP Newark schools, labeled as TEAM in Winters’s data, were suspended that year. A fifth of the students at Newark’s Uncommon Schools, labeled as North Star Academy, received out-of-school suspensions.

Asked whether his research could inform the current debate around charters in Newark, Winters said it can, but his data can’t give particular insight into the four schools Newark Public Schools Superintendent Roger León singled out for closure in the final week of 2019. Assuming that it’s safe to close all the charter schools unaffiliated with KIPP or Uncommon is also foolhardy, because while those other charters didn’t show meaningful gains as a group, some likely did result in student improvement.

“If it’s the case that some of those schools are effective, you would be harming those kids,” Winters said.

Winters has studied charter schools from different angles. He’s looked at why charter schools in Denver enroll fewer students with disabilities, and those in New York City enroll fewer English language learners, than do public schools. As for whether charters, which have been in the U.S. for three decades, have fulfilled their promise of improving student outcomes nationwide, the data are mixed. Charters aren’t having the free-market effect their proponents said they would on all schools by forcing them to improve through competition.

“I think it’s fair to say we haven’t seen that,” Winters said. “We’ve seen the public schools get a little bit better in some places, but nowhere near kind of the claims that advocates were hoping for. But we also definitely haven’t seen public school systems get dramatically worse because of choice, either.”

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter