Education Lab: Using tutoring to reverse pandemic-era learning loss

Editor’s Note: This is part of a series called Inside the Lab, which gives audiences a first-hand look at the research laboratories at the University of Chicago and the scholars who are tackling some of the world’s most complex problems.

Discover more from this series:

COVID-19 created a once-a-century public health crisis that, in turn, created a once-a-century public education crisis. A total of 50 million students nationwide lost the learning equivalent of a half a year of school. The learning losses were even larger—sometimes much larger—among the nation’s most socially and economically disadvantaged children, particularly in low-income communities of color.

In contrast to the rapid progress the country has made addressing the public health crisis, the education crisis has been slower to resolve. Left unaddressed, this ongoing crisis may have massive consequences for both economic mobility and overall economic growth and competitiveness.

“As a country and as a city, we need to have our hair on fire as much about the learning loss problem as we had about the COVID problem because the stakes are arguably just as big,” said Jens Ludwig, an economist and the Edwin A. and Betty L. Bergman Distinguished Service Professor at the University of Chicago’s Harris School of Public Policy.

Ludwig serves as the Pritzker Faculty Co-Director of the University of Chicago Education Lab. The lab uses the tools of social science and data science to help set students up for success, as well as the public school systems that serve them. It has partnered with Chicago Public Schools and districts around the country on finding ways to help students overcome pandemic-era learning loss by using intensive tutoring. Findings show that individualized instruction, one tutor to two students, several days a week during school can double, even triple, what students learn in a year.

To better understand the impact of the Education Lab’s work, we spoke with Jens Ludwig; Monica Bhatt, the Education Lab’s Senior Research Director; and Sadie Stockdale Jefferson, Executive Director of the Lab.

What does the Education Lab do?

Jefferson: Of course, there are many challenges to education that are easy to see, including poverty, inequality, and a school funding system that relies too heavily on local property taxes. That, in turn, means that school systems serving disproportionately low-income student populations, the ones that need resources the most—like those in Chicago—are the least well-positioned to raise the necessary funds. But there is also a big problem that’s harder to see—the challenge of identifying those parts of the system that aren’t performing optimally and finding ways to improve them.

Ludwig: The U.S. spends $800 billion a year on K-12 education, with far too little research on how to ensure that funding does as much social good as possible. Our public schools are, understandably, designed to and fully absorbed with the task of running this massive system day to day. To improve education in America, public schools need a research partner. This is what the Education Lab was set up to do.

What does this look like in practice?

Bhatt: You need lots of data and ways to make sense of those data. It’s our window into understanding what’s happening on the ground. Data science tools like machine learning are our way of understanding which children and schools are on a trajectory for future success and which ones need more support. And social science tools like causal inference econometrics is our way of understanding which policies and programs are helpful to students and their families and which aren’t.

One of our inspirations was carried out over a decade ago by the University of Chicago Consortium on School Research. They identified a handful of ninth-grade performance indicators that were highly predictive of successful high school graduation, created a “ninth grade on track” dashboard to help schools see which students were on track and which weren’t, and helped the Chicago Public Schools (CPS) deploy that at scale. While the research on the causal effects of this intervention is ongoing by our UChicago colleagues, there are initial signs that this may have been remarkably effective at improving student outcomes. That helped expand the imaginations of researchers everywhere about how data can create impact at large scale and got us to appreciate the collection of great public sector partners here in Chicago.

How receptive have your partners been to this sort of work?

Ludwig: They’ve been surprisingly receptive. For instance, back in 2008, when Richard M. Daley was mayor of Chicago, the city was really worried about the problem of youth violence and what strategies we could use to address that problem by preventing shootings and other violence in the first place rather than just relying purely on “get tough” approaches.

Our own research team partnered up with City Hall and CPS to raise $1 million from local philanthropy to pilot a program called Becoming a Man (BAM) created by the local nonprofit Youth Guidance. This project showed that BAM reduced violent crimes among participants by 50% and generated $5 to $30 in social benefits for every dollar spent. Seeing the success, Mayor Daley’s successors, Mayors Emanuel and Lightfoot, collectively invested $60 million into similar programs, using both city funds and other donor contributions. By helping redirect government spending toward evidence-based uses, this public-private partnership was able to turn that initial $1 million investment into potentially hundreds of millions of dollars of social impact.

Bhatt: The success of BAM in our initial studies and the success in getting the city to scale it then created its own challenge. As BAM got bigger, the effectiveness of the program may have become less consistent. The data are less clear than we would like, but there are some signs of that happening, which is common throughout social policy. This problem of scale is a widespread challenge—and one we are working intensely to help solve.

How have you been working with school systems to solve pandemic learning loss?

Bhatt: Even before the pandemic, we had been working with CPS to solve the problem of what you might call “academic mismatch”: teachers assigned to, say, a fifth-grade classroom are asked by the school to teach fifth-grade material. But the average fifth-grade class contains students working at a wide range of academic levels, from fifth grade (or even higher) to third grade (and sometimes even lower). That makes it really hard for students who are behind grade level to benefit from regular classroom instruction. This is one of the hardest parts of teaching.

As Steve Raudenbush, one of our wonderful colleagues at the University of Chicago’s Committee on Education, put it: “Dealing with heterogeneity is the problem of education.” The only way to ensure that every child, no matter their background or academic level, is really able to fully benefit from classroom instruction is to build a system that catches students back up to grade level whenever they fall behind. We’ve been trying to test and build that sort of academic safety-net system using bursts of intensive tutoring in partnership with CPS and nonprofit Saga Education.

Ludwig: The pandemic is like the 100-year flood for education that has greatly exacerbated the academic mismatch problem. The number of students who are behind grade level and need to catch up so they can fully benefit from classroom instruction has exploded. We can see that these bursts of tutoring can double or triple the amount students learn in a year, which can catch them up quickly. The challenge is how to do that at lowest possible cost so we can serve as many students as possible.

How has the lab started to tackle this problem of scale?

Bhatt: A new study just released by the Education Lab tries to solve the challenge of scaling up these benefits to as many children as possible by trying to combine technology and tutoring in different ways. Our new working paper shows that it’s possible to reduce costs by one-third and halve the number of tutors we need without compromising effectiveness. You rarely see this in public policy—ways to lower costs and increase scalability without changing program impacts. It’s like the proverbial $20 bill just sitting there on the sidewalk.

Those findings helped inspire a big national push to figure out how to do this not just in Chicago, but also in Fulton County, Georgia; Miami-Dade; Greenville, South Carolina; and the state of New Mexico—what we call the Personalized Learning Initiative. Preliminary results from the first year of the initiative of approximately 2,000 students show that in-school high-dosage tutoring can work, even when delivered in the aftermath of the pandemic and in diverse academic settings–especially in math.