Prof. Kerwin Charles ticks through new ways researchers can bring evidence to public policymaking, from tapping voluminous administrative data sets to deploying machine learning.
Such innovations provide surprises that have long driven the research of the Edwin and Betty L. Bergman Distinguished Service Professor and his colleagues at the University of Chicago Harris School of Public Policy, creating new opportunities for better policymaking.
While the capacity to produce evidence grows, Charles, who became interim dean of Chicago Harris in July 2016, knows the school’s rigorous, empirical approach often delivers complexities and new questions. One response is pushback against an evidence-based approach that can be seen from confronting global climate change to the U.S. presidential race.
Charles recently spoke with UChicago News about the rewards—and challenges—of evidence-based policymaking. The conversation followed “The Future of Harris,” a recent event highlighting the school’s approach to public policymaking and its future home at the Keller Center.
How does evidence inform policymaking in the current political climate?
We’re in this moment where the capacity to produce convincing evidence has perhaps never been better. We have more data of more types, more widely available than ever before. That would lead you to suppose that evidence matters more in policymaking than ever before.
However, we see in some areas such as climate change that people are either unaware of the best evidence that might be brought to bear on a policy question, are willfully dismissive of that evidence, or cannot sift through the different findings available.
When we don’t turn to evidence, what happens to policymaking?
It reduces its quality. If one can proceed in the absence of evidence, it becomes easier to do things the evidence may show to be unwise because one is not constrained by it. The problem with evidence is that while it leads us to a truth, it also often suggests the path to good policymaking is more complicated, more nuanced, more frustrating than policymakers presumed.
A consistent theme to my empirical research—and to the research of many people at Harris—is the degree to which evidence surprises you. One is surprised nearly always by the magnitude, or that the effects are there for some groups but not others. There is also the constant reminder as a scholar of the incompleteness of evidence: What we have is the best evidence today given prevailing methods, data and understandings. We are always in this process of relentlessly improving, chipping away at misunderstandings, tightening the standard error. The thing we think of as research—at the University of Chicago, generally, and Harris, particularly—is not about finding the one immutable truth, but about refining the understanding that now prevails.
Can you explain how the capacity to produce good evidence is better now than ever before?
One big innovation is the use of administrative data. These data are not from a select sample of private individuals who were surveyed. Instead, every time, for example, the government mails a check, a box is ticked off somewhere, eventually producing a massive data set. The promise of administrative data is that one could have every record of every payment the Social Security Administration made. I am more and more amazed by the richness such data brings to answering a question—even an old question—by social scientists at Harris and elsewhere, but particularly at Harris.
We are seeing new tools emerge such as machine learning. These tools, which were not at the disposal of social scientists five or 10 years ago, are an exciting area to think about: How administrative data can be combined with tools from computer science and data analytics.
One final area is that scholars increasingly are engaged in collecting their own data, rather than receiving it from an outside source like the government. This becomes a more and more prevalent feature of serious scholarly work. So if I think about our recent faculty hires for The Pearson Institute for the Study and Resolution of Global Conflicts—Profs. James Robinson, Chris Blattman and Oeindrila Dube—these scholars go to developing countries and conduct surveys, and the surveys are designed to elicit information about things that are simply not present in your typical data set—administrative or otherwise. This is a big innovation, and it will have massive consequences on the kind of work we do and its usefulness to policymaking.
In your role as interim dean, how do you ensure Harris has a distinctive point of view on public policy that’s rooted in science, data and evidence?
One of the biggest parts of my job is consistently articulating what is distinctive about us. Faculty understand this already, but I must set the tone for students and people we engage with externally. Because there are pressures to be different. There are pressures to be not as rigorous. Pressures to be not as relentlessly serious and skeptical and careful. But if from the dean it is understood that’s who we are, then the faculty take it from there.
—Prof. Jacqueline Stewart
