For the budding social scientists out there who hope that their research will someday positively affect public policy, my colleague Jerry Ellig recently pointed out a 2012 publication from the National Research Council called “Using Science as Evidence in Public Policy.” (It takes a few clicks to download, but you can get it for free).
From the intro, the council’s goal was:
[T]o review the knowledge utilization and other relevant literature to assess what is known about how social science knowledge is used in policy making . . . [and] to develop a framework for further research that can improve the use of social science knowledge in policy making.
The authors conclude that, while “knowledge from all the sciences is relevant to policy choices,” it is difficult to explain exactly how that knowledge is used in the public policy sphere. They go on to develop a framework for research on how science is used. The entire report is interesting, especially if you care about using science as evidence in public policy, and doubly so if you are a Ph.D. student or recently minted Ph.D. I particularly liked the stark recognition of the fact that political actors will consider their own agendas (i.e., re-election) and values (i.e., the values most likely to help in a re-election bid) regardless of scientific evidence. That’s not a hopeless statement, though – there’s still room for science to influence policy, but, as public choice scholars have pointed out for decades, the government is run by people who will, on average, rationally act in their own self-interest. Here are another couple of lines to that point:
Holding to a sharp, a priori distinction between science and politics is nonsense if the goal is to develop an understanding of the use of science in public policy. Policy making, far from being a sphere in which science can be neatly separated from politics, is a sphere in which they necessarily come together… Our position is that the use of [scientific] evidence or adoption of that [evidence-based] policy cannot be studied without also considering politics and values.
One thing in particular stands out to anyone who has worked on the economic analysis of regulations. The introduction to this report includes this summary of science’s role in policy:
Science has five tasks related to policy:
(1) identify problems, such as endangered species, obesity, unemployment, and vulnerability to natural disasters or terrorist acts;
(2) measure their magnitude and seriousness;
(3) review alternative policy interventions;
(4) systematically assess the likely consequences of particular policy actions—intended and unintended, desired and unwanted; and
(5) evaluate what, in fact, results from policy.
This sounds almost exactly like the process of performing an economic analysis of a regulation, at least when it’s done well (if you want to know well agencies actually perform regulatory analysis, read this, and for how well they actually use the analysis in decision-making, read this). Executive Order 12866, issued by President Bill Clinton in 1993, instructs federal executive agencies on the role of analysis in creating regulations, including each of the following instructions. Below I’ve slightly rearranged some excerpts and slightly paraphrased other parts from Executive Order 12866, and I have added in the bold numbers to map these instructions back to summary of science’s role quoted above. (For the admin law wonks, I’ve noted the exact section and paragraph of the Executive Order that each element is contained in.):
(1) Each agency shall identify the problem that it intends to address (including, where applicable, the failures of private markets or public institutions that warrant new agency action). [Section 1(b)(1)]
(2) Each agency shall assess the significance of that problem. [Section 1(b)(1)]
(3) Each agency shall identify and assess available alternatives to direct regulation, including providing economic incentives to encourage the desired behavior, such as user fees or marketable permits, or providing information upon which choices can be made by the public. Each agency shall identify and assess alternative forms of regulation. [Section 1(b)(3) and Section 1(b)(8)]
(4) When an agency determines that a regulation is the best available method of achieving the regulatory objective, it shall design its regulations in the most cost-effective manner to achieve the regulatory objective. In doing so, each agency shall consider incentives for innovation, consistency, predictability, the costs of enforcement and compliance (to the government, regulated entities, and the public), flexibility, distributive impacts, and equity. [Section 1(b)(5)]
(5) Each agency shall periodically review its existing significant regulations to determine whether any such regulations should be modified or eliminated so as to make the agency’s regulatory program more effective in achieving the regulatory objectives, less burdensome, or in greater alignment with the President’s priorities and the principles set forth in this Executive order. [Section 5(a)]
OMB’s Circular A-4—the instruction guide for government economists tasked with analyzing regulatory impacts—similarly directs economists to include three basic elements in their regulatory analyses (again, the bold numbers are mine to help map these elements back to the summary of science’s role):
(1 & 2) a statement of the need for the proposed action,
(3) an examination of alternative approaches, and
(4) an evaluation of the benefits and costs—quantitative and qualitative—of the proposed action and the main alternatives identified by the analysis.
The statement of the need for proposed action is equivalent to the first (identifying problems) and second tasks (measuring their magnitude and seriousness) from NRC report. The examination of alternative approaches and evaluation of the benefits and costs of the possible alternatives are equivalent to tasks 3 (review alternative policy interventions) and 4 (assess the likely consequences).
It’s also noteworthy that the NRC points out the importance of measuring the magnitude and seriousness of problems. A lot of public time and money gets spent trying to fix problems that are not widespread or systemic. There may be better ways to use those resources. Evaluating the seriousness of problems allows a prioritization of limited resources.
Finally, I want to point out how this parallels a project here at Mercatus. Not coincidentally, the statement of science’s role in policy reads like the grading criteria of the Mercatus Regulatory Report Card, which are:
1. Systemic Problem: How well does the analysis identify and demonstrate the existence of a market failure or other systemic problem the regulation is supposed to solve?
2. Alternatives: How well does the analysis assess the effectiveness of alternative approaches?
3. Benefits (or other Outcomes): How well does the analysis identify the benefits or other desired outcomes and demonstrate that the regulation will achieve them?
4. Costs: How well does the analysis assess costs?
5. Use of Analysis: Does the proposed rule or the RIA present evidence that the agency used the Regulatory Impact Analysis in any decisions?
6. Cognizance of Net Benefits: Did the agency maximize net benefits or explain why it chose another alternative?
The big difference is that the Report Card contains elements that emphasize measuring whether the analysis is actually used – bringing us back to the original goal of the research council – to determine “how social science knowledge is used in policy making.”