Thursday, February 21, 2013

Point Estimates and Division of Labor vs. Technocracy

In his recent book, Public Policy in an Uncertain World, Charles F. Manski notes
Modern democratic societies have created an institutional separation between policy analysis and decisions, with professional analysts reporting findings to representative governments. Separation of the tasks of analysis and decision-making, the former aiming to inform the latter, appears advantageous from the perspective of division of labor...
However, the current practice of policy analysis does not serve decision makers well. The problem is that consumers of policy analysis cannot trust the producers...I recommend that journalists reporting on policy analysis should assess whether and how researchers express uncertainty about their findings, and should be deeply skeptical of studies that assert certitude. That caution and advice extend to all readers of policy analysis. (pp. 173-174)
The rest of the book is quite good, and makes the case for expressing social science results in confidence intervals rather than point estimates, which give (at best) a feeling of false precision. An example of this methodology in practice can be found in Manski's paper on recidivism with Daniel Nagin (1998).

Returning to the quote above, an implication seems to be that technocracy (having the analysts make the decisions in addition to doing the analysis) might be more desirable than representative democracy. It would foster more accountability on both ends - politicians can't blame analysts for policy mistakes after the fact, and vice-versa - and I'd speculate that the benefits of division of labor in politics are a bit overstated.

However, it's hard to imagine the personality types represented by statisticians and actuaries as running successful political campaigns, so division of labor is what we've got, like it or not. Manski's caution about the false precision of point estimates might be a step in the right direction though. For example, the CBO would be a lot more credible if they gave estimates in confidence intervals, instead of specific estimates that are consistently wrong.

No comments:

Post a Comment