New evidence on the domestic policy influence of global performance assessments

Using a multilevel linear model to account for the hierarchical structure of our survey data, we find evidence that performance assessments yield greater policy influence when they make an explicit comparison of government performance across countries and allow assessed governments to participate in the assessment process. This finding is robust to a variety of tests, including country-fixed and respondent-fixed effects.

Whoa. That’s from a new AIDDATA working paper on global performance assessments (international measures of how well countries do at combating corruption, ensuing fair elections, opening data, or what have you).

Those findings aren’t shocking (that ranking countries can motivate govt actors in crude ways has become almost as much a platitude as the idea that participatory research enhances uptake), but their exciting because they are so clearly and directly relevant to the design of comparative assessments, and this study “feels” robust enough to carry some weight in conversations with the people who manage the  budgets and optics of such projects. Based on econometric analysis of data from the 2014 Reform Efforts Survey (n= 3,400 gov officials in 123 low and middle-income countries) and 103 different GPAs, the use of elite survey data is a smart way to get around the problems of measuring influence by policy outcome.

The treatment of GPA characteristics is a bit crude though. 8 independent variables are identified at the GPA level (bi-/multilateral, whether govt’s are involved, whether data is public, etc), but none of these address policy areas or types of norms. Dan Honig’s research has suggested how important this can be for mobilizing the soft power of GPAs, and my own work on OGP suggests that it’s critical.

research links w 14-17

The weeds are deep in this one.


All the findings: @3ieNews‏ has mapped out existing evidence on citizen-state relations, put together a linked matrix organized according to the interventions and outcomes measured, plus confidence levels. It includes “18 completed systematic reviews and two systematic review protocols, 305 completed impact evaluations reported in 280 papers, 60 ongoing impact evaluations reported in 59 papers.” And everything is linked. And it’s not ugly. Swoon. h/t @OpenGovHub 

Meanwhile, @bbcmediaaction ‏ blogs on their new data portal, which collects survey data on “rarely polled” people in 13 developing countries, providing insights on media use, governance and freedom of expression perspectives. Continue reading “research links w 14-17”