Methodical Snark critical reflections on how we measure and assess civic tech

When do global do-gooders influence government behavior? (a mini lit review)

W

More specifically:  when and how do international mechanisms that promote norms on governance and transparency (like the OGP, the EITI, and the IATI) actually influence government policy or programming? What does the research literature say about this, and about how and when we can attribute changes in policy and behavior to global do-gooders?

Here’s a long ranging exploration of the literature on international relations, policy diffusion, public administration, global policy assessments and multi-stakeholder initiatives, where I try to draw some conclusions about what we know and what we don’t. I wrap it up by proposing six research questions that could directly inform the design of global do-goodery.

There’s references at the end, a bulleted summary up top. As always, let me know if I’m wrong or if you need help accessing literature.

TL;DR:

  • Early insights from IR scholars about norms seem to hold, particularly regarding the importance of domestic political context and space for political contestation.
  • Contemporary norms are messier and more ambiguous though, which is likely part of the reason the literature is so fragmented and dominated by deep case studies.
  • Ambiguity can help get countries to sign up to norms, but can get the way of actual adoption and implementation.
  • Rankings seem to work. They influence government behavior by activating identity and competition at both a country-level, and within government structures.
  • Non-ranking, non-numerical mechanisms like the OGP haven’t been studied in the same way, so there isn’t similar evidence that they are effective.
  • There is significant evidence that international norm entrepreneurship is useful for opening up civic spaces to debate policy and reform, but it is less clear how meaningful these spaces are for influencing government behavior.
  • Multi-stakeholder initiatives are better at improving pro-active than reactive transparency, and there’s more evidence about the things that can go wrong than any impact they’re having.
  • Contemporary critical thinking among global do-gooders seems to be a step behind of research on numerical rankings and a step ahead of MSI –style initiatives. I propose 6 research questions that would directly inform smart design of global do-goodery.

In the beginning there were human rights norm entrepreneurs

There’s not a lot of contemporary research on this. So to start, we need to take a step back. The seminal piece of scholarly work on norms and government behavior is Finnemore and Sikkink’s (1998) International Norm Dynamics and Political Change. This book was notable for breaking through a research literature otherwise dominated by international relations realists, who argued that governments only changed behavior when forced to (ie: by signing up to treaties or through pressure in trade arenas). Finnemore and Sikkink were at the crest of a scholarly wave interested in the power of norms for opening up spaces of domestic contestation. This body of work emphasized the role of “norm entrepreneurs” who move between international and national policy arenas, promoting and negotiating norms, and ushered in a whole new wave of social constructivist attention to the influence of norms on government behavior.

Much of this constructivist turn is focused on human rights norms, which is a useful object of study because the norms are very clear. A norm on the prohibition on torture is spelled out precisely in international treaties, making it easy to study compliance, observance and instances of advocacy. There is generally broad agreement that governments endorse human rights norms in response to social pressure and in order to improve international standing (though the latter seems not to be rewarded in the human rights context (Nielsen and Simmons, 2015)). The role of international organizations in these processes have been well documented (Finnemore, 1993) and Risse-Kappen et al.’s (1999) boomerang model demonstrates the importance of interaction between national and international advocacy communities in motivating government behavior change through a series of case studies. Most usefully, this book mobilizes the idea of “rhetorical entrapment” –  that getting governments to agree on the principled value of a norm is the first step in getting them to observe it. This is subject to disagreement, but getting governments to legislate norms certainly seems to entrap behavior. There’s pretty clear empirical evidence showing that ratification of human rights treaties leads to better rights observance domestically (Simmons, 2009).

Context matters

The social constructivist literature broadly agrees that global norms only have an impact after being worked through domestic policy debates, political cultures, institutional legacies, logics of appropriateness, and power struggles (Checkel, 1997). This has been documented convincingly in the literature on norms entrepreneurship (Cortell and Davis, 2002) and in studies of policy transfer (Johnson and Hagström, 2005; Park et al., 2016; Stone, 2004). Put simply, international norms never simply get taken up, they get translated into national contexts and reshaped according to preferences and power struggles that dominate those contexts.

Reviewing literature on international norms, Cortell and Davis (2002) note the primary mechanisms through domestic context flexes its muscles:

“Two national-level factors have been shown to condition the effects of international norms on domestic political processes and provide explanations for important cross-national variation in compliance with and interpretation of international norms: the domestic salience or legitimacy of the norm, and the structural context within which the domestic policy debate transpires” (66).

This distinction between national policy structures and underlying belief systems resonates in the international management literature (Common, 2013), and has been empirically demonstrated in studies of EU policy fit (Mastenbroek and Kaeding, 2006). It’s commonsensical to anyone doing international policy advocacy, and is part of why context analysis is so important for international governance programming (T. G. Falleti and Lynch, 2009; Tulia G. Falleti and Lynch, 2009) . In the transparency and accountability community, these dynamics have received renewed attention with an emphasis on thinking politically in program design and attention to complex systems in program contexts (Fox et al., 2016). As you’ll hear repeated at every international civic technology context ever: “context matters.” Why and how, we’re not so sure.

Research on contemporary do-gooders

There’s at least three ways in which many contemporary mechanisms pushing global good governance norms are different than the human rights norm entrepreneurship.

1. Norm structure (interpretive ambiguity and hierarchies)

Literature on contemporary global norms for open government, transparency and good governance note that those norms are marked by a complex and ambiguous structure, and by the inclusiveness and relational quality of the social hierarchies they engender.

Firstly, contemporary do-gooder norms not as clean and simple as human rights norms. The broad ways in which “open” and open government have been interpreted (Francoli and Clarke, 2014; Pomerantz and Peek, 2017) demonstrate how important it is to follow Clara Winston (2017) in thinking about “norm clusters” that are both stable and flexible, open to strategic and instrumental engagement.

The rhetoric of governance norms has received only passing attention in research, and rhetorical ambiguity is often seen as a way to broaden the appeal of norms (Goldstein and Weinstein, 2012). Of course it’s more complicated than this.

De Blasio et al. (2016) and Gonzalez-Zapata and Heeks (2014) have assessed how different rhetorical dimensions of open government motivate different stakeholder groups in different policy contexts. Implicitly, this reinforces attention to ideational spaces for debating and translating norms, and the key roles of policy-makers for defining that space. Gilardi (2015) demonstrated the importance of policy makers “ideological positions and prior beliefs” in determining whether they take account of the experience of others in policy adoption, and policy scholars have emphasized policy translation processes as sites of contestation between the interests and perspectives of different stakeholder groups (Johnson and Hagström, 2005).

This focus on contestation supports a social constructivist attention to domestic salience and legitimacy of norms. It adds a little bit of detail to theories of how norms get empowered in local policy contexts, but doesn’t provide close investigation of how that happens. After all, it’s not a huge surprise to find that once translated into national contexts, global norms tend to resemble the dominant political ideologies. The pressing question for do-gooders is how this happens, and how external sources of rhetorical credibility (either from national civil society or global actors) influence that process. These dynamics are pursued in every OGP consultation process, but I haven’t found any within-case analysis in relevant literature.

Another approach to understanding global norms and their effects on government behavior turns on social hierarchies, which structure analyses of international human rights politics (Terman and Voeten, 2017) and policy diffusion (Dobbin et al., 2007). Reviewing international relations and political science literature on international norms, Towns and Rumelili (2017) argue that governments are motivated by the social hierarchies that norms activate. In doing so, they distinguish between hierarchies based on absolute  or relative standards (rankings vs non-ranking governance indices), and between “heterogenizing norms [that] rely heavily and expressly upon difference [and] homogenizing norms [that] order actors by positioning them within a single social category and validating their shared identity” (14). These distinctions are drawn from several empirical examples, and used to construct a four quadrant matrix.

Again, I haven’t found research exploring how these distinctions are manifest, but it makes intuitive sense that these kinds of design decisions matter for international advocacy mechanisms. Muti stakeholder initiatives like the OGP and the EITI are driven by norms that are absolute and homogenizing , so would presumably fall into the fourth quadrant. Towns and Rumelili note that this type of hierarchy is marked by a less competitive dynamic than hierarchies built on relative norms.

2. Data and numbers

Contemporary governance advocacy is marked by data and digitization at the global level just as much as it is at the national level. This has resulted in a proliferation of comparative assessment initiatives over the last couple of decades, and most of these tend to use computational and numerical methods. There are a lot of rankings out there, and a lot of evidence that rankings motivate behavior changes.   

There is broad sociological agreement that numerical and comparative assessments exert a social force on behavior (Espeland and Sauder, 2007; Sauder and Espeland, 2009), and these frameworks have been convincingly applied to government behavior (Kelley, 2017; Kelley and Simmons, 2015). Empirical work has demonstrated ranking effects in the context of international governance assessments such as the World Bank’s Ease of Doing Business corruption index (Kelley et al., 2016), the Aid Transparency Index (Honig and Weaver, 2017).

There’s a few different ways to address this literature. For many theorists, it is the inherent power and credibility of numbers that drives social dynamics and behavior change. change in such dynamics (Broome and Quirk, 2015). Numerical clarity is, moreover, the most consistently cited factor in explaining state behavior change in response to global performance assessments. It is difficult to distinguish this effect from the effect of precision in policy recommendations and assessment metrics, however, as exemplified by the clear perceived utility of IATI as a “policy blueprint” in Honig and Weaver’s analysis.

Emphasis on (numerical) precision is also problematic from the policy advocate perspective. Michener (2015) argues convincingly that comparative and numerical global assessments can incentivize box checking rather than actual norm adoption. More importantly, it’s difficult to disentangle the effects of precise policy and assessment from the social dynamics underpinning them. As described in Honig and Weaver’s study of IATI (2017):

Normative power is enhanced by [global performance assessments’] productive power, defined in the sociological literature as the ability to define “good performance” through commensuration – the quantification, or simplification, of evaluative aspects of actor behavior that enables categorization, rating and relative peer ranking that would otherwise be too complex or incomparable (8).

This emphasis on the social utility of precise measures is consistently emphasized in the literature on global performance assessments, and also underpins the dynamics of competition and emulation that are understood to drive policy diffusion across states (Dobbin et al., 2007; Gerven et al., 2014).

I have not, however, found analyses that disentangle these dynamics.

This is especially important because the ambiguity of governance norms and a preference towards country-driven assessments have resulted in an increasing number of global normative and assessment mechanisms that do not incorporate precise or comparative metrics. There is lots of evidence that global normative mechanisms using numerical, comparative assessments and precise measures lead to government behavior change, and no hard evidence for mechanisms like the OGP and the EITI that do incorporate have these aspects. But this this might simply be because the latter type of mechanism hasn’t been sufficiently studied.

Lastly, it’s important to note that inter-governmental social dynamics have also been  well documented at the subnational level (Park et al., 2014; Pinheiro et al., 2017; Porto de Oliveira, n.d.). Honig and Weaver find behavior change motivated by competition and identity relationships between national governments, between institutions, and between individuals within institutions. They also find that these social mechanisms operate in direct response to the relationships activated by international normative mechanisms. Institutional rivalry, for example, “appears to especially resonate in countries where there are multiple aid agencies who are vying for favorable positions in the eyes of common principals” (18). Similar dynamics are documented in studies of World Bank assessments and compliance with EU directives.

3. Multi-stakeholderism

Perhaps the most notable difference between human rights entrepreneurship and contemporary do-goodery is the increasing dominance of multi-stakeholder logics. Multi-stakeholder initiatives (MSIs) such as the EITI and OGP hardwire the interaction and participation of multiple sectors in order to force interaction between governments and civil society actors in the translation of global good governance norms, believing that doing so puts checks and balances on interests that compete to direct norm translation, and that it this is  a more effective mechanism for creating civic spaces for reform (Guerzovich and Moses, 2016).

This approach is increasingly common, and a report from the Global Development Incubator found that a “conservative, non-exhaustive count shows a more than fourfold increase in these types of efforts between 2000 and 2015 alone” (Stern et al., 2015).

Though there is little research on MSI’s influence on government behavior,  incentives for membership in MSIs have gained some attention, with compelling insights. David-Barrett and Okamura (2016) note how social mechanisms and norm structure feed into country decisions to join the EITI, and find that that developing countries join the initiative in order to improve their reputations, and in contrast to evidence on human rights treaty ratification, are rewarded for doing so.

Systematic analysis of MSI impact on government behavior is sparse. Complementing  a handful of OGP unstructured country case studies in the grey literature (Guerzovich and Moses, 2016; Montero, 2015a, 2015b; Montero and Taxell, 2015; Schneider, 2015), Brockmyer and Fox’s (2015) report for the Transparency and Accountability Initiative and Brockmeyer’s related and more comprehensive doctoral dissertation are likely the most explicit and systematic effort.

Brockmeyer’s literature review outlines the ways in which MSIs might contribute to improving governance by setting standards, altering perceived legitimacy, applying political pressure, and through information empowerment. His empirical analysis finds some evidence to suggest that these mechanisms have led to pro-active government transparency reform, but not to reactive reform (in response to civil society demands) or meaningful accountability. Though the causal mechanisms through which this occurs are not clearly identified, Brockmeyer’s fuzzy set qualitative comparative analysis of 9 MSIs identifies 3 “pathways” to proactive transparency reform. These pathways are defined by contextual factors and reminiscent of Kosack and Fung’s (2014) typology of the “worlds” in which accountability activism is pursued. Key enabling conditions for the three pathways include regular evaluations, visible political support and capable civil society.

Brockmeyer and Fox’s condensed report (87 pgs) assess five MSIs (EITI, OGP, CoST, GIFT, and OGP), noting differences in structure and implementation, and generally finding a lack of evidence on which to assess their effectiveness. Despite this, they offer specific recommendations for structuring interaction between stakeholder groups, which are generally supported by some of the literature referenced above. Two are particularly noteworthy.

Brockmeyer and Fox recommend developing broad coalitions of government reformers with ample opportunities for interaction across levels and branches of government. This resonates with the dynamics of competition and emulation described above, and the potential to strengthen this dynamic by increasing interactions. This is in line with Parks and Masaki’s (2017) finding  that when countries get to participate in their own assessments it increases the perception of benefits in terms of credibility signaling to third parties, and encourages policy makers to adjust behavior.

Brockmeyer and Fox also recommend “more sophisticated monitoring and evaluation practices that establish coherent links between their strategic objectives, their activities, and their medium and long-term metrics of progress” (62), which recalls the importance of clarity and precision in normative standards and assessment metrics.

Lastly, it’s worth noting that the fundamental logic of the MSI model resonates strongly with many of the findings on more numerically and comparatively structured assessment models. Structuring a civic space in which government, civil society and global norm entrepreneurs are forced to actively participate in the translation of global good governance norms emphasizes the social power of that interaction. It is in this sense that Towns and Rumelili (2017) suggest that numerical assessments have the greatest motivational force when they speak directly to national identities, and Honig and Weaver note that incentives are created in the domestic actors shared perspectives in regard to international actors.

Making sense of it

The landscape for global norm entrepreneurship on governance has changed significantly in recent decades; but the fundamental insights of social constructivists and IR scholars appear to hold. Here’s some key insights, with thoughts on how they should be considered in contemporary practice.

Domestic context and the structure of policy discourse

IR and social constructivists and norm theorists argued that domestic structures (institutions, laws, formal power balances) and the domestic salience of norms are the two primary ways in which global norms get translated into national contexts. Subsequent research has emphasized the discourses and processes of contestation in which this takes place, and suggests that the design of civic spaces in which this takes place has important consequences for the influence of global norms on government behavior.

While multi-stakeholder fora are quite popular in international practice, there is little evidence that this is more or less effective in motivating policy change.  The devil is likely in the details, having to do with the credibility and resonance of particular stakeholder positions in specific national contexts. Though if well designed, social constructivist scholarship gives good reason to think that MSI fora can be powerful mechanisms for civil society to assert legitimacy in normative contestation.

There is better evidence for the importance of policy precision and interactions beyond what is traditionally understood as domestic MSI constellations. Policy precision and comparative assessments are both shown to have a significant social and motivational force in a variety of contexts. This is closely related to the potential to negotiate policy and assessments in a way that shapes expectations and plans of action. Compellingly, the best evidence for this having a concrete influence on behavior is when those interactions take place within government or between government and international norm entrepreneurs, not between national government and civil society actors. This doesn’t suggest that traditional MSI structures aren’t valuable, but that it might be worth considering broader channels of communication and more precise policy measures in their design.

Ambiguity is an opportunity and a trap

The inherent ambiguity of contemporary norms like “open government” or “transparency” can be useful to motivate norm adoption by states. This has been demonstrated in the case of EITI membership, and can explain the wild popularity of initiatives like the OGP. If one subscribes to the notion that rhetorical entrapment is the first step towards normative compliance, then this is a useful tool, and research on human rights norm entrepreneurship has lots of insights on strategies and tactics to pursue this.  Ambiguity can also inhibit behavior change, however, insofar as it enables “open washing.” The literature offers compelling arguments that this phenomenon is driven by over-confidence in comparative assessments and by imprecise notions of things like “participation.”  Here too, the emphasis on policy precision in recent global performance assessment scholarship suggests that policy definition and benchmarking occur exclusively at the national level, this might inhibit the effect of global norms on government behavior. My own research on the implementation of OGP in Nordic countries suggests that the intense identification with the idea of open government at a superficial level con directly and completely inhibit the implementation of more specific policy recommendations.

Leadership matters, and is fragile

Political leadership matters. In social constructivist theory, this is has to do with rhetoric, and “repeated declarations by state leaders on the legitimacy of the obligations that an international norm places on states usually raise the norm’s salience in the national arena” (Cortell and Davis, 2002: 76). Political leadership also implies material power to coerce other government agencies or political actors, which is why executive branches are often the focus of campaigns and efforts towards rhetorical entrapment.  Both these factors have been emphasized in contemporary scholarship, and visible political support is one of Brockmyer’s key factors for MSI success in achieving government transparency.

Other research has complicated this truism. David-Barrett and Okamura (2016) note that “top-level commitment to joining EITI might easily turn out to be mere posturing, or the ambition of a lone reformer who has little clout. Even genuine  commitment is fickle, and Luna-reyes and Harrison’s (2016) assessment of open government initiatives in three countries notes the inherent instability of political leadership in functioning democracies (elections happen), finding that political leadership is “the most vulnerable element” to successful open government initiatives.

This is not news for global do-gooders. The veracity and sustainability of political leaders’ commitment to global norms is a hot topic, particularly for MSI-style initiatives that formalize commitments to process rather than specific policy. The above evidence suggests that dynamics of precision might also meaningfully address challenges related to national political leadership.

Future research

This quick exercise found some strong evidence for normative influence on government behavior, across a number of disciplines. It also found quite a few holes in the literature.

Here are my top picks for research to inform smart design of global do-goodery:

Within-case analysis on norm contestation and translation in MSI contexts

  1.      How do appeals to international normative authority by national civil society influence contestation? What factors make such appeals effective?
  2.      When does the ambiguous structure of norms get instrumentalized and what political and social factors constrain that instrumentalization?

On Commensuration (the quantification, or simplification, of evaluative aspects of actor behavior that enables categorization, rating and relative peer ranking that would otherwise be too complex or incomparable)

  1.      Across case analysis: Does the inclusion of a comparative mechanism influence the effects of commensuration on government behavior.
  2.      Within-case analysis: how does the introduction of national civil society to commensuration between government actors, or between government actors and international norm entrepreneurs and standard-setters, influence those processes?

On ranking vs. bespoke assessments

  1.      Comparative analysis of whether there is a lesser tendency for governments to backslide on their implementation of international normative mechanisms that include a comparative assessment, than those that do not.
  2.      Within-case or across-case analysis of how comparative assessments are instrumentalized within government to mobilize resources and mandates across agencies and decentralized institutions.

References

Brockmyer B and Fox J (2015) Assessing the Evidence: The Effectiveness and Impact of Public Governance-Oriented Multi-Stakeholder Initiatives. Available from: http://transparencyinitiative.theideabureau.netdna-cdn.com/wp-content/uploads/2015/09/Assessing-the-Evidence-MSIs.pdf.

Broome A and Quirk J (2015) Governing the world at a distance: the practice of global benchmarking. Review of International Studies 41(5): 819–841. Available from: http://www.journals.cambridge.org/abstract_S0260210515000340.

Checkel JT (1997) International Norms and Domestic Politics: Bridging the Rationalist-Constructivist Divide. European Journal of International Relations1 3(4): 473–495.

Common R (2013) When Policy Diffusion does not lead to Policy Transfer: explaining resistance to international learning in public management reform. In: Carroll P and Common R (eds), Policy Transfer and Learning in Public Policy and Management: International contexts, content and development, New York: Routledge, pp. 13–29.

Cortell AP and Davis JW (2002) Understanding the Domestic Impact of International Norms: A Research Agenda. International Studies Review 2(1): 65–87. Available from: http://onlinelibrary.wiley.com/doi/10.1111/1521-9488.00184/abstract.

David-Barrett E and Okamura K (2016) Norm Diffusion and Reputation: The Rise of the Extractive Industries Transparency Initiative. Governance 29(2): 227–246.

De Blasio E, Selva D, Blasio E De, et al. (2016) Why Choose Open Government? Motivations for the Adoption of Open Government Policies in Four European Countries. Policy & Internet: 1944–2866. Available from: http://dx.doi.org/10.1002/poi3.118.

Dobbin F, Simmons B and Garret G (2007) The Global Diffusion of Public Policies: Social Construction, Coercion,\nCompetition, or Learning? Annual Review of Sociology 33: 449–72. Available from: http://www.scopus.com/inward/record.url?eid=2-s2.0-34848903268&partnerID=40&md5=5eb01f513c196228ac8832997f027769.

Espeland WNN and Sauder M (2007) Rankings and Reactivity: How Public Measures Recreate Social Worlds. American Journal of Sociology 113(1): 1–40. Available from: http://www.journals.uchicago.edu/doi/abs/10.1086/517897%5Cnhttp://www.journals.uchicago.edu/doi/pdfplus/10.1086/517897%5Cnhttp://files/2782/Espeland y Sauder – 2007 – Rankings and Reactivity How Public Measures Recre.pdf%5Cnhttp://files/2783/517897.html.

Falleti T. G. and Lynch JF (2009) Context and Causal Mechanisms in Political Analysis. Comparative Political Studies, SAGE PublicationsSage CA: Los Angeles, CA 42(9): 1143–1166. Available from: http://cps.sagepub.com/cgi/doi/10.1177/0010414009331724 (accessed 5 April 2017).

Falleti Tulia G. and Lynch JF (2009) Context and Causal Analysis. Comparative Political Studies 42(9): 1143–1166.

Finnemore M (1993) International Organizations as Teachers of Norms: The United Nations Educational, Scientific, and Cutural Organization and Science Policy. International Organization 47(4): 565–597.

Finnemore M and Sikkink K (1998) International Norm Dynamics and Political Change. International Organization 53(2): 887–917.

Fox J, Halloran B, Levy A, et al. (2016) Connecting the Dots for Accountability: Civil Society Policy Monitoring and Advocacy Strategies. Available from: http://transparencyinitiative.theideabureau.netdna-cdn.com/wp-content/uploads/2016/03/Connecting-the-Dots-for-Accountability.pdf.

Francoli M and Clarke A (2014) What’s in a name? A comparison of ‘open government’ definitions across seven Open Government Partnership members. JeDEM – eJournal of eDemocracy and Open Government 6(3): 248–266. Available from: http://www.jedem.org/article/view/227.

Gerven M Van, Vanhercke B and Gürocak S (2014) Policy Learning, Aid Conditionality or Domestic Politics? The Europeanization of Dutch and Spanish Activation Policies Through the European Social Fund. Journal of European Public Policy, Taylor & Francis 21(July): 509–527. Available from: http://www.tandfonline.com/doi/abs/10.1080/13501763.2013.862175.

Gilardi F (2015) Who Learns from What in Policy Diffusion Processes ? American Journal of Political Science 54(3): 650–666.

Goldstein J and Weinstein J (2012) The Benefits of a Big Tent: Opening Up Government in Developing Countries. UCLA Law Review Discourse 178(2012): 178–208. Available from: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2012489.

Gonzalez-Zapata F and Heeks R (2014) The multiple meanings of open government data: Understanding different stakeholders and their perspectives. Government Information Quarterly, Elsevier Inc. 32(4): 441–452. Available from: http://dx.doi.org/10.1016/j.giq.2015.09.001.

Guerzovich F and Moses M (2016) Learning How to Open Government: Findings and reflections on how the Open Government.

Honig D and Weaver C (2017) A Race to the Top?: The Aid Transparency Index and the Normative Power of Global Performance Assessments. (May 2016): 1–47.

Johnson B and Hagström B (2005) The Translation Perspective as an Alternative to the Policy Diffusion Paradigm: The Case of the Swedish Methadone Maintenance Treatment. Journal of Social Policy 34(3): 365–388.

Kelley JG (2017) Scorecard diplomacy : grading states to influence their reputation and behavior. Cambridge: Cambridge University Press.

Kelley JG and Simmons BA (2015) Politics by number: Indicators as social pressure in international relations. American Journal of Political Science 59(1): 55–70.

Kelley JG, Simmons BA and Doshi R (2016) The Power of Ranking: The Ease of Doing Business Indicator as a Form of Social Pressure.

Kosack S and Fung A (2014) Does Transparency Improve Governance? Annual Review of Political Science 17(1): 65–87. Available from: http://www.annualreviews.org/doi/abs/10.1146/annurev-polisci-032210-144356.

Luna-reyes LF and Harrison TM (2016) Open Data and Open Government : From Abstract Principles to Institutionalized Practices. In: Proceedings of the 17th International Digital Government Research Conference on Digital Government Research, pp. 76–85.

Mastenbroek E and Kaeding M (2006) Europeanization Beyond the Goodness of Fit: Domestic Politics in the Forefront1. Comparative European Politics 4(4): 331–354.

Michener G (2015) Policy Evaluation via Composite Indexes: Qualitative Lessons from International Transparency Policy Indexes. World Development, Elsevier Ltd 74: 184–196. Available from: http://dx.doi.org/10.1016/j.worlddev.2015.04.016.

Montero AG (2015a) Open government in Uruguay: Strengthening dialogue to make up for institutional challenges. U4 Report. Available from: http://www.u4.no/publications/open-government-in-uruguay-strengthening-dialogue-to-make-up-for-institutional-challenges/.

Montero AG (2015b) Open government and transparency reform in Chile: Balancing leadership, ambition and implementation capacity. U4 Report. Available from: http://www.u4.no/publications/open-government-and-transparency-reform-in-chile-balancing-leadership-ambition-and-implementation-capacity/#sthash.TfaKklD0.dpuf.

Montero AG and Taxell N (2015) Open government reforms The challenge of making public consultations meaningful in Croatia. U4 Report. Available from: http://www.u4.no/publications/open-government-reforms-the-challenge-of-making-public-consultations-meaningful-in-croatia/.

Nielsen RA and Simmons BA (2015) Rewards for Ratification: Payoffs for Participating in the International Human Rights Regime? International Studies Quarterly 59(2): 197–208.

Park C, Wilding M and Chung C (2014) The importance of feedback: Policy transfer, translation and the role of communication. Policy Studies, Taylor & Francis 35(4): 397–412. Available from: http://www.tandfonline.com/doi/abs/10.1080/01442872.2013.875155.

Park C, Lee J and Wilding M (2016) Distorted policy transfer? South Korea’s adaptation of UK social enterprise policy. Policy Studies, Taylor & Francis 38(1): 39–58. Available from: http://www.tandfonline.com/doi/full/10.1080/01442872.2016.1188904.

Parks BC and Masaki T (2017) When Do Global Performance Assessments Influence Policy Behavior? Micro-Evidence from the 2014 Reform Efforts Survey.

Pinheiro R, Berg LN, Kekäle J, et al. (2017) Exploring the Interplay between ‘Fashion’ and ‘Evidence-based’ Policy: A Comparative Account of Higher Education and Health Care in the Nordics. Scandinavian Journal of Public Administration 21(1): 33–55.

Pomerantz J and Peek R (2017) Fifty shades of open. First Monday. Available from: http://www.ojphi.org/ojs/index.php/fm/article/view/6360.

Porto de Oliveira O (n.d.) International Policy Diffusion and Participatory Budgeting. Cham, Switzerland: Palgrave Macmillan.

Risse-Kappen T, Ropp SC and Sikkink K (1999) The Power of Human Rights: International Norms and Domestic Change. American Foreign Policy Interests 22(4): 318. Available from: http://books.google.com.co/books/about/The_Power_of_Human_Rights.html?id=kpsDPvaCOCAC&pgis=1.

Sauder M and Espeland WN (2009) The Discipline of Rankings: Tight Coupling and Organizational Change. American Sociological Review 74(1): 63–82. Available from: http://www.jstor.org/stable/27736048%5Cnhttp://www.jstor.org/stable/pdfplus/10.2307/27736048.pdf?acceptTC=true%5Cnhttp://files/2785/Sauder y Espeland – 2009 – The Discipline of Rankings Tight Coupling and Org.pdf.

Schneider J (2015) Bringing Government Data Into the Light: Slovakia’s Open Data Initiative, 2011-2015. Innovations for Successful Societies which. Available from: http://successfulsocieties.princeton.edu/publications/bringing-government-data-light-slovakias-open-data-initiative-2011-2015%5Cnfiles/247/Schneider – 2015 – Bringing Government Data Into the Light Slovakia&.pdf%5Cnfiles/248/bringing-government-data-light.

Simmons B (2009) Mobilizing for Human Rights: International law in Domestic Politics. Cambridge, Mas: Cambridge University Press. Available from: https://books.google.com/books?hl=en&lr=&id=qmDxDMruTMMC&oi=fnd&pg=PR7&dq=SIMMONA,+2009&ots=B8zsAp6bBG&sig=GHAK1e2nDPj7tdmijCvWDJplPs

Stern A, Kingston D and Ke J (2015) More Than the Sum of Its Parts: Making Multi-Stakeholder Initiatives Work. Available from: http://globaldevincubator.org/.

Stone D (2004) Transfer agents and global networks in the ‘transnationalization’ of policy. Journal of European Public Policy 11(3): 545–566. Available from: http://dx.doi.org/10.1080/13501760410001694291%5Cnhttp://www.tandfonline.com/doi/abs/10.1080/13501760410001694291.

Terman R and Voeten E (2017) The relational politics of shame: Evidence from the universal periodic review. Review of International Organizations, The Review of International Organizations: 1–23. Available from: http://dx.doi.org/10.1007/s11558-016-9264-x.

Towns AE and Rumelili B (2017) Taking the pressure: Unpacking the relation between norms, social hierarchies, and social pressures on states. European Journal of International Relations: 135406611668207. Available from: http://journals.sagepub.com/doi/10.1177/1354066116682070.

Winston C (2017) Norm structure, diffusion, and evolution: A conceptual approach. European Journal of International Relations: 135406611772079. Available from: http://journals.sagepub.com/doi/10.1177/1354066117720794.

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech

Tags