Methodical Snark critical reflections on how we measure and assess civic tech



So last month I finally defended my doctoral thesis. Here’s a short post to close that chapter, looking at the process, product, and next steps.

1: The process

This was a three year affair. Short by many doctoral standards. I spent the first six months of it catching up on my reading list. Nothing but reading. It was heavenly. I’ll never have another six months like it.

After that I got to work trying to simultaneously produce quality academic work, and to find smart ways to support the practical uptake of evidence. I largely failed on the latter account (post forthcoming), but am pretty happy with the former. In the course of those three years I produced four articles for peer reviewed journals and a sizable synthesis chapter (table overview).

I spent two years of that time in Washington DC, failing to conduct field work for a case study on open government in the USA. That failure also merits a dedicated post, but it was productive. I learned tons during my time and at Georgetown University, where I taught a course on technology and government transparency at the School of Foregin Service, served as a Fellow at the Beeck Center for Policy Impact and Innovation, and tried to make sense of the fragile pact between civil service and political leadership in American government.

Then on Aug 28 this year I defended. This included a 45m trial lecture on the Open Government Partnership in Theories of Democratic Governance and a 3.5 hr public defense. It was gobs of fun.
(20 min slide deck with concise presentation of the research).

A special thanks to my advisor, Øyvind Ihlen, and to @SuzPiotrowski, Jean-Patrick Villeneuve and Terje Rasmussen for their work on my committee, and kind words at the event.

2: The product

My dissertation can be downloaded here, and you can also check out specific posts on findings related to norms and socialization. I’ll try to get more up.


Global multi-stakeholder initiatives (MSIs) leverage collaboration between government and civil society to improve public sector governance in participating countries. Noting the prominence of public governance MSIs in international policy discourse, and the lack of long-term data against which to assess their impact, this doctoral research proposes three alternative strategies for evaluating their effectiveness and influence. Defining the causal mechanisms through which MSIs aim to influence national policy allows testing of effectiveness prior to long-term outcomes. Measuring effects in adjacent policy areas allows for a more nuanced assessment of MSIs’ institutional influence. Lastly, more rigorous quality metrics can be applied to the norms promoted by MSIs, in order to evaluate their potential impact.

These strategies are applied in multi-method analysis of the Open Government Partnership’s (OGP) promotion of civic participation to member countries. Causal mechanisms related to argumentation and policy learning are demonstrated to drive OGP’s policy influence, and OGP is found to have a modest institutional effect on countries’ public governance outside of OGP-specific policy fora, including the informal institutionalization of policy. OGP’s effect is most pronounced in countries that already enjoy strong traditions for civic participation, and in regard to more progressive norms related to collaborative decision-making. The application of quality metrics suggests, however, that if adopted, the norms and policies promoted in an OGP context are not likely to contribute to the OGP’s end goal of more responsive and accountable governance.

This analysis validates the three evaluation strategies proposed above and adds nuance to current discourse on MSI effectiveness. After presenting the full scope of research, this cover chapter closes with implications for MSI advocacy strategies and theories of global norm promotion.

3: What’s next

I’ll be in my current position at the University of Oslo through April 2020, teaching on the new masters program on Political Communication in the Nordic Context. I’ll also be continuing as a remote fellow with the Digital Service Collaborative at Georgetown University’s Beeck Center. Hopefully I’ll have time to do some more blogging here.

Research-wise, I’ll continue to work on political participation, technology, and open government. Some of this will be a direct continuation of the doctoral work (including case study work on the US, UK and other Nordic countries that I couldn’t fit in the dissertation). I’m also interested in researching how governments talk about problematic technologies like facial recognition software and algorithmic decision-making. I’m particularly interested in  what consequences this has for how problemotech gets rolled out, as well as impacts on public trust and engagement.

Practice-wise, I’m going to keep thinking about how the civic tech community can better benefit from the academy (and avoid it in many cases), and reflecting on why my efforts over the last three years failed to bear fruit. I’ll be chatting and writing about this in the next months, and trying to come up with some new ideas that might be useful.

If you’re interested in any of the above and want to compare notes or look for opportunities, send me a note or just book a chat.

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech