Methodical Snark critical reflections on how we measure and assess civic tech

Roundup: circumvention on the rise, costing closed contracting, better case selection, and a check list for digital methods



The costs of closed contracting: A recent study on contracting in Hungary finds corruption increases when abandoning open auction procurement for closed negotiation (specifically, costs rise 8% and productivity drops 12%.

Collaboration and adaptation lead to better development outcomes. This from a recently completed multi-year literature review, which finds positive outcomes associated with collaborative and locally led-approaches, as well as providing project staff opportunities to reflect, continuous learning, and trust and decision-making capacity at the front line of project implementation.

Conditions for engagement and reform: Multilevel analysis of data from the European Social Survey suggests that  citizens who distrust government institutions are more likely to sign protests, boycott products and participate in demonstrations. Interestingly, that effect is minimized by cultures of openness.  When seeking to increase engagement with participatory budgeting processes, on the other hand, citizen proposals, debates, and budgeting allocation are among the features identified in a review of over 60 applications and platforms. Meanwhile, countries adopt access to information laws when they have strong civil society and low political combination, and when international norms are salient. At least that’s the perfect cocktail suggested by fuzzy set qualitative comparative analysis in Government Information Quarterly, though “Necessity analysis shows that there are no necessary conditions for ATI law adoption.”

Circumvention on the rise. A new study finds that a whopping 11% of Chinese internet users are using circumvention tools (up from previous estimates of 2-3%), and they tend to be politically engaged and cynical of news media, but it’s hard to say much more than that about whether circumvention is advancing democratic values, as promised by article’s title. #clickbait.

Case studies & Concepts

The Krisis journal has a special issue on Data Activism with a philosophical focus, including theoretical discussions of the ambitions of MySociety, the influence of surveillance, and digital mapping initiatives. There’s also an article proposing A Framework for Strengthening Data Ecosystems to Serve Humanitarian Purposes.

An article in Canadian Public Administration suggests that govt ethics offices should use twitter to advance government ethics, as it’s quicker and cheaper than legislation. Also,  “If the Trump Presidency and the Trudeau vacation transgression have taught us nothing else, it is that even developed democracies need to be vigilant about protecting and enhancing their democratic institutions.” Sigh.


Miren Gutiérrez has a new book on Data Activism and Social Change which doesn’t seem to offer much in the way of findings.

The Journal of Information Technology & Politics has book reviews on Zhao’s Networked Publics and Digital Contention: The Politics of Everyday Life in Tunisia and Karpf’s Analytic activism.


Oxfam is hiring a Deputy Head of Research and @feamster asks if we need a clearinghouse for internet research.


Using Stata? The World Bank’s Git repository of Ado files will make your life easier.

Digital Methods: A new NM&S article from Grey et al provides a reality check(list) for digital methods. It aims to help researchers to maintain fine distinctions between medium and message when using digital methods, and closes with a handy  aide-mémoire of eight questions to pose to your findings and interpretations.

Using quant on participation: An article in American Behavioral Scientist provides a step-by-step description of how to apply econometric methods to studying discrepancies in participation and their causes (using National Institute of Health data on funding access and demographic data on individuals), and closes with reflections on the integration of qual methods.

Resources online: The US National Archives publishes online dashboard of investigations into missing public records, which couples nicely with this article on Lessons from DataRescue. Meanwhile, Sunlight Foundation has created a number of open data user personas and Harvard’s Ash Center has a new online catalogue of smart city projects, or as they put it a “searchable database indexes visual and geospatial solutions to critical urban problems.”

Lastly, there are 2018 updates (!!!) to the African Integrity Indicators and ODI’s Data Ethics Canvas.

In the Methodological Weeds

How to fix the case selection problem? Researchers are often drawn to case analysis of unlikely successes in development, advocacy or governance reform, and tend to ignore the problems of generalization that follow outliers with a good reputation. A new DLP Research Paper presents a mixed methods approach for dealing with positive deviance in case selection, in the context of anti-corruption interventions. It’s a nuanced adaptation of the standard start-with-quant,-pay-attention-to-what-you’re-doing-approach, and well tailored to the civic tech sector.

In other news, VoxDev has a nice discussion on appropriate uses of satellite data in developing countries, and people overestimate predictably, which has implications for crowdsourcing projects, according to a new paper in Journal of the Royal Society Interface.

Miscellanea and Absurdum

LSE impact blog shared a whirligig of of notable findings on academic publishing last week, noting though peer review is bad for your mental health, academic papers that undergo more revisions get cited more often, although, then again, the most highly cited papers aren`t the most significant.
In other news, research shows that social impact program evaluations show disappointing outcomes, so researchers suggest that the answer is more funding for program evaluations. And tech has introduced at least one new social practice. It’s called walking your phone.

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech


Get in touch

Suggest research to be reviewed or mini-lit reviews. Ask questions or tell me why I'm wrong.