Methodical Snark critical reflections on how we measure and assess civic tech

Roundup: circumvention on the rise, costing closed contracting, better case selection, and a check list for digital methods



The costs of closed contracting: A recent study on contracting in Hungary finds corruption increases when abandoning open auction procurement for closed negotiation (specifically, costs rise 8% and productivity drops 12%.

Collaboration and adaptation lead to better development outcomes. This from a recently completed multi-year literature review, which finds positive outcomes associated with collaborative and locally led-approaches, as well as providing project staff opportunities to reflect, continuous learning, and trust and decision-making capacity at the front line of project implementation.

Conditions for engagement and reform: Multilevel analysis of data from the European Social Survey suggests that  citizens who distrust government institutions are more likely to sign protests, boycott products and participate in demonstrations. Interestingly, that effect is minimized by cultures of openness.  When seeking to increase engagement with participatory budgeting processes, on the other hand, citizen proposals, debates, and budgeting allocation are among the features identified in a review of over 60 applications and platforms. Meanwhile, countries adopt access to information laws when they have strong civil society and low political combination, and when international norms are salient. At least that’s the perfect cocktail suggested by fuzzy set qualitative comparative analysis in Government Information Quarterly, though “Necessity analysis shows that there are no necessary conditions for ATI law adoption.”

Circumvention on the rise. A new study finds that a whopping 11% of Chinese internet users are using circumvention tools (up from previous estimates of 2-3%), and they tend to be politically engaged and cynical of news media, but it’s hard to say much more than that about whether circumvention is advancing democratic values, as promised by article’s title. #clickbait.

Case studies & Concepts

The Krisis journal has a special issue on Data Activism with a philosophical focus, including theoretical discussions of the ambitions of MySociety, the influence of surveillance, and digital mapping initiatives. There’s also an article proposing A Framework for Strengthening Data Ecosystems to Serve Humanitarian Purposes.

An article in Canadian Public Administration suggests that govt ethics offices should use twitter to advance government ethics, as it’s quicker and cheaper than legislation. Also,  “If the Trump Presidency and the Trudeau vacation transgression have taught us nothing else, it is that even developed democracies need to be vigilant about protecting and enhancing their democratic institutions.” Sigh.


Miren Gutiérrez has a new book on Data Activism and Social Change which doesn’t seem to offer much in the way of findings. UPDATE: correction and description of findings from the author in comments below. 

The Journal of Information Technology & Politics has book reviews on Zhao’s Networked Publics and Digital Contention: The Politics of Everyday Life in Tunisia and Karpf’s Analytic activism.


Oxfam is hiring a Deputy Head of Research and @feamster asks if we need a clearinghouse for internet research.


Using Stata? The World Bank’s Git repository of Ado files will make your life easier.

Digital Methods: A new NM&S article from Grey et al provides a reality check(list) for digital methods. It aims to help researchers to maintain fine distinctions between medium and message when using digital methods, and closes with a handy  aide-mémoire of eight questions to pose to your findings and interpretations.

Using quant on participation: An article in American Behavioral Scientist provides a step-by-step description of how to apply econometric methods to studying discrepancies in participation and their causes (using National Institute of Health data on funding access and demographic data on individuals), and closes with reflections on the integration of qual methods.

Resources online: The US National Archives publishes online dashboard of investigations into missing public records, which couples nicely with this article on Lessons from DataRescue. Meanwhile, Sunlight Foundation has created a number of open data user personas and Harvard’s Ash Center has a new online catalogue of smart city projects, or as they put it a “searchable database indexes visual and geospatial solutions to critical urban problems.”

Lastly, there are 2018 updates (!!!) to the African Integrity Indicators and ODI’s Data Ethics Canvas.

In the Methodological Weeds

How to fix the case selection problem? Researchers are often drawn to case analysis of unlikely successes in development, advocacy or governance reform, and tend to ignore the problems of generalization that follow outliers with a good reputation. A new DLP Research Paper presents a mixed methods approach for dealing with positive deviance in case selection, in the context of anti-corruption interventions. It’s a nuanced adaptation of the standard start-with-quant,-pay-attention-to-what-you’re-doing-approach, and well tailored to the civic tech sector.

In other news, VoxDev has a nice discussion on appropriate uses of satellite data in developing countries, and people overestimate predictably, which has implications for crowdsourcing projects, according to a new paper in Journal of the Royal Society Interface.

Miscellanea and Absurdum

LSE impact blog shared a whirligig of of notable findings on academic publishing last week, noting though peer review is bad for your mental health, academic papers that undergo more revisions get cited more often, although, then again, the most highly cited papers aren`t the most significant.
In other news, research shows that social impact program evaluations show disappointing outcomes, so researchers suggest that the answer is more funding for program evaluations. And tech has introduced at least one new social practice. It’s called walking your phone.

1 Comment

  • Dear Christopher Ben,
    I read your blog “Roundup: Circumvention on the rise, costing closed contracting, better case selection and a checklist for digital methods” with interest. In response to your comment on my book, I would like to suggest that you read it. I welcome justified criticism, and I am well aware of my book’s many failings, but the lack of findings is not one. As far as I know, my book is the first comprehensive account of data activism.
    You may say it is descriptive, and you would be right. However, there are some valuable proposals emerging from the dozens of interviews and case studies that are the basis for this study, including a non-normative classification of data activist organisations into skills transferrers (i.e. which transfer data and social science skills, create platforms and tools, and trigger collaborative opportunities); data journalism producers; catalysts, providing the funds and resources; and data activists proper. I also explore the reasons why data activists choose to act as they do. For example, the lack of data journalism and the discredit of journalism in Spain seem to have inspired some non-governmental organisations to fill the gap and produce journalistic content supported by data, while they practice advocacy at the same time.
    I also identify five main ways in which data activists go about generating datasets. Data activist projects and organisations can a) rely on whistle-blowers for data; b) resort to public (and sometimes open) datasets; c) use crowdsourcing tools to collect citizen data; d) turn to appropriating data; and e) get data from primary research that can be datafied or from data-capturing devices such as sensors and drones. Examples for each group are analysed and examined in relation with each other.
    Besides, I detect eleven attributes of data activism based on the case studies and interviews. Three of the qualities that are more common include data activists’ inclination to collaborate and generate alliances, their unapologetic hybridisation (i.e. data activists have no qualms in crossing lines separating areas of action such as advocacy and campaigning, funding, research, training, journalism and media work, and humanitarianism) and their preference for maps when they visualise data.
    I also look into the presence of these attributes’ intensity in each case and the relationships among the attributes and in relation with the type of data activism. For example, a data activist organisation that is dedicated to transferring skills, sometimes provides resources to support data projects, produces limited data activist content, occasionally generates data visualisations, works in alliances, and often provides match-making opportunities between organisations to deliver data activist projects can then be categorised as a skill transferer that also has something of a catalyst. This is the case of DataKind, for instance. All data activist organisations examined are granted values for more than one attribute; cross-pollination is in itself an attribute.
    Finally, a model for effective data activism is offered as a theoretical heuristic tool to examine other cases of data activism or to help design other initiatives of data activism beyond this study. The model can help design theories of chance and strategies for new data endeavours.
    This study has shown how the data infrastructure can help individuals and groups critically approach datafication and data agency, independently produce their datasets, shape practices and issues, generate, and ultimately change things.
    If you are interested, I would be happy to send you a copy. Best wishes, Miren Gutierrez

Methodical Snark critical reflections on how we measure and assess civic tech