Methodical Snark critical reflections on how we measure and assess civic tech

research links w 11-17



Voice online:
Twitter advocacy can bypass mainstream media that excludes non-elite voices, according to a study of how #IfTheyGunnedMeDown was used following 2014 police shootings in Ferguson, Missouri. That’s good news for digital advocacy innovators, but important to remember that people don’t feel safe online and don’t understand how their personal information gets used, but most aren’t willing to get trained. This according to survey of “Mozilla community members” (n=30,000+) by the Mozilla Foundation.

All the FOIA researchin the US:
Freedom of information in the US has been suffering in recent years from reduced access to information and increased FOIA denials, and a survey of “336 freedom of information experts” suggests that it is only going to get worse. On the other hand, “data pulled from over 30,000 FOIA requests” by @MuckRock suggests that the US govt is much better at responding to FOIA requests than we tend to think. Digging into the details, US FOIA gets used by business actors more than any other group,  single actors dominate requests from most groups (@MuckRock among news groups, @JudicialWatch among NGOs) and Democratic committees dramatically out-FOIA their Republican counterparts (81/7). These and other fascinating insights in @galka_max‘s analysis of 229,000 requests to 85 government agencies.

Standards and state of play:
This week saw more harsh critiques of Brazilian open government, as a review of open data portals in “561 municipalities” finds that ” vast majority of assessed data portals did not comply with the basic requirements stated by national law.” The European Center for Not-for-profit Law has released a report arguing that there is a general framework for participatory decision-making in Europe, based on a desk review of literature and European legislation. There are multiple case studies and the bibliography is good. Meanwhile, an NRGI report finds that a majority of countries surveyed are “disclosing at least some of their extractives contract or licenses,” and track a dramatic increase in recent years in tandem with international policy advances. A US-based lawyer group reported on two years of research on police body cams, with 10 recommendations for better policies.

Assumptions confirmed:
Participatory research from MAVC finds that African tech innovation hubs are rarely engaging with policy, but that there are significant opportunties to do so. If that piques your interest, Govt Information Quarterly has a special issue on Open Innovation in Public Sector with lots of good stuff.

Otherwise, in climate advocacy as elsewhere, E-participation only a game changer when coupled with offline engagement (survey of experts in three countries), surveys and interviews related to planning of community projects in Thailand (50 villages) confirms that more participatory rules increase participation in participatory policy-making, and a survey of Malaysian undergrads (n=168) confirms that increased online access to political information correlates with increased online political participation and political interest, but not policy satisfaction.

Research on three cities’ open data ecosystems concludes that “current open data platforms do not take into account the complexity of democratic processes,” and (sigh) suggests a conceptual remedy. Research from SF State Univ suggests that larger cities have open data portals with more features.


There’s also a new publication with some interesting chapters on open data ecosystems (including an interesting comparison of US/MX/RU and a deep dive on Latin American infomediaries), but alas, paywall. In any case, a review of literature on open data and accountability finds it woefully lacking (only 12 of 290 “simultaneously address open data and accountability theory,” only one “focuses on the complete accountability process”). This feels like a poor sample to me. Of course, the authors suggest fixing with a conceptual model.

ARTICLE 19 and CIMA have produced an intro guidebook to internet governance. Open Data Watch offers a couple examples of how their index has been used by countries to improve the quality of their open data, and Sunlight Foundation has been reviewing literature and case studies on open data impact and citizen engagement to produce a guide for cities on how to make data useful. It’s in Beta and open for comments (FTW!).

Pet Peeves:
The New York’s new Online Transparency Index has been getting a lot of shares. It seems to be a platforming for crowdsourcing citizen assessments of transparency in counties, cities, towns, etc, kind of like a completely open Indaba (#insidejoke). But I really don’t get it. There’s no description of the methodology or how they quality control contributions, and the project isn’t mentioned on the website of the organization that’s presumably behind it (their logo at the header of the index links back to the site). Clicking on cities, it lists dozens, only one of which has data, for one of twenty nine unspecified indicators. Is this poor roll out by social good consultancy gig or vaporware?

In the Methodological Weeds

There’s a new tool and tutorial for web scraping in difficult contexts, what @OpenUpSA calls “smashing a serious site’s defences against scraping, fooling Captchas, and masquerading as just a regular ol’ browser.”

Someone has trained algorithms to predict community capacity and participation on the basis of open government data.

An RCT was recently run to compare the merits of individual interviews and focus groups for data collection in sensitive contexts. The results suggest that interviews are better for generating large numbers of inputs (brainstorming), but focus groups are better for surfacing sensitive information. There’s a compelling discussion of the explanations for this. The article’s gated, but there’s a summary here.

Academic Opps

Democracy Fund announced support to efforts to fight misinformation, through “$1 million in grants with an average size of about $50,000″ (Deadline: 3 April)

Democracy Fund is also hiring a research fellow on elections.

Funding available for meta-research in the social sciences, call for proposals aiming towards “strengthening reliability and validity of social science research findings.” (Deadline 20 March)

Registration open for ICTs and Society Conference: Activism, Research & Critique in the Age of Big Data Capitalism (20-21 May, London)

Calls for Papers:

Miscellanea & Absurdum

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech