Methodical Snark critical reflections on how we measure and assess civic tech

research links w8-17



#participationwashing? Participatory mechanisms promise to empower the marginalized, and can provide the illusion of power, but an ethnographic study on development processes in Boston shows how participation can simply reinforce existing power dynamics: “residents appear empowered, while officials retain ultimate decision-making authority.” Worse than that, a (peer reviewed but unpublished?) article on Vietnam demonstrates how  e-government is not e-democracy, and authoritarian states can digitize just as well as anyone else, while G20 countries are  breaking promises to release anti-corruption data, according to a report from the Web Foundation, which notes the quality of what they do release isn’t that great either.

So how to make government more responsive? Put the mayor on Twitter says  a social network analysis of citizen-state social media interaction in Seoul, Korea. Meanwhile, a new research report from MAVC supports the common assumption that crowdsourced information is inherently political, due in no small part to the behavior and interaction of crowdsourcing infomediaries, which is itself messy, while a survey of 57 Swiss legislators suggests that making lawmakers argue on the basis actual performance evidence changes the way they budget, but also increases polarization.

There’s little media diversity in the US, and it’s getting worse,  according to field analysis by Democracy Fund, while the Association of Internet Researchers list serve just circulated a fantastic bibliography of research on activist hashtags (thread title “[Air-L] Air-L Digest, Vol 151, Issue 26”), while a piece in Nature analysed google trends related to the ALS Ice Bucket Challenge to conclude that viral campaigns are dynamic and fast paced, but to last they require deep and slow offline engagement.

Better information can influence voting behavior, even in polemic referenda, but the information effect is mediated both by perceived risk and the perceived proportion of undecided voters, suggests an experimental study on the Scottish referendum on independence. Meanwhile, who knew that electronic voting saves lives and increases health spending? This according to a natural experiment (!) in Brazil, Duncan Green gives a quick summary.

Lastly, a new edition of Public Administration and Information Technology has several articles on e-gov in small island pacific states, including arguments about their contributions to global policy development, evidence that government are using social media, but not really talking to civil society, and analysis of the interaction between regional and national level policy dialogues.


Advice: Excel’s the worst. Or at least problematic for storing and managing research data, as it often silently corrupts that data. @danfowler offers strategies and tools for getting data off of excel and onto safer platforms, without giving up the basic functionality like grouped tables and column labels, which are commonly missed in csv formats. @dalgoso meanwhile offers strategies for organizations to turn data into organizational learning and practice. The tips are good, the bigger question is how to get orgs to care and devote the time it takes.

New stuff!: Analyze survey data for free is website that presents R scripts to automate data download from a bunch of surveys. Much data, headache very less. Plus there’s a new social media handbook for governments, a Princeton academic has released an encryption primer for policy-makers, and a new report from Nesta maps digital democracy, apps and all, suggesting a typology and lessons from 6 case studies (have a clear plan, have the support and resources you need, choose the right tools).

Duncan Green reviews a new(ish) global index that measures countries global citizenship, by which they mean participation in things like treaties for human rights, gender equality, all that good stuff…

Careful consideration:  Andrew Gelman tries to distinguish between unethical research methods and being a “bad” researcher, British researchers argue that think tanks need to reinvent their role as the arbiters of policy advice in a post-fact world. Meanwhile, noting a spate of recent corruption scandals and regulatory reactions in Latin America, a report from the Inter-American Dialogue reviews comparative data for the region (Latinobarometro, AmericasBarometer, CPI and Bank’s Worldwide Governance Indicators) to conclude that it’s not due to increased corruption, but  a steady norm change and increased awareness regarding corruption, that likely leads to increased reporting.

In the Methodological Weeds

Innovative Methods in Media and Communication Research is a new book with a critical bent. Chapters span both wildly innovative methods (…Participatory Visual Approaches in Audience Research), surveys of methods for tackling tricky stuff (…Ways of Knowing Algorithms) and innovative perspectives on established methods (…Critical Literacy for Data Visualizations as Research Objects and Research Devices).

Confessions of a national researcher. South African contributor to the GODI blogs about her experience trying to measure the openness of four countries’ data, and lays bare some of the methodological challenges built into this model of comparative research.

Pet peeves:

  • ODI has a 3 part blog series on “Researching Progress,” by which they mean communicating development results. The second in the series describes strategies for managing common challenges of analyzing progress, and notes that “it’s hard to prove causation,” so suggests instead using “judgement calls” and “forgoing attributional precision,” which is kind of the definition of NOT proving causation.
  • Sunlight presented a much shared article on big data and corruption by noting that “The paper finds that ‘Open data can put vast quantities of information into the hands of countless watchdogs and whistleblowers….” To be clear, it doesn’t find that. The paper argues that, and does not present any findings or new evidence.
  • More research marveling at the magic sauce of technology. (“the framework applies to all officials.” PERIOD. paywall)

Academic Opps


Calls for Papers

Miscellanea & Absurdum

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech