Methodical Snark critical reflections on how we measure and assess civic tech

Research Links (w 36-37)


Papers / Findings

  • Dont trust the crowd! Paper in PLOS-One finds that the subjective experiences of contributors to crowdsourced map-making are influenced by their nationality and degree of expertise. Moreover-surprise-this influences what they report to the map. Data sampled from geowiki, methods a bit too technical for me to assess.
  • New empirical analysis in Administration and Society shows stong positive correlation between government transparency and citizen efficacy (feeling that their political actions make a difference), but that this is strongly controlled by level of education. This based on a citizen survey run in 27 countries in 2004 (ISSP) and WEF perception data on transparency. There’s lots of caveats to draw around the findings (cross-sectional, multilevel, old data, subjective indicators), but the citizenship data set looks really rich and the article closes with a nice and trite response to the the recent populist disavowal of expertise (“the information-rich hold governments accountable, allowing the information-poor to free ride”).
  • Meta-analysis of survey research on social media and politics suggests positive correlations of many kinds of social media use (expressive, informational, but not entertainment) with civic engagement and political participation. It’s a careful and well documented empirical exercise, but there’s only 22 studies to draw on, so I think the more granular assertions should be taken with a grain of salt. Also, it’s all survey data, and they’ve used two values to code for the country focus variable (US and other). Still, there are some statistically relevant correlations.
  • The above comes from a special issue on Political Engagement in New Media & Society. NM&S is often deep in the theoretical weeds, but this issue has a number of original and empirical research pieces, including a longitudinal study of MySociety platforms and citizen engagement, an Israeli study that links hybrid media habits with greater political engagement and attempts to definitively undercut the idea of slactivism, and an Italian study suggesting that the most effcetive twitter campaigns during elections are the most negative. And more, a lot more.
  • Karen Okamoto conducted desk research to discover how NYC open data is being used. That’s exciting because NYC is a real frontrunner in open local government, and because there have been almost no studies of this kind. But it’s a bit lackluster. The research is not systematic, and the conclusions confirm everone’s assumptions. And while it’s interesting to see how heterogenous the apps and websites using open data are (she found 77  “items” which she organized into 31 categories), and that a few types of data tend to get used most (311, crime, transportation), you don’t learn anything about which data sets get released but never used, which might be just as interesting.

In the Methodological Weeds

  • FiveThirtyEight has a nice write up of research on evictions, which discusses the dangers of manageable and structured survey instruments, and how hard it can be to ask the right questions when surveying vulnerable and hidden populations. Also some provocative thoughts on cash incentives (“sent out mailings, with a dollar or two inside so tenants would look at them before throwing them out.”)

Community / Commentary

  • Oxfam has launched a new M&E wonk blog called real(geeks), targeting “anyone who sees evaluation and research as one of the essential tools for learning and for development work.” The Oxfam consistently put out useful reflections on the nuts and bolts of measurement, and also how to use it, so will be nice to have this gathered (though they haven’t set up the rss yet).
  • @MishaAngrist shares a call for preprints, from the 1960s
  • New books about research on and via technology. The new Research Handbook on Digital Transformations has chapters on privacy and innovation. @participatory offers some early praise for Salganik’s Bit By Bit: Social Research in the Digital Age up now for comment in draft form.
  • OGP blogged about new data from the Open Budget Partnership, suggesting among other things that data showing “trends in document disclosure over time” could help open budget champions time their advocacy efforts. I got excited about that idea, until I thought about it closely, and thought about how all the activists I’ve ever worked with know more about their political context than they could ever be bothered to put into words. About how in-country advocates are the ones you go to to explain this kind of data, and about how much HQ always loves this kind of comparative stuff. Then I decided it was just a new degree of sophistication and subtelty in data fetishization.

Add Comment

Methodical Snark critical reflections on how we measure and assess civic tech