The (other) problem with scholarship on digital politics

One of the great dangers of the digital moment we currently are liveing through is that the discipline as a whole will succumb to a particularly virulent form of availability bias. It is easy to gather Twitter data. It is harder to navigate the Facebook terms of service, and even harder still to cobble together a comprehensive email dataset. As a result, both academic journals and academic conferences feature mountains of Twitter papers, molehills written about Facebook, and an awkward silence regarding email. We study the kinds of social media that we can access, regardless of their relative importance in political life. […]

It is not enough to study digital trace data as an alternative to surveys and content analysis. We must also attend to the messy, flawed, incomplete organizational logics that incorporate this data into strategic deliberation.

That’s David Karpf, closing out his book, Analytical Activism (pp. 174-5), on how campaigning orgs are using internal metrics to make strategic decisions. There’s a lot to it, and strategic data use isn’t the only blind spot. Perhaps more important is the kinds of organizations that tend to get studied, and how that skews the field more generally, making research outputs less useful to the kinds of orgs that actually need them.

I’m writing a review on the book, which takes issue with how the book defines it’s scope and how it’s contribution might be made more widely relevant to activist orgs. Link and summary forthcoming.


Case by case: what development economics can teach the civic tech and accountability field about building an evidence base

Warning: long post, deep weeds.

Last week saw some really interesting thinking in the development economics blogosphere, focused on design questions for external validity (the applicability of case-specific findings to other cases). This is a central question for research on civic tech and accountability programming, which talks a lot about wanting an evidence base, but remains dominated by case studies, enthusiasm and a handful of ametuer researchers. We see this clearly a couple times a year (at TicTech, the Open Data Research Forum), where the community gathers to talk about evidence, share novel case studies, acknowledge that we can’t generalize from case studies, and then talk some more as if we can.

If we want to develop the kinds of general heuristics and and rules of thumb that would be useful for the people who actually design and prioritize programming modalities, the kind that would make it possible to learn across country contexts, then we have to be smarter about how we design our research and conceptualize our evidence base. There’s a lot to learn from development economics in that regard. Development studies is like pubescent civic tech and accountability’s older uncle, who used to be cool, but still knows how to get shit done. In particular, there was a lot to learn from last week’s discussions about generalization and validity.

Continue reading “Case by case: what development economics can teach the civic tech and accountability field about building an evidence base”

Crimes against data, talk by Andrew Gelman


Andrew Gelmen gives a great talk on how data gets abused in research and politics. He goes a bit into the statistical weeds at times with T & P values and the like, but he’s also a pleasure to listen to. And he gives some great examples of both academics and public figures that either “treat statistics as a means to prove what they already know, or as hoops to be jumped through.” Continue reading “Crimes against data, talk by Andrew Gelman”

Against the civic tech sector

Stefaan G. Verhulst recently offered some suggestions on how to “build a civic tech field that can last and stand the test of time.” Stefaan is a smart guy, connected, well informed, and his suggestions make smart sense of a messy landscape. But they also accept a fundamental premise which tends to go unchecked in international discussions about civic tech.

His introduction:

…we are yet to witness a true tech-enabled transformation of how government works and how citizens engage with institutions and with each other to solve societal problems. In many ways, civic tech still operates under the radar screen and often lacks broad acceptance. So how do we accelerate and expand the civic tech sector? How can we build a civic tech field that can last and stand the test of time?

I think this represents a popular perspective, but would argue that there’s a hidden question begged. The need for a strong sector or community does not follow directly from widely recognized promise and lack of significant impact. I’m all for the exciting way in which civic tech can strengthen governance, civic engagement and quality of life, but would like to suggest that we might not need a civic tech sector for that at all, at least not in the sense of a scope of work defined by common interests. It might even be getting in the way. Continue reading “Against the civic tech sector”

What’s e-gov got to do with it?


Emily Shaw posted a great piece on the relevance of e-governance research for civic technology earlier this month. She argues that academic e-government research dwarfs the nearly non-existent academic interest in civic tech (as evidenced by 169,000 vs 185 hits on google scholar), and that civic technologists should care about research on e-government.

And in the civic tech world, we can certainly derive value from the wisdom of our e-government colleagues who’ve been working to understand what happens when government service meets the internet. To the extent that civic tech implementation requires at least an open mind—and better, an enthusiastic partnership—on the side of our government partners, it is best if we know where they’ve been coming from.

I think she’s absolutely right, but want to challenge a couple of the distinctions she makes, and look for more proactive ways that civic technologists might engage e-government learnings.   Continue reading “What’s e-gov got to do with it?”

Building on TICTec: more thinking about research pls


Last week I joined the Impacts of Civic Technology Conference 2016, a sort of annual mixer for researchers and the civic tech community, organized by MySociety to “promote and share rigorous and meaningful research into online technologies and digital democracy around the world.”

The event was good (write ups here, here, here, and here), but notable for being so firmly grounded in the idea of research, without talking about it all that much. I left inspired, but frustrated, wishing there was a forum for addressing some of the thornier issues surrounding this still fuzzy idea of research and evidence on civic technology. Because throughout the event, the idea of “research” influencing programming got mentioned a lot, but never examined. Here’s a quick run through some of those issues, and thoughts about why they aren’t yet getting the attention they deserve. Continue reading “Building on TICTec: more thinking about research pls”