Case by case: what development economics can teach the civic tech and accountability field about building an evidence base

Warning: long post, deep weeds.

Last week saw some really interesting thinking in the development economics blogosphere, focused on design questions for external validity (the applicability of case-specific findings to other cases). This is a central question for research on civic tech and accountability programming, which talks a lot about wanting an evidence base, but remains dominated by case studies, enthusiasm and a handful of ametuer researchers. We see this clearly a couple times a year (at TicTech, the Open Data Research Forum), where the community gathers to talk about evidence, share novel case studies, acknowledge that we can’t generalize from case studies, and then talk some more as if we can.

If we want to develop the kinds of general heuristics and and rules of thumb that would be useful for the people who actually design and prioritize programming modalities, the kind that would make it possible to learn across country contexts, then we have to be smarter about how we design our research and conceptualize our evidence base. There’s a lot to learn from development economics in that regard. Development studies is like pubescent civic tech and accountability’s older uncle, who used to be cool, but still knows how to get shit done. In particular, there was a lot to learn from last week’s discussions about generalization and validity.

Continue reading “Case by case: what development economics can teach the civic tech and accountability field about building an evidence base”

research links w 48-49

Papers and Findings

A new Brookings report aims to answer the question “Does Open Government Work?” NBD. Not surprisingly, the report doesn’t provide a definitive answer. It does suggest six structural conditions for open government initiatives to achieve their objectives. The framework is nuanced and useful, but it’s not at all clear how the authors came up with it. It would be nice to know more about their “analysis of hundreds of reports, articles, and peer-reviewed academic studies discussing the effectiveness of particular programs.” Presumably they looked at evidence internationally, but there’s no clear distinctions made between different political and cultural contexts…

Meanwhile, an article in the ARPR assesses the implementation of the OGP in the US (OGP didn’t do much to change the way the US does transparency) and Portuguese researchers have proposed a “transparency ontology” to guide the development and implementation of open data initiatives, in order to make them more relevant for citizens. The paper relies on journalists’ role as “information brokers,” which is reflected in their method. They don’t seem to have interviewed any actual citizens.

Globally, the OECD has a new book out summarizing the future of Open Government, while the 2016 UN E-government survey paints a rosy picture. It finds that 90 countries have a portal for open data or services,  148 countries provide at “online transactional services” and “an increasing number of countries are moving towards participatory decision-making.” #devilinthedetails Continue reading “research links w 48-49”

Book Review: The Global Impact of Open Data

catThe Global Impact of Open Data: Key Findings from Detailed Case Studies Around the World
By Stefaan Verhulst, Andrew Young
Publisher: O’Reilly
Released: September 2016

 

 

 

 

O’Reilly recently released a book documenting GovLab’s case studies on open data impact around the world. Some of the key findings were presented for feedback at the IODC last week, and were forecast in a report released some months ago. This book includes full versions of all 19 country case studies, so weighs in at a whopping 459 pages, daunting for many. Continue reading “Book Review: The Global Impact of Open Data”

Research Links w 39

Papers and Findings