Methodical Snark critical reflections on how we measure and assess civic tech

October Roundup: evidence on unfair citizen reviews, unready governments and the failure of global norms

In Research Roundup
November 4, 2018
14 Min Read
Add Comment
O

Findings

Voice and advocacy

Contrary to popular talking points, the 2018 American Institutional Confidence Poll finds that “Social media use is not strongly associated with dissatisfaction with democracy and national institutions.” Research on social media reviews of hospitals conducted by Indiana University finds that citizen reviews of health services aren’t “fair” (due to lack of expertise among the rankers,) while a U4 report argues that Right2Info portals are facilitating the emergence of a new kind of civil society.  Analysis of messages to parliamentarians on MySociety’s WriteToThem platform finds that women are (slightly) more likely to write to female parliamentary representatives.

Taking a step back to the diffusion of global advocacy, @engnroom has blogged that interviews conducted in Nigeria and Ghana (no publication or details on methods yet) suggests that national organizations think about open data and accountability much differently than their international counterparts. In particularly, they prioritize generating new data over using existing data, and see a more fractured strategic landscape that international advocates.

Digital govt

There’s a strong correlation digitization of municipal government and citizen satisfaction, according to analysis of Swedish survey and administrative data (which notably finds this correlation to be as strong as any other predictors, and relevant for satisfaction with government effectiveness, transparency and their own influence over government). This won’t help Canada, whose government isn’t ready for digital, according to a year of interviews conducted by the Public Policy Forum (there’s also recommendations).

Effects of open:

Adopting an open data policy decreases the public records requests received by cities, according to Sunlight Foundation analysis of panel data from 52 US cities. Notably, this effect is greater when associated with “robust” open data policies, and increases over time. They also did some interesting analysis on what kinds of data ppl are requesting (police records top the list).

In other news, this Bangladeshi survey (n=275) finds a positive relationship between open government data use and citizen empowerment (but I can’t get past the paywall to check their methods or measurements), and there’s some kind of relationship between government open data, accountability and service delivery. That’s the simplest headline I can draw from regressions based on four years of panel data in 25 African countries. Though there are some fairly clear positive correlations, and the author attempts to account for some intervening variables, the causal relationship between these variables isn’t clearly delineated and the analysis isn’t explained quite thoroughly enough for me to make quick sense of it. Some of this likely is has to do with cosmetic details like poor labeling of tables, but I’ve also got issues with concept validity stemming from the use of Ibrahim Index data to operationalize service delivery, and endoneniety regarding the bundle of variables. It’s conference paper from last month’s ODRS, so hopefully this will get polished and tightened in peer review, using insights from ppl smarter than me.

Political contexts

Limits on important civic freedoms are linked to negative economic outcomes for business. That’s the conclusion drawn by BTeam from comparing V-Dem data with countries’ GDP. There’s no info on the statistical methods they use (cue the dinosaur dozen), or reflections on construct validity (is GDP the same as business outcomes?). Nevertheless, they’re quick to conclude that  “countries with higher degrees of respect for civic rights experience higher economic growth rates as well as higher levels of human development. The research shows that these economic growth rates are especially linked to the state of civic rights in the Middle East and North Africa, Sub-Saharan Africa and East Asia.”

Useful Research

Global Integrity and Transparency International have been researching how citizens decide to take action against corruption. On the basis of “desk research and empirical research conducted over the course of four weeks in Tunisia and Georgia,” they’ve developed a  three-level model for influences on the decision to take action. The 66pg research report includes recommendations for national and international anti-corruption advocates, and 12 pages on research design which is skinny on analytical methods, but has some really great models for case selection and considering generalizability.

NewTactics’ Tactical Mapping tool equips activists to collaboratively expand their understanding of relationships and develop strategic and effective action, and DistrictBuilder is “an open-source software redistricting application designed to give the public transparent, accessible, and easy-to-use online mapping tools. As they show, the goal is for all citizens to have access to the same information that legislators use when drawing congressional maps—and use that data to create maps of their own.” (from the blurb for the book by Micah Altman and Michael P. McDonald)

Resources and Data

Empowering People Award has 200,000€ grants for low-tech to solve basic supply problems in developing countries (Siemens Stiftung Foundation). The World Bank’s World Development Indicators have a new website with discovery tool and storytelling platform for WDI data (WB), and Elvis (map me tender) is a new platform for managing data on government tenders. There’s also a new database with 360 years of United States case law on offer and Politico has a fake news database.

Guides:

Concepts and Frameworks

How do government institutions adopt e-government? It’s generally agreed that it’s a progressive process, and there’s a lot of models out there describing it. A survey of subnational institutions working with cultural heritage (1560 in 11 countries) looks at advanced e-gov implementation, and suggests that there are three primary pathways: “from social media use to crowd-sourcing or collaborative content creation, […] from social media use and centrally managed data through digitization to open content, [and] from centrally managed metadata through open data to linked data.” Some mix of these appears to be at play for most institutions.

How do global norms fail local activism? This paper suggests three common models: “misrepresentation occurs when international organizations accept and support activists who do not represent local voices […], misperception happens when [locals don’t get it, and] mismatch occurs when pre-existing local movements mistakenly use international rights claims for their own goals, lead local discussions, and overshadow transnational activism.”

There’s also frameworks for evaluating e-services (emphasizing the potential for feedback loops and validated in Azerbaijan), for assessing the risks and feasibility of public–private partnerships for e-government in developing countries, for assessing the internet’s contribution to identity building, agenda-setting, control and criticism, and deliberation in the public sphere.

Lastly, this chapter applies mediatization theory to e-participation. I’ve been waiting for someone to do this. But why is it so very paywalled and the publication so very obscure?!?!?

Case studies

 

Community & Curation

Is Open Government a field of academic study? This systematic review of 189 articles (in three top journals, 2011 to 2015) suggests it is, notes a lack of quantitative and explicative-correlational studies, a disproportionate anglo-American focus, and wide variety of topical interests. Similarly, this stocktaking finds that “The open data discussion is still exclusive, as there is still much elitism surrounding access to data,” while this article maps scholarly research on open data across disciplines (Engineering, Health, Public Administration and Management studies), and proposes a research agenda for each. Why? Did anyone in those fields ask for that?

Collections

Asks:

  • The OGP is asking for feedback on it’s Flagship Report on Open Government. The draft itself is based on consultations and organized according to 12 themes (Freedom of Assembly, Freedom of Association, Defending Activists and Journalists, Open Contracting, Beneficial Ownership, Education Services, Health Services, Water and Sanitation Services, Fiscal Openness, Right to Information, Open Justice, Open Policy-Making)
  • Code4Africa is crowdsourcing data on human trafficking.

Events:

Books

Perspectives

My gripes

  • The LSE Impact Blog is tooting the Unpaywall horn again. As when it was first launched, this sounds great, but I for one can’t find anything with it. At least very week I get stuck behind a paywall because of limits on my University’s subscriptions. AndI click on that little lock dutifully, but to no avail, and am still end up emailing the author, checking with friends at other universities, or digging into scihub. This is most frustrating because I really want that extension to work exactly as it’s described.

 

In the Methodological Weeds

The authors of this conference paper argue that crowdsourcing local research data is cheaper, easier, and more reliable that traditional methods of data collection. They demonstrate this by crowdsourcing research on several hundred US cities (who is the mayor and have cities experienced financial crisis), then demonstrate accuracy and validity in comparison with data collected from traditional methods.

There’s a curated list of World Bank blog posts on methodological issues, including external validity and analysis issues super relevant to civic tech research.

Miscellanea and Absurdum

 

  • New orgs:  The Omidyar Network’s Governance and Civic Engagement Initiative has spun out as the Luminate Group, and MyData Global is an NGO network aiming to empower individuals by improving their right to self-determination regarding their personal data.

 

  • “Borg Complex” is a term “to describe the attitude and rhetoric of modern-day utopians who believe that computer technology is an unstoppable force for good and that anyone who resists or even looks critically at the expanding hegemony of the digital is a benighted fool. (The Borg is an alien race in Star Trek that sucks up the minds of other races, telling its victims that ‘resistance is futile.’)” (from The future’s so bright, I gotta wear blinders, at Rough Type)

Add Comment

Read more

Methodical Snark critical reflections on how we measure and assess civic tech
This is a blog about how the civic tech and accountability field measures its work and its impact. It’s based on a critical perspective, but tries to be more productive than conference snark. It’s an effort to highlight how much work is being done in silos across the academic-NGO divide, to de-mystify important methods and findings, and to call out research that is sloppy or unhelpful.

Tags

africa anti-corruption big data brazil China civic engagement collaboration comparative indices corruption crowdsourcing development e-government elections europe facebook fake news field comment FOIA gender ghana health human rights impact India intermediaries Kenya latin america literature review methods OGP open data participation participatory budgeting procurement protest RCTs social media spain survey transparency trust twitter uganda US visualization