Appendix

Appendix I: Indicator tables for Charter principles

The indicator tables are a result of the analysis conducted through comparative mapping of five indicators based on the indicators provided by ODB, GODI, ODIN, OURdata, and EODMA. See an initial mapping of this work in this spreadsheet, and the original indicator tables document.

Authors (in alphabetical order):

• Alison Rygh, OECD

• Ana Brandusescu, Web Foundation

• Arturo Rivera Perez, OECD

• Carlos Iglesias, Web Foundation

• Danny Lämmerhirt, Open Knowledge International

• Eric Swanson, Open Data Watch

• Heleen Vollers, Capgemini

• Jamison Crowell, Open Data Watch

• Reginald Dadzie, OECD

• Stephen Gates, Open Data Institute Australian Network

• Wendy Carrara, Capgemini

The comprehensive Indicator Tables document is available in AirTable.

Appendix II: Measurement Guide methodology

As demonstrated in this guide, Charter principle commitments include policy elements, aspects of information management, questions of data quality, but also engagement processes within government and between government and external parties. On the surface, the Charter principles offer valuable commitments. But what exactly do they mean? How are key terms of the Charter principles defined? What indicators can we use to measure progress against Charter commitments? To answer these questions, working group members organised several meetings, firstly extracting key terms in the Charter commitments, and looked how well indicators map onto these. The goal of these exercises was to answer the following questions:

• How well are key terms in commitments defined?

• How do Charter commitments relate to one another, where are overlaps and relationships?

• What reference points and criteria are suited to measure commitments and their key terms?

• Where do we disagree how indicators should be defined?

• What is not measurable?

The Charter MAWG members contributed to a mapping exercise to understand which parts of the Charter principles and their commitments are currently measurable, which could be measured in the future, and which may not be directly measurable. The reflections also helped us understand the extent to which measurement tools cover all of the Charter commitments. This work was exploratory. The purpose was to gather expert views on possible ways to measure the Charter. The results include additional observations. Simultaneously, the analysis covered how well-suited measurement tools are to assess aspects of the Charter principles. A ‘task force’ was created that includes researchers from five major open government data assessment tools (ODB, GODI, ODIN, OURdata, EODMA), that identified similarities, differences and gaps.

Title	Description
Principle commitment	Listed the text of each principle commitment. This was the basis to list all components of a commitment.
Commitment component	Isolated different aspects that a commitment is talking about. For example, if a commitment says that Charter adopters will provide timely, high-quality data we extracted as components 1) timely data, and 2) high-quality data.
Indicator	Describes the phenomenon that is actually measured.
Target value	What is the best value that can be achieved? How is it defined?
Assessment process of target value	In your assessment process what information do you consult to assess whether a target value is met?
Unit of analysis	A unit of analysis is the ‘thing’ being measured. Every indicator assesses at least one unit of analysis. Often, there are several interpretations of it. For example, data findability can include website observation, naming URLs, or the time it takes to find a dataset. Given this, different measurement tools use different units of analysis.
Type of metric	What metric does the indicator use? (e.g. Boolean value, likert scale, multiple choice, etc)

Appendix III: Useful resources

A Maturity Model for Prioritizing Open Government Data. Available at: https://opengovdata.io/2014/maturity-model-for-prioritization/

Benchmarking open data automatically. Available at: https://theodi.org/guides/benchmarking-data-automatically

Data on the Web Best Practices. Available at: https://www.w3.org/TR/dwbp/

Documenting the development of open standards for data. Available at: https://theodi.org/blog/documenting-the-development-of-open-standards-for-data

European Open Data Portal in 2015 (“Creating Value through Open Data: Study on the Impact of Re-use of Public Data Resources”

Governing by indicators. Global Power through Quantification and Rankings. Available at: https://global.oup.com/academic/product/governance-by-indicators-9780199658244?cc=nl&lang=en&

Governing by rankings: How the Global Open Data Index helps advance the open data agenda. Available at: https://research.okfn.org/governing-by-rankings/

ODI Maturity Model: Guide - Assessing your open data publishing and use. Available at: https://www.scribd.com/document/260481608/ODI-Maturity-Model-Guide-Assessing-your-open-data-publishing-and-use#download&from_embed

OECD Handbook on Constructing Composite Indicators. Methodology and User Guide. Available at: https://www.oecd.org/std/42495745.pdf

Open Data Barometer Global Report. Available at: https://opendatabarometer.org/4thedition/report/

Open Data Charter Agricultural Open Data Package. Available at: https://opendatacharter.net/agriculture-open-data-package/

Open Data Impact: Open Data Is Changing The World In Four Ways. Available at: http://odimpact.org/

Open Data Maturity Report 2017. Available at: https://www.europeandataportal.eu/en/highlights/open-data-maturity-europe-2017

Open Definition 2.1. Available at: http://opendefinition.org/

Sunlight Foundation’s Open Data Policy Guidelines. Available at: https://sunlightfoundation.com/opendataguidelines/

The New European Interoperability Framework. Available at: https://ec.europa.eu/isa2/eif_en

The State of Open Government Data in 2017. Available at: https://blog.okfn.org/files/2017/06/FinalreportTheStateofOpenGovernmentDatain2017.pdf