Executive Summary

The Measurement Guide helps governments, civil society, and researchers to understand how to assess open data activities based on the Open Data Charter (the ‘Charter’) principles. It seeks to shed light on the often opaque and jargon-filled world of open data measurement. The Measurement Guide is an analysis of the Charter principles and how they are assessed based on current open government data measurement tools – with a focus on commitments that can be measured, commitments that cannot be measured, and existing gaps (e.g. commitments that have not been measured).

The Measurement Guide is made for governments, civil society, and researchers to understand how the Charter principles can be measured. It provides an analysis of the indicators, which includes comprehensive tables of global indicators (e.g. indicator tables) per each Charter principle.

• For governments, the guide summarizes the most important insights in this section, the Executive Summary.

• For civil society and communicators, the indicator tables and our analysis provide transparency about existing measurement tools (‘Five open data assessment tools’) and what they measure. This can help civil society to oversee the progress of open data policy at a country level.

• For researchers, the guide explains the methodology to map open data indicators against Charter commitments. The indicator tables created can be used to compare existing data measurement tools and develop new indicators.

The Measurement Guide provides insights from open data experts and members of organizations who work on open data measurement tools. Analysis of the coverage of the five leading open data measurement tools – the Open Data Barometer (ODB), Global Open Data Index (GODI), Open Data Inventory (ODIN), Open Useful Reusable Government Data (OURdata), and the European Open Data Maturity Assessment (EODMA) – reveals that only parts of Charter principle commitments, and their components, are being measured; or that some commitments could be measured in the future. However, some Charter concepts are either too broad (e.g. “high-quality data”, “usability by the widest range of users”), or lack a shared interpretation, which makes them difficult to find a common indicator.

The Measurement Guide also covers how existing indicators metrify key open data concepts. It is important to note that not all aspects of a commitment are clearly defined. Multiple ways of measuring currently exist for some commitments. Some commitments need to be defined and measured on a country-by-country basis to incorporate local context.

Principle 1: Open by Default The creation of policies and the promotion of a culture of openness are mostly measurable through the indexes of readiness, data policies and assessments of government maturity.

Principle 2: Timely and Comprehensive The measurement of these commitments is largely made by manual inspection of data and supporting resources, and hence is very labor intensive. Principle 3: Accessible and Usable The technical and legal aspects of open data accessibility are easy to measure and can partly be automated, however measuring the degree to which people can make use of open data is more difficult. Principle 4: Comparable and Interoperable While the existence of metadata and documentation can be measured, more guidance is needed to enable data comparison and interoperability through the use of agreed standards. Principle 5: For Improved Governance and Citizen Engagement In most cases, commitments require further clarification due to terms and wording that may be interpreted differently, including reference to local legislation that would make comparisons between countries challenging. Principle 6: For Inclusive Development and Innovation The number and quality of open data initiatives, business products and services developed, educational programs and research partnerships created can be partly measured.