Towards a Robust Assessment Framework for the EU Open Data Maturity Index

Strengthening Open Data policies is a priority for the Public Administrations of European countries because, firstly, they leverage their development momentum through economic, social, and environmental impacts and, secondly, they enhance transparency and accountability. In this context, the EU Commission publishes every year a report ranking a set of countries with respect to their maturity to Open Data. The methodology followed by the EU has given different results from year to year in terms of the ranking of a country, and the question that has been raised is the reliability of the method applied. The aim is to explore different decision methods to test their consistency and stability in relation to the existing method. In this study, a qualitative and quantitative research was conducted using a questionnaire and the AHP method was applied to modify the weights of the criteria that constitute the Open Data Maturity Index. Several countries show high volatility in their performance on sub-indices that are quite difficult to determine objectively. The AHP application showed that these sub-indices should have a fairly low weighting, having little impact on countries’ performance. Based on the revised weights, the study arrives at a different ranking of the countries under evaluation and, combined with the use of the k-means method, a different clustering. A more structured and robust evaluation framework is proposed using ranking algorithms such as TOPSIS and PROMETHEE II.