Assessing the Performance of Routine Health Information Systems in Low-Resource Settings: Insights from Ethiopia

Abstract Background: It is critical to regularly monitor and evaluate the effectiveness of the health information system (HIS), strengthening efforts to support a country’s ability to plan and directing health initiatives, providing evidence to support decisions on policies and programs, and promoting better health outcomes. This case study documents Ethiopia’s journey in developing, testing, and revising and HIS performance measurement tool to be applied in resource limited setting. Methods/design: The process of tool development and revision involved four stages across six years (2016 - 2021): 1) formation of the technical team; 2) Design of the HIS measurement tool (tool development); 3) implementation of the HIS tool; and 4) revision of the HIS measurement tool. Tool development was an iterative and consultative process involving a desk review of relevant national documents, repeated meetings of Technical Working Groups (TWGS), and workshops with key stakeholders. The tool revision process took place in a workshop setting. Drawing on lessons from initial implementation, the TWG, along with other public health experts with policy and ground-level experience was engaged in the process. The revised measurement tool was reviewed and approved in a two-day workshop involving a total of 48 experts. The tool: The HIS measurement tool covers three major domains of HIS: i) HIS structure and resources, ii) data quality, and iii) data use. Each domain was assigned a percentage range based on a 100-percentage point scale: HIS Structure and Resources was assigned a score of up to 30 points, and Data Quality was assigned a score of up to 30 percentage points, and Data Use was assigned a score up to 40 points. The tool was used to assess health facilities’ HIS performance across these three domains at regular time intervals, Based on their respective HIS performance scores, health institutions were classified as “Emerging”, “Candidate (low and high)”, “Model”, and “Demonstration” institutions. Lessons learned: This case study demonstrates that it is possible to develop a HIS performance measurement tool that can be applied regularly to monitor and evaluate the effectiveness of HIS strengthening efforts in low- resource settings. This exercise also showed that regular revision of a measurement tool according to practical experiences, national priorities, and ongoing developments in the health information system, is essential. [Ethiop. J. Health Dev. 2024; 38(SI-2)] Keywords: HIS performance, tool development, measurement, Information revolution, Ethiopia, low resource settings, data use