Assessing the comparative effectiveness of health informatics education: short-term knowledge gain and technology awareness in health science students

Health informatics (HI) competencies are becoming increasingly vital for healthcare professionals in Nepal, where accelerating digital health adoption demands an informatically literate workforce. Rigorous controlled evaluations of HI courses remain scarce in low- and middle-income country (LMIC) settings. To evaluate the effectiveness of the Health Informatics-471 blended course at Madan Bhandari Academy of Health Sciences by comparing knowledge acquisition, domain-specific learning, and technology awareness between Bachelor of Public Health (BPH) intervention students (G1) and Bachelor of Pharmacy (B.Pharm) control students receiving no dedicated HI instruction (G2). A controlled pre-test/post-test quasi-experimental design with a non-equivalent control group was used. Both cohorts (G1: BPH, n = 17; G2: B.Pharm, n = 17 randomly sampled) completed a 44-item, previously developed knowledge assessment and an 11-item technology awareness instrument at T1 (Week 1) and T2 (Week 15). Non-parametric statistics (Mann–Whitney U, Wilcoxon signed-rank, Friedman, McNemar) with bootstrapped effect sizes were used throughout the study. BPH students demonstrated significant total knowledge gain (median T1:58.3, T2:88.9; Wilcoxon W = 0, p < 0.001, r = 0.879). B.Pharm students showed no significant change (median T1:41.7, T2:42.6; p = 0.352, ns). The primary between-group contrast in gain scores demonstrated a large between-group difference (Mann–Whitney U = 284, p < 0.001, r = 0.824 [95% CI: 0.748, 0.857]; Δ = + 29.6 [+ 24.1, + 34.3]). BPH students reached 100% technology awareness on 9/11 tools at T2; six tools Health Information Management System (HIMS), Database Management System (DBMS), Python, Programming Language, Telehealth platforms, and Remote Patient Monitoring (RPM) gained from 0 to 100%. All findings were robust across sensitivity analyses (Table 6), including a post-hoc analysis excluding reverse-discriminating items. The HI471 blended course was associated with large, statistically robust short-term knowledge gains and near-complete technology awareness across all content domains, a pattern suggestive of an association with course participation beyond background academic progression, rather than definitive evidence of a course-specific causal effect, though residual baseline non-equivalence between groups warrants cautious interpretation. Domain analysis supports targeted curriculum strengthening in Data Analysis and Coding, which showed the lowest baseline performance and the most challenging assessment items. ClinicalTrials.gov NCT07690241. First submitted 02 July 2026; posted 07 July 2026. Retrospectively registered.