Culturally Anchored Target Construction for Amharic Psychiatric Voice Artificial Intelligence: A Multimodal Data-Centric Study

Abstract Background High-stakes clinical voice artificial intelligence requires supervisory targets that are culturally valid in the language and population of deployment and that remain separate from the clinical measures later used for independent validation. In Ethiopian populations with extensive documented trauma, conventional self-report screening systematically understates severity — the defensiveness gap — motivating a voice-based severity channel that does not depend on questionnaire disclosure. This study was designed to answer a specific measurement question: can trained Ethiopian raters, under a culturally adapted listening protocol, judge psychiatric severity from Amharic speech reliably enough—and under what aggregation design—to serve as machine-learning supervisory targets independent of self-report? We report the completed analysis of a culturally anchored voice-labeling snapshot from the Lenegewa Women’s Rehabilitation and Skills Development Center that tests this question, together with the pre-specified validity architecture built on it. Methods The multimodal resource links five layers: 3,617 supported beneficiaries; 4,163 voice-sample records from 2,600 clients; 19,610 standardized survey records; 111,059 item-level symptom responses; and 67,274 clinical, service, and risk documentation records. Three trained Ethiopian raters scored each Amharic sample on the 0–39 Emotional Distress Index (EDI) and independently rated fear, anger, neutral, sadness, and happiness on 0–4 scales under a culturally adapted listening protocol. Clinical scales, diagnoses, safety indicators, and outcomes were withheld from raters and reserved as independent validation targets. The locked analysis snapshot comprised 2,479 samples, 7,919 rating attempts, 7,086 usable ratings, and 2,267 complete three-rater EDI matrices; four severity bands were derived from sample-level EDI scores rather than separately assigned. Results Derived severity was Low in 720 samples (29.0%), Moderate in 1,446 (58.3%), High in 300 (12.1%), and Severe in 13 (0.5%); median EDI was 13.0 (IQR 9.0–17.0). Ordinal Krippendorff’s alpha was 0.46 (95% CI 0.43–0.48) for derived severity bands and ranged from 0.28 to 0.51 across the five voice types; mean pairwise quadratic-weighted kappa was 0.47. EDI reliability was substantially higher for the three-rater mean (ICC(2,k) = 0.78, 95% CI 0.76–0.80) than for a single rating (ICC(2,1) = 0.54, 95% CI 0.52–0.57), with a standard error of measurement of 4.80 EDI points; rater means differed by at most 1.12 points, while Bland–Altman limits documented substantial sample-level disagreement. Conclusions The research question is answered affirmatively with a design qualification: severity judgment of Amharic speech is reliable enough to supervise machine learning only as a multi-rater aggregate, and a single rating is too noisy a target. The analysis establishes culturally anchored target construction, quantifies annotation uncertainty, preserves non-circular validation, and fixes the machine-learning validity architecture — audio-only inference, client-level data splits, constrained augmentation, calibration, subgroup checks, and teacher–student deployment separation — before any model fitting, without claiming model performance. Full-corpus labeling and subsequent algorithm performance evaluation will be reported separately.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-10684265/v1
Primary Topic
Emotion and Mood Recognition
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Culturally Anchored Target Construction for Amharic Psychiatric Voice Artificial Intelligence: A Multimodal Data-Centric Study

Yared Alemu, Habib Mohammed, Patrick Ohiomoba, Octavian Balatel et al.
Research Square
Emotion and Mood Recognition
preprint

Culturally Anchored Target Construction for Amharic Psychiatric Voice Artificial Intelligence: A Multimodal Data-Centric Study

Yared Alemu, Habib Mohammed, Patrick Ohiomoba, Octavian Balatel, Jale Bikila Abeya
preprint en

Abstract

Abstract Background High-stakes clinical voice artificial intelligence requires supervisory targets that are culturally valid in the language and population of deployment and that remain separate from the clinical measures later used for independent validation. In Ethiopian populations with extensive documented trauma, conventional self-report screening systematically understates severity — the defensiveness gap — motivating a voice-based severity channel that does not depend on questionnaire disclosure. This study was designed to answer a specific measurement question: can trained Ethiopian raters, under a culturally adapted listening protocol, judge psychiatric severity from Amharic speech reliably enough—and under what aggregation design—to serve as machine-learning supervisory targets independent of self-report? We report the completed analysis of a culturally anchored voice-labeling snapshot from the Lenegewa Women’s Rehabilitation and Skills Development Center that tests this question, together with the pre-specified validity architecture built on it. Methods The multimodal resource links five layers: 3,617 supported beneficiaries; 4,163 voice-sample records from 2,600 clients; 19,610 standardized survey records; 111,059 item-level symptom responses; and 67,274 clinical, service, and risk documentation records. Three trained Ethiopian raters scored each Amharic sample on the 0–39 Emotional Distress Index (EDI) and independently rated fear, anger, neutral, sadness, and happiness on 0–4 scales under a culturally adapted listening protocol. Clinical scales, diagnoses, safety indicators, and outcomes were withheld from raters and reserved as independent validation targets. The locked analysis snapshot comprised 2,479 samples, 7,919 rating attempts, 7,086 usable ratings, and 2,267 complete three-rater EDI matrices; four severity bands were derived from sample-level EDI scores rather than separately assigned. Results Derived severity was Low in 720 samples (29.0%), Moderate in 1,446 (58.3%), High in 300 (12.1%), and Severe in 13 (0.5%); median EDI was 13.0 (IQR 9.0–17.0). Ordinal Krippendorff’s alpha was 0.46 (95% CI 0.43–0.48) for derived severity bands and ranged from 0.28 to 0.51 across the five voice types; mean pairwise quadratic-weighted kappa was 0.47. EDI reliability was substantially higher for the three-rater mean (ICC(2,k) = 0.78, 95% CI 0.76–0.80) than for a single rating (ICC(2,1) = 0.54, 95% CI 0.52–0.57), with a standard error of measurement of 4.80 EDI points; rater means differed by at most 1.12 points, while Bland–Altman limits documented substantial sample-level disagreement. Conclusions The research question is answered affirmatively with a design qualification: severity judgment of Amharic speech is reliable enough to supervise machine learning only as a multi-rater aggregate, and a single rating is too noisy a target. The analysis establishes culturally anchored target construction, quantifies annotation uncertainty, preserves non-circular validation, and fixes the machine-learning validity architecture — audio-only inference, client-level data splits, constrained augmentation, calibration, subgroup checks, and teacher–student deployment separation — before any model fitting, without claiming model performance. Full-corpus labeling and subsequent algorithm performance evaluation will be reported separately.

Research Square
Addis Ababa Science and Technology University (ET), Woosong University (KR)
Woosong University
Good health and well-being, Reduced inequalities
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon