A Cross-sectional Pilot Study Evaluating the Validity and Reliability of the MASTISHQ CoreScan-Surface Module as a Digital Mental Health Intervention

Abstract Background: Digital tools for psychiatric screening have the potential to improve scalability and efficiency, especially in settings with limited resources. This pilot study explores how well MASTISHQ CoreScan-Surface, a semi-automated digital mental health screening instrument, performs compared to traditional clinical diagnoses made by psychiatrists. Methodology: A total of 46 participants completed assessments using MASTISHQ CoreScan-Surface, followed by independent evaluations conducted by psychiatrists. The study examined various psychometric properties, including sensitivity, specificity, positive predictive value and negative predictive value (NPV), and inter-rater reliability (using Cohen’s kappa), and internal consistency (via Cronbach’s alpha). To visualize the results, ROC and radar plots were employed to interpret diagnostic performance. Results: The results showed that MASTISHQ CoreScan-Surface achieved perfect sensitivity (100%) and NPV (100%), with a Cohen’s kappa over 0.84, indicating strong agreement with the clinical diagnoses. The internal consistency was solid, with a Cronbach’s alpha of 0.81 across different diagnostic areas, and the specificity was 90%, which is acceptable for screening purposes. The structured digital format and randomized questions likely supported the reliability of the tool. However, limitations included a small sample size from an urban setting and the lack of assessment of digital literacy among participants. Conclusion: MASTISHQ CoreScan-Surface shows promising validity and reliability as a digital mental health screening option suitable for both clinical and community environments. Further studies involving larger and more diverse populations, along with long-term follow-up, are recommended to confirm its generalizability and predictive accuracy over time.