Bridging the gap: feasibility and mixed-methods refinement of a comprehensive school health promotion instrument in India

School based health promotion programs have become a popular means to enhance the health and well being of adolescents. The evaluation effectiveness of these programs greatly depends on the clarity, reliability, and contextual appropriateness of the instruments utilized. It is true that pilot testing is normally done before big surveys, however, the involvement of this testing in a systematic manner to different evaluation tools is seldom reported, especially in India which is a low, and middle, income country. A pilot and feasibility study was the first step in an exploratory sequential mixed method design aimed at refining a multidomain questionnaire for the evaluation of school health promotion initiatives. A pilot feasibility study ( N = 94) was carried out. The sample size was sufficient to identify major item performance issues and ceiling/floor effects and not to confirm stable factor structures, in line with the best practices for instrument development in resource constrained settings. As a pilot study, the sample size was sufficient to identify gross item performance issues and feasibility constraints and not full factor structure stability. Student responses served as the basis for psychometric diagnostics, such as a comprehensive consistency study through Cronbach’s alpha, item, total correlations, alpha, if, item, deleted analysis, split, half reliability estimation, response distribution analysis, and the identification of missing data patterns. The teachers’ and parents’ qualitative feedback and the field observations were used to determine the face validity, content relevance, and feasibility. The integration of quantitative and qualitative findings informed the systematic instrument refinement for the next step of full, scale deployment. The different questionnaire domains varied in their internal consistency. The domain of student participation showed good reliability (Cronbach’s alpha = 0.73), whereas the health education domain had a lower internal consistency (Cronbach’s alpha = 0.55). The rest of the domains had low, unstable, or even negative internal consistency values that point to differences in the content of the items, the applicability of the conditions, and the mixed response formats. At the item level, diagnostics revealed that some items had weak or negative item-total correlations, redundancy, ceiling effects, and a high level of missing data. The feasibility evaluation uncovered difficulties related to the understanding of policy, oriented items and conditional questions by respondents. These results led to the targeted refinements such as the rewording of items, the removal of selected items from composite scale scoring, the division of heterogeneous domains into conceptually coherent subscales, and the addition of not applicable response options for conditionally relevant items. The initial internal consistency was variable (range = − 0.08 to 0.73, with two domains not estimable due to excessive missingness and mixed response formats). Through mixed methods triangulation, certain cultural barriers to item comprehension were identified, which resulted in the removal or modification of 23% of the original items. Pilot testing was crucial from a methodological point of view in upgrading the clarity, the internal coherence, and the contextual relevance of the evaluation instrument. Thorough psychometric analysis and qualitative triangulation of pilot data without data recollection facilitated evidence based refinement before the wide scale application. Such a way serves as an open and feasible methodological template for school health promotion researchers operating in resource limited educational settings.