Experience with 3-anchor script concordance test in undergraduate clinical assessment: A cross-sectional study

Objectives: The aim of this study was to assess the item analysis parameters of a 3-anchor script concordance test (SCT) in terms of discrimination and difficulty indices. Materials and Methods: After getting the required permissions, four SCT questions with a 3-point Likert scale were used among a total of 30 questions after faculty and student orientation. Results: The average discrimination index of the SCTs was 0.12 (average discrimination for all questions was 0.22), and the average difficulty index for the SCTs was 0.79 (average difficulty index for all questions was 0.69). Limitations: There are very few studies available to assess the validity of SCT. A larger study to compare three-item and five-item SCT is planned. Conclusion: The study showed that both discrimination and the difficulty of the SCT were acceptable but less than the average for the whole test. We attribute this to the element of guessing, which can be part of the SCT. While SCTs may be an effective and more objective method of assessing higher-level PS as compared to open-ended questions, the element of guessing needs to be factored in while designing the test.