The inexact science of fairness panel reviews: Can they make assessments more culturally relevant?

Key digested message This article shares the results from a systematic analysis of fairness reviews documented in a sample of technical manuals for commercial tests. This review found this documentation limited and lacking the details necessary to support the cultural fairness of the test. Details such as panelists’ demographic background and culturally relevant expertise, instructions or training, and results were rarely provided. Future implications are a call for more research on the effectiveness of training and orienting review panelists, appropriate frameworks or criteria for making decisions, and how to structure the review process to improve the identification of culturally problematic items.