143 Assessing gender literacy: A novel methodology for evaluating eligibility criteria in clinical research

Objectives/Goals: The objective of this project is to develop a tool for evaluating clinical trial (CT) eligibility criteria for demonstrated “gender literacy,” defined as the recognition that biologically assigned “sex” is distinct from personally defined “gender identity,” as a way to quantify the inclusion of gender minority populations. Methods/Study Population: The study is validating an assessment scale that evaluates gender literacy based on CT eligibility criteria (EC). Two health professionals will serve as “coders,” tasked with grading 15 CTs. EC for all CTs will be exported from clinicaltrials.gov. Once trained with using the scale, each coder will give a score for each trial. After this first scoring period, coders will share their scores and experiences using the scale. Coders will be tasked again with grading 15 new CTs. This second scoring period will yield final scores to calculate the inter-rater reliability (IRR), or the extent to which qualitative measurements are consistent and not due to random chance. IRR will be quantified by Cohen’s kappa to validate the scale. Results/Anticipated Results: Cohen’s kappa is on a continuous interval from 0 to 1, where 0 means no agreement and 1 means perfect agreement. It is expected that the Cohen’s kappa for this assessment scale will exceed 0.80. Such validation is necessary to ensure the scale is robust and dynamic for multiple use-cases and consistent across any coder. By having a discussion after the initial scoring period, we can identify confusions or challenges with the scale early on and correct them before the secondary scoring period. In comparing these two coders’ performance, it is expected that the second scores will be more similar, thus a kappa closer to 1. However, if the kappa is low, this may be because gender literacy is a learned skill, through the internalized recognition that gender is truly different from sex. Discussion/Significance of Impact: Systemic barriers and exclusionary language have excluded gender minorities from CT spaces for too long. Tools such as these, paired with standardized language for sex and gender eligibility criteria, will greatly bolster the representation of this population and spark change for a more inclusive future.