Development, validity, and reliability of the pre-attendance conference evaluator (PACE) tool for identifying predatory conferences

Background Predatory conferences (PCs) scam researchers by compromising them financially and professionally. This study presents the empirical development of the Pre-Attendance Conference Evaluator (PACE) to help academics assess a conference’s legitimacy.Materials and Method An expert group of six academics validated a preliminary list of PC traits. After identifying latent constructs through pilot testing, the list underwent four iterations based on statistical factor loadings to assess reliability, predictive validity, and concurrent validity. Later, confirmatory factor analysis included a battery of model-fit indices to assess the scale’s dimensionality.Results Content validity ratio and semi-structured interviews reduced the number of PC traits from 70 to 29. Cronbach’s alpha and the Kaiser-Meyer-Olkin test indicated 15 items suitable for factor analysis. Parallel analysis of pilot-testing data from 800 conference-level observations revealed the scale’s unidimensionality. Eventually, 11 items showed strong factor loadings and a high fit across model indices. Concurrent validity assessed by 616 conference-level observations showed high correlations (rs = 0.73–0.80) with existing tools. High predictive validity (>0.9) and test-retest reliability (ICC = 0.47–0.68) indicated moderate-to-good temporal stability. The ROC analysis yielded a cutoff score of < 14 to deem a conference predatory.Conclusion PACE demonstrated acceptable validity and reliability as a tool for diagnosing potentially predatory conferences.