Does citation context information enhance the validity of citation analysis for measuring research quality? An empirical comparison of peer assessments and enriched citations

Abstract As citation context information has entered the Web of Science (Clarivate) in recent years, one wonders whether the legitimacy and soundness of citation analyses for measuring research quality of publications can be improved by considering this information. The legitimacy and soundness of citation analysis can be established by assessing its construct validity. This can be done by studying how citation impact variables relate to generally accepted external validation data, like human expert assessments of research quality. In this study, we investigated the statistical associations of information extracted from structured citation context data—such as the position of citations in certain sections—with postpublication peer quality ratings from Faculty Opinions (H1 Connect) as external validity criterion data for publications in medicine and life sciences. We found that quantitative variables derived from citation contexts are correlated with human expert assessments over and above ordinary citation counts. Hence, the construct validity of citation analyses can be improved by incorporating citation context data into measurement models. The legitimacy and soundness of citation analyses for measuring research quality can be improved by considering this information.