Errors in document-type classification: a focus on engineering publications and their publishers

Abstract This study investigates document-type (DT) classification errors—such as misclassifications of research articles , reviews , conference proceedings , editorials , etc.—in the bibliometric databases Scopus and Web of Science, focusing on the field of engineering . Such misclassifications can adversely affect academic research and research quality assessments, with potential repercussions on researchers' careers and the allocation of funding in academic institutions. By analysing a corpus of approximately 10,000 publications, a semi-automated approach is employed to identify misclassified documents and attribute errors to the responsible database. Additionally, the role of publishers (e.g., “Elsevier”, “Springer Nature”, “Taylor & Francis”, etc.) is investigated, based on the hypothesis that certain publishers may contribute more significantly than others to DT-classification errors, due to specific editorial practices or inconsistencies in metadata. The results reveal that classification errors are non-negligible (i.e., they occur at rates of a few percentage points) and that the extent of publishers' contributions varies significantly in both Scopus and WoS. However, the most problematic publishers for each database appear to be uncorrelated. Integrating various statistical tests, this study provides insights that may be valuable to researchers, research evaluators, database operators, and publishers, raising awareness of the issue and offering preliminary indications for identifying possible remedies to mitigate such errors and enhance DT-classification accuracy. Future research will further investigate the reasons behind the concentration of errors among certain publishers.