Evaluating the visual design of science publications—a quantitative approach comparing legitimate and predatory journal papers

Abstract The rise of predatory publishing poses a significant challenge to the integrity of scientific research, potentially undermining the credibility of scholarly communications. As parts of the academic community grapple with distinguishing legitimate from dubious publications, understanding the subtle differences between them becomes paramount. Therefore, this study focuses on some of these subtleties by examining the aesthetic differences in journal research papers published by potential predatory publishers versus legitimate ones. A comprehensive analysis was undertaken on 443 legitimate and 555 predatory Open Access publications, utilizing a rigorous quantitative approach. This investigation encompassed an evaluation of metadata, layout elements (such as typography, white space, page sizes, and figures), and other measurable visual attributes. Not only do the findings reveal statistically significant disparities in the visual presentation and embedded metadata of the published PDFs (potentially shedding light on the tools used for document creation), but the investigation also serves as a proof of concept for the employed analytical method: Using Python as scripting language, we offer a scalable solution for scrutinizing large datasets of PDF files based on design criteria, all while upholding a stringent quantitative approach.