An Automated Scholarly Literature Collection Workflow Using Official Services for AI for Science
AI for Science depends on scientific corpora that can be acquired, checked, and updated reproducibly. This work presents a Python literature collection workflow based on official scholarly services. Crossref and Europe PMC support journal identification and record discovery, NLM PMC Cloud supplies eligible article files and supplementary materials, and separate Elsevier and Wiley adapters use the publishers' official APIs where the operator has the required access. The PMC implementation records provenance and licenses, validates file integrity, checkpoints completed records, and resumes using verified local files. A historical smoke test documents one article PDF and one supplementary DOCX from 11 search hits. A live check of the current release downloaded one 13-page Chemical Science PDF from 21 query hits and confirmed reuse on a second run, while also exposing ambiguity between the query term Mg and the mass unit mg. The release passed 38 offline regression tests and a desktop interface smoke test. File downloads default to an interval of 10 min, with longer intervals recommended where required by the service. The workflow provides a reference implementation for local corpus preparation; corpus completeness and downstream extraction accuracy remain outside the present validation.
Authors
- Ruijie Zhu (ORCID: https://orcid.org/0000-0001-5317-1151)
- Masaki Matsui (ORCID: https://orcid.org/0000-0003-1499-7457)
- Baoyin Yuan (ORCID: https://orcid.org/0000-0003-4970-9718)
- Huijun Yang (ORCID: https://orcid.org/0000-0001-5037-7090)
- Zhuoqi Zeng (ORCID: https://orcid.org/0009-0001-1793-4729)
- Yuxing Gong
- Zechen Li
- Jingtong Yu
- Hailun Xu
- Wei Zhang
Institutions
- Dalian University of Technology (CN)
- Guangzhou University (CN)
- The University of Osaka (JP)
Publication Details
- Journal
- ChemRxiv
- Published
- 2026-10-26
- DOI
- https://doi.org/10.26434/chemrxiv.15011077/v1
- Primary Topic
- Research Data Management Practices
- Type
- preprint