BioMetaHarmonizer: An Integrated Pipeline for Retrieval, Harmonization and Standardization of NCBI BioSample Metadata for Large-Scale Bacterial Genomic Epidemiological Analysis

Large-scale comparative genomics and antimicrobial resistance surveillance depend on structured, machine-readable sample metadata, yet NCBI BioSample, the primary repository for microbial genomic provenance, contains heterogeneous submitter-supplied metadata despite field-specific validation. Attribute names can vary across submitters, date representations may require normalization, and geographic fields combine country, region, and locality information. Without harmonization and structured extraction, phylogeographic analysis, cross-sector AMR surveillance, and One Health source attribution at the scale of tens to hundreds of thousands of genomes can be labor-intensive and require manual preprocessing that does not scale. We designed BioMetaHarmonizer, an open-source Python package that directly fetches BioSample records from the NCBI Entrez API and transforms them into a standardized 53-column output in CSV, TSV, Excel, Parquet, or JSONL format through a modular harmonization pipeline covering attribute synonym resolution, ISO 8601 date normalization, geospatial parsing with ISO 3166-1 country-code assignment, and automated One Health category inference. Performance of BioMetaHarmonizer was evaluated using 17,573 Bacillus cereus group and 217,226 Klebsiella pneumoniae BioSample records. On a modest cloud virtual machine, the pipeline exhibited end-to-end processing in under 5 and 27 minutes, respectively, with One Health classification accuracies of 0.9488 and 0.9945. Source code is freely available under the MIT license.

Authors

Institutions

Publication Details

Journal
Informatics
Published
2026-11-02
DOI
https://doi.org/10.3390/informatics13110184
Primary Topic
Research Data Management Practices
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

BioMetaHarmonizer: An Integrated Pipeline for Retrieval, Harmonization and Standardization of NCBI BioSample Metadata for Large-Scale Bacterial Genomic Epidemiological Analysis

Rustam N. Heydarov, Vladimir M. Mikhailovich, Ksenia A. Romanova
2 links
Informatics
Research Data Management Practices
article

BioMetaHarmonizer: An Integrated Pipeline for Retrieval, Harmonization and Standardization of NCBI BioSample Metadata for Large-Scale Bacterial Genomic Epidemiological Analysis

Rustam N. Heydarov, Vladimir M. Mikhailovich, Ksenia A. Romanova
article en

Abstract

Large-scale comparative genomics and antimicrobial resistance surveillance depend on structured, machine-readable sample metadata, yet NCBI BioSample, the primary repository for microbial genomic provenance, contains heterogeneous submitter-supplied metadata despite field-specific validation. Attribute names can vary across submitters, date representations may require normalization, and geographic fields combine country, region, and locality information. Without harmonization and structured extraction, phylogeographic analysis, cross-sector AMR surveillance, and One Health source attribution at the scale of tens to hundreds of thousands of genomes can be labor-intensive and require manual preprocessing that does not scale. We designed BioMetaHarmonizer, an open-source Python package that directly fetches BioSample records from the NCBI Entrez API and transforms them into a standardized 53-column output in CSV, TSV, Excel, Parquet, or JSONL format through a modular harmonization pipeline covering attribute synonym resolution, ISO 8601 date normalization, geospatial parsing with ISO 3166-1 country-code assignment, and automated One Health category inference. Performance of BioMetaHarmonizer was evaluated using 17,573 Bacillus cereus group and 217,226 Klebsiella pneumoniae BioSample records. On a modest cloud virtual machine, the pipeline exhibited end-to-end processing in under 5 and 27 minutes, respectively, with One Health classification accuracies of 0.9488 and 0.9945. Source code is freely available under the MIT license.

InformaticsVol. 13(11)
Engelhardt Institute of Molecular Biology (RU), Russian Academy of Sciences (RU)
Russian Science Foundation
Openalex Percentile: Top 10%
Research Data Management Practices
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon