Text mining methods for automated web data extraction to create a tropical weeds traits database

Weeds are among the most significant biotic reducing factors to crop production, causing major yield losses, particularly in tropical agroecosystems where they benefit from favorable climate conditions. However, the high diversity of tropical weeds (more than 1,700 species observed) is still poorly known, especially from a trait-based perspective. Beyond impeding their identification, this knowledge gap strongly limits our understanding of how they respond to the environment and agricultural practices, which significantly compromises the development of appropriate agroecological management. Currently, quantitative data on tropical weed traits are scarce and often difficult to extract. Moreover, qualitative data are heterogeneous from a semantic, syntactic and structural point of view and come from various sources (e.g., databases or herbarium websites such as TRY, GBIF, Kew Garden, WIKTROP, Seed Information Database) in various formats (e.g., numeric, textual, graphical) making them difficult to access, integrate, analyze and model. Though, there is a need to gather, consolidate and standardize the existing knowledge on tropical weed traits to better understand how biotic and abiotic filters influence their growth, development, and effects on cropping systems performances. Our work aims to create a global trait database dedicated to tropical weeds based on existing online sources. Based on an extensive list of tropical weed species, we use and compare different web scraping and text mining algorithms to collect, sort out, clean and organize relevant qualitative and quantitative data on plant traits in a large sense (i.e. information on plants and organs morphology, growth, physiology, phenology, ecology) from various pre-identified data sources. Using these methods provides access to nonnumerical data (e.g. plant reproduction habits) and allows us to explore “grey literature” to enrich our database. Then, the information gathered is filtered and standardized into a database to enable comparative analyses of species traits. We then address the benefits of our approach in the context of weed science. Drawing on recent data science methods, this work represents a crucial step towards filling in the knowledge gaps concerning these numerous tropical weed species and should help the development of sustainable weed management practices in tropical cropping systems.