A Data Downpour: Extending long-term UK climatological rainfall series by blending rescued historic data with current station sources.

The Rainfall Rescue project was a remarkable effort to digitise historic UK monthly rainfall data. After scanning 66,000 pages of journals and weather logs dating back to the 17th century, the project, with the help of over 16,000 volunteers, transcribed 5.2 million observations stored in the Met Office National Meteorological Archive. This effort brought a wealth of historic data back to life. Considering this newly recovered data, our project set out to create a comprehensive, fully traceable, blended monthly rainfall series for the UK that merged Rainfall Rescue with other primary sources, including MIDAS open (Met Office Integrated Data Archive System), which contains a large publicly available collection of UK mainland surface observations. A key challenge when blending data sources is to avoid duplication. To tackle this, we developed efficient software to match combinations of gauge locations, station names and accumulated values. Once matched with certain criteria, we were able to extend back records of existing sites in our primary database, incorporating new data whilst preventing double counting. As a Scientific Software Engineer (the Met Office equivalent of an RSE), I will present an overview of techniques and tooling we applied for this task, following the Research Software Development Principles to deliver maintainable and reproducible software. I will outline the methodologies and challenges of this work and emphasise steps we took to allow traceability and configurability. The preliminary results from this project have extended records for numerous sites, offering a richer monthly rainfall series to advance climate monitoring and research. A recording of this session is available on YouTube: https://youtu.be/5MH7S1qL2IE