Generic Quality Control Tool for Environmental Marine Datasets

In order to ensure the reliability and accuracy of Marine observation data, there is a need to make sure that the data are reliable and of good quality. This is important because in situ datasets obtained via observation platforms are usually subject to measurement errors, faulty instruments, failed communications, and environmental disturbances, which leads to data corruption and misinterpretation. In this paper, a Generic Quality Control Tool for Marine Observation Datasets that automatically determines anomalies in time series is proposed. The framework incorporates several quality control methods, including range testing, spike detection, and stuck value determination. It is implemented in Python language, with the use of Pandas library. The system assigns each record a quality flag depending on the set validation criteria and helps in distinguishing invalid observations from valid ones. The system can be easily customized for different types of environmental variables, including sea surface temperature, salinity, and wave height. The experimental results show the effectiveness of the proposed tool. The proposed system provides a scalable and efficient tool for quality assessment on an automated basis, which will help to increase the reliability of data and environmental monitoring systems.