A web-based tool for automated metadata annotation and FAIR data management of autonomous driving sensor data

Autonomous driving systems (ADS) are becoming increasingly sophisticated, and the added complexity is largely borne out by the multitude of sensors that produce vast amounts of data for decision making algorithms. Researchers require reliable platforms to collate and share large sensor data along with their associated uncertainties. At the National Physical Laboratory (NPL), we have developed an advanced data pipeline that interacts with an object storage facility powered by the Hitachi Content Platform (HCP). This pipeline, embedded within a back-end application programming interface (API) server and a user-facing web application, archives sensor data and metadata. Our contribution lies in the automatic annotation and uploading of metadata with each data upload, a significant leap from traditional storage solutions that require manual metadata entry. This automation bolsters the fairness (Findability, Accessibility, Interoperability, and Reusability) of our data, ensuring efficient querying based on metadata properties. Our approach streamlines data access for researchers and therefore accelerates the development of ADS modelling algorithms, marking an advancement in data management and research facilitation.