D2.2 – Containerized EMERALDS Toolset v1

EMERALDS’s vision is to design, develop and create an urban data-oriented Mobility Analytics as a Service (MAaaS) toolset, consisting of the proclaimed ‘emeralds’ services, compiled in a proof-of-concept prototype, capable of exploiting the untapped potential of extreme urban mobility data. The toolset enables stakeholders of the urban mobility ecosystem to collect and manage ubiquitous spatio-temporal data of high-volume, high-velocity and of high-variety, analyse them both in online and offline settings, import them to real-time responsive AI/ML algorithms and visualise results in interactive dashboards, whilst implementing privacy preservation techniques at all data modalities and at all levels of a mobility data analytics workflow architecture. The toolset offers advanced capabilities in data mining (searching and processing) of large amounts and varieties of urban mobility data. This deliverable (of type OTHER) presents a comprehensive version of ‘emeralds’ services, organized into different groups based on a taxonomy serving the project’s research goals on the fields of extreme scale data mining, filtering, aggregation and analytics, and releases an integrated version of developments from WP3 and WP4 into a reusable and containerized toolset. The following emeralds typologies have been identified: 1) Privacy-aware in situ Data Harvesting, 2) Data Fusion and Management, 3) Extreme-Scale Cloud and Fog Data Processing, 4) Extreme Scale Mobility Data Analytics at Computer Continuum, 5) Extreme Scale Mobility Data Analytics at Compute Continuum and 6) Security and Data Governance. The containerized services constituting the EMERALDS toolset will be demonstrated in applications foreseen within WP5 and exploited through the planned activities of WP6. Continuous integration and deployment software development principles facilitate the utilization of an efficient, interoperable and easy-to-deploy MAaaS toolset, including the necessary tooling for substantiating extreme data workflows. The implementation of these principles is demonstrated and explained through detailed code walkthroughs, and a full list of links to all Github repositories is provided.