DataMap: A Scalable Microservices-Based Platform for Democratising and Analysing Massive Biological Knowledge Graphs

Researchers across the UK Dementia Research Institute (UK DRI) generate hundreds of TB of data monthly, including genomics, proteomics, transcriptomics, behavioural, and neuroimaging data. Despite this vast output, much of the data remains siloed within individual labs, making it easy to share and collaborate on single datasets but difficult to integrate and collaborate on combined analyses across the institute and broader community. To address this, we developed DataMap—a network-based, interactive platform designed to democratise access to UK DRI’s heterogeneous research data by allowing researchers an easy and reliable mechanism for viewing and analysing their findings. DataMap builds on a large-scale knowledge graph stored in Neo4j, integrating multimodal datasets and enabling researchers to query and explore complex biological relationships. However, querying such a massive and evolving graph is computationally intensive. To overcome this, we adopted a microservices-based architecture, supported by in-memory data structures and metaprogramming. This allows us to decouple memory-heavy operations, scale components independently, and adapt the platform to evolving user needs and new data types. Here, we demonstrate how DataMap enables seamless and efficient interaction of UK DRI datasets via a user-centric web interface. Users can navigate biological entities, explore connections between data layers, and retrieve insights—all without requiring technical expertise. The system’s flexibility and scalability make it well-suited for the demands of dynamic research environments, allowing us to maximise knowledge extraction from our generated data.A recording of this session is available on YouTube: https://youtu.be/F2Xq7DrBiUU