Integrated Management of Research Instrumentation, Software, and Data for Surgical Data Science and Tactile Internet for Surgery

The recent rapid artificial intelligence (AI) and data science innovations are increasing the potential for transformations in healthcare. Surgical data science is becoming a particularly promising domain for improving surgical healthcare quality, including patient outcomes, clinical training, education, and workflows. To enable the development of AI tools for surgery, there is a strong need for multimodal surgical data encompassing patient-specific, video, imaging, kinematic, and contextual information collected from pre-, intra-, and post-operative sources [1]. Additionally, the tactile internet - enabling real-time, ultra-low latency interaction - offers new possibilities for medicine and surgery, while also posing similar technical challenges like large data volumes and complex metadata handling [2]. This vast amount and scope of data come with significant technical and ethical challenges to fully realize its full potential. Technical challenges include infrastructure for data acquisition, storage, access, annotation, sharing, and analytics within regulatory constraints [3]. In this study, we focused on developing solutions for three research facets, i.e., instrumentation, software, and data management (RISD-M) in our institution. Specifically, the (1) clinical video and patient data from our surgical department and (2) synchronized data from multiple novel actuators, sensors, and video streams collected during pre-clinical experimental surgeries for research on cognitive robotics and the tactile internet [4], [5]. Here, we present our collaborative and multidisciplinary approach to developing comprehensive data management solutions for these diverse and large datasets. These solutions are custom developed software platforms, with a particular focus on the storage, sharing, and curation of clinical and experimental data according to the FAIR principles. The platform's functionality includes the collection, storage, and access to raw data files and synchronized data streams, which are appropriately contextualized by comprehensive metadata. The metadata is organized in NoSQL databases, facilitating efficient and rapid data retrieval using search terms. These platforms have been instrumental in streamlining workflows for clinician scientists, medical students, computer scientists, and engineers engaged in diverse research projects of the Centre for Tactile Internet with Human-in-the-Loop (CeTI) and the University Hospital Carl Gustav Carus at the TUD Dresden University of Technology. The platform's capacity to facilitate the rapid development of AI solutions for surgery is also expected, as the data is now FAIR and ready for machine learning applications. Despite the initial success of the platforms, further challenges still need to be addressed. Subsequent work will focus on standardization efforts, expansion, and merging of the platforms, and further development of ethical and legal channels for the dissemination of the data. This presentation will demonstrate the impact of robust user-oriented solutions to overcome some of the technical challenges facing the field of surgical data science and tactile internet in surgery.