OsiriXGPT: An Innovative AI Co-pilot Plug-In for Seamless Deployment of Generative AI Models in Scan-to-Scan Reporting Workflows

Generative Artificial Intelligence (GenAI) has the potential to transform radiology by reducing reporting burdens, enhancing diagnostic workflows and facilitating communication of complex radiological information. However, research and adoption remain limited due to the lack of seamless integration with medical imaging viewers. This study introduces OsiriXgrpc, an open-source API plug-in that bridges this gap, enabling real-time communication between OsiriX, a CE-marked and FDA-approved DICOM viewer, and AI-driven tools deployed in any supported programming language (e.g., Python). OsiriXgrpc's design provides users with a unified platform to query, interact with, and visualise AI-generated outputs directly within OsiriX. To demonstrate its potential, we developed an AI Co-pilot for radiology that leverages OsiriXgrpc for iterative "request-to-answer" interactions between users and GenAI models, allowing real-time data queries and AI-generated output visualisation within the same DICOM viewer. We have adapted OsiriXgrpc to allow users to: (i) interrogate Foundation Large-Language Models (LLMs) to generate text from text-based prompts, (ii) employ Foundation Vision-Language Models (VLMs) to generate text by combining text and image prompts, and (iii) employ a one-click Foundation AI-driven segmentation model to generate Regions of Interest (ROIs) by combining points/bounding boxes with text prompts. For this proof-of-concept report, we applied OpenAI's LLMs and VLMs for text generation and the Segment Anything Model (SAM) for generating ROIs. We provide evidence for successful implementation of the plug-in, including visualisation of the AI-generated outputs for each model tested. We hypothesise that OsiriXgrpc can lower adoption barriers, facilitating GenAI models integration into clinical trials and routine healthcare, even in resource-limited settings, including low/middle income countries (LMICs).