On the Encapsulation of Medical Imaging AI Algorithms

In the context of collaborative AI research and development projects, it would be ideal to have self-contained encapsulated algorithms that can be easily shared between different parties, executed and validated on data at different sites, or trained in a federated manner. In practice, all of this is possible but greatly complicated, because human supervision and expert knowledge is needed to set up the execution of algorithms based on their documentation, possibly implicit assumptions, and knowledge about the execution environment and data involved. Focusing on medical imaging AI algorithms, we derive and formulate a range of detailed requirements from the above goal and from specific use cases: * Algorithms that are bundled with enough metadata and that support open, standardized interfaces can be distributed via more or less public channels or "app stores". * We consider the reuse of algorithms by colleagues or collaborators, but also running published algorithms on new datasets as a reference for benchmarking new algorithms. * A related use case are medical image computing challenges – competitions whose organizers would like to run algorithms submitted by participants on secret test data. * Finally, medical applications often come with strong data protection requirements that motivate moving algorithms to the data (often within closed hospital IT systems) instead of the other way round. Such applications (e.g., federated or swarm learning) are another strong motivation for algorithms that do not require much manual interaction and IT knowledge to set up but come with machine-readable meta-information that allows appropriate execution environments to run them on the desired data. Furthermore, we refer to a number of existing APIs and implementations –challenge platforms, federation frameworks, AI model repositories, research software, application ecosystems, and data management platforms –- and review which aspects each of them addresses (comprising identity, source, purpose, inputs, and outputs), which problems are still open, and which public standards and ontologies are relevant. To conclude, our contribution is a comprehensive collection of aspects that have not yet been addressed in their entirety by any single solution. Most reviewed approaches cover only basic aspects such as the identity and source of algorithms to varying degrees, usually sufficiently well, but sometimes limited in practice by lack of curation or review leading to many missing fields. The mhub.ai repository for medical imaging AI models and the EMPAIA ecosystem for histopathology diagnostics stand out by offering the most detailed model of algorithm inputs and outputs, allowing execution environments to determine automatically which algorithms can be run on which data, and to interpret and visualize results appropriately. On the other hand, these systems require significantly more work than just uploading some trained model for packaging the algorithm to match the respective specification. Further work on standardized algorithm interfaces and specifications for descriptions of encapsulated algorithms is needed and should lead to more sustainable algorithm ecosystems. This also relates to the FAIR principles for research data and software, while this work focused on interoperability and (re)usability of medical imaging AI algorithms.