Using deep vision-language models improves multi-task performance in assistance applications for endoscopic ENT surgery
We developed a vision-language model capable of integrating image and text data for endoscopic ENT assistance tasks, that is able to replace three isolated models, delivering multi-task assistance while outperforming prior and general‑purpose baselines. Remaining challenges include the handling of imbalanced class distributions and limited gains on templated structured text.
Authors
- Richard Bieck (ORCID: https://orcid.org/0000-0003-0950-0637)
- Markus Pirlich (ORCID: https://orcid.org/0000-0002-8581-0453)
- Martin K. Sorge
- Thomas Neumuth (ORCID: https://orcid.org/0000-0001-6999-5024)
- Viktor Kunz (ORCID: https://orcid.org/0000-0002-2415-7841)
- Katharina Heuermann
Institutions
- Leipzig University (DE)
Publication Details
- Journal
- International Journal of Computer Assisted Radiology and Surgery
- Published
- 2025-12-22
- DOI
- https://doi.org/10.1007/s11548-025-03512-z
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Bundesministerium für Bildung und Forschung
- Universität Leipzig