Zero-Shot Diagnosis of Unseen Pulmonary Diseases via Spatial Domain Adaptive Correction and Guidance by ChatGPT-4o

In medical image analysis, visual-language models facilitate zero-shot diagnosis by assessing the similarity between images and textual disease descriptions. However, challenges such as multi-center data variability and fine-grained lesion recognition constrain their full potential. To address these limitations, we propose a method that combines spatial domain adaptive preprocessing of images with the knowledge distillation capabilities of ChatGPT4o. Specifically, we developed a Spatial Domain Adaptive Correction module for multi-center medical images. This module analyzes the spatial pixel distribution and applies adaptive enhancement strategies to optimize image quality, thereby restoring details and enhancing edge features. Second, disease description texts are distilled using ChatGPT-4o and subsequently revised by radiology experts. These texts are further enhanced with lesion location and morphology information derived from large-scale chest imaging reports, and guiding words are added to emulate the case report style. Finally, text and image features are extracted using separate encoders, and prediction probabilities for various disease types are calculated based on cosine similarity. We validated our method on seven datasets containing three unseen lung diseases. The results demonstrate superior performance and significant clinical application potential in diagnosing unseen and rare diseases. The experimental code has been open-sourced at https://github.com/wtf633/Zero-Shot-Diagnosis-of-Unseen-Pulmonary-Diseases.