A Modality-Dropout Cross-Attention Transformer for Robust Pneumonia Detection Using Chest X-Ray Images and Clinical Notes
Automated pneumonia detection from chest radiographs is clinically important, but image-only systems can be brittle when radiographic findings are subtle, labels are report-derived, or contextual clinical evidence is unavailable at deployment. We propose a Modality-Dropout Cross-Attention Transformer (MDCAT), a multimodal architecture that combines chest X-ray image tokens with temporally eligible clinical-note or ED-text tokens using bidirectional crossattention and trains under explicit missing-modality simulation. The model includes modality-availability embeddings, learned absent-modality tokens, auxiliary unimodal heads, and a consistency term that aligns masked-modality predictions with full-input predictions when both modalities are observed. Because pneumonia labels in common chest X-ray datasets are often derived from radiology reports, we specify a leakage-controlled data protocol that separates preimaging clinical text from same-study radiology reports and treats report text only as a secondary ablation. The evaluation design covers full multimodal inference, image-only inference, text-only inference, stochastic modality dropout, real missingness, image corruption, note truncation, calibration, and external image-only validation
Authors
- Sowjanya Jindam (ORCID: https://orcid.org/0000-0001-6959-2110)
- Aniketh Chittiprolu
- Sathwika Reddy Gundra
- Rachamalla Surya Prakash
- Hari Shankar Punna
Publication Details
- Journal
- International Journal of Drug Delivery Technology
- Published
- 2026-11-14
- DOI
- https://doi.org/10.25258/ijddt.16.6.155
- Primary Topic
- COVID-19 diagnosis using AI
- Type
- article
- Field-Weighted Citation Impact
- 0.00