A Comparison of Object Detection and Phrase Grounding Models in Chest X-Ray Abnormality Localization Using Eye-Tracking Data

Chest diseases rank among the most prevalent and dangerous global health issues. Object detection and phrase grounding deep learning models interpret complex radiology data to as-sist healthcare professionals in diagnosis. Object detection locates abnormalities for classes, while phrase grounding lo-cates abnormalities for textual descriptions. This paper inves-tigates how text enhances abnormality localization in chest X-rays by comparing the performance and explainability of these two tasks. To establish an explainability benchmark, we proposed an automatic pipeline to generate image regions for report sentences using radiologists' eye-tracking data<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>. The better performance - mIoU = 36% vs. 20% - and ex-plainability - Containment ratio 48% vs. 26% - of the phrase grounding model infers the effectiveness of text in enhancing chest X-ray abnormality localization.