Deep Learning with an Attention Mechanism for Enhancing Automated Modified Total Sharp/van der Heijde Scoring of Hand X-ray Images in Rheumatoid Arthritis

Abstract Purpose Rheumatoid arthritis (RA) is a chronic autoimmune disease whose accurate diagnosis relies on clinical symptoms, serological markers, and imaging studies, particularly X-ray evaluations of joint damage. A widely adopted metric for evaluating RA severity in hand and foot radiographs, the modified Total Sharp Score, quantifies joint erosion and joint space narrowing. However, manual scoring is labor-intensive, time-consuming, and prone to variability, limiting its reliability and consistency in clinical practice. Methods This study introduces a deep learning–based approach augmented with an attention mechanism to automate joint localization and damage classification in hand X-rays of patients with RA. The You Only Look Once v7 model was used for joint localization, incorporating performance enhancements via window-level transformation and the Distance Intersection over Union algorithm. Results The model achieved a localization accuracy of 99.66% [email protected], significantly enhancing the detection precision. This model outperformed the vision transformer in scenarios involving limited data and small feature regions, achieving an overall classification accuracy of 88%. Conclusion The proposed method improved feature learning and classification, especially with limited data, and the attention mechanisms significantly enhanced the performance of the RA automatic diagnosis system. Data imbalance was effectively addressed via a modified EfficientNetV2 model with an integrated attention mechanism for joint damage classification.