Utilizing knowledge distillation and incremental learning for osteoporosis detection on knee X-ray images

Osteoporosis is a bone disease that mostly affects malnourished and elderly people by reducing their bone mass, resulting in higher chances of bone fractures which are difficult to recover from and sometimes prove fatal. To detect the disease, a doctor may order a Bone Mineral Density (BMD) test to detect osteoporosis and low bone density. This is commonly done using dual-energy x-ray absorptiometry (DXA), which uses multiple low energy X-rays to measure the BMD of our skeleton. Routine screening for osteoporosis is a time-consuming and laborious task, requiring specialized equipment not available in rural areas. According to the American Medical Association (AMA) Survey conducted in 2024, almost 66% of the physicians are using computer-aided diagnosis to ensure in-time disease diagnosis, that is about 28% high as compared to the previous fiscal year i.e., 2023. However, there are few X-ray osteoporosis datasets that are publicly available to train on. In this paper, an automated system has been proposed that detects osteoporosis from X-ray images of Knee and utilizes knowledge distillation and incremental learning techniques to improve the generalizability of DL models across the two datasets that are publicly available. Knowledge distillation is utilized to distill the patterns learned by teacher models onto smaller student models and incremental learning techniques are utilized to overcome catastrophic forgetting when training on multiple datasets. However, experimentation showed that knowledge distilled models had no significant impact and that the best performance with Class Incremental Learning was by training DenseNet-121 using Learning without Forgetting which resulted in an accuracy of 62.6% with a performance gap of 30% when trained on direct 3-class data (92.9%) indicating that direct training is still essential for achieving high accuracy and class incremental learning is just a constrained baseline for data-limited environments.