Knowledge Distillation with Student-Teacher Model for Whole Heart and Great Vessel Segmentation in Congenital Heart Disease Using CT-Scan Image

Congenital Heart Disease (CHD) results from imper-fect heart anatomy during early fetal development. According to the Perhimpunan Dokter Spesialis Kardiovaskular Indonesia (PERKI), around 8–10 babies out of every 1000 births in Indonesia suffer from CHD. However, with advances in medical technology, many children with heart defects can survive. Cardiac CT-Scan images are used as the basis for appropriate treatment, while image analysis allows evaluation of abnormalities and monitoring of disease progression posttherapy. Developing a system with lightweight computing and good performance that can segment the cardiac CT scan image to support the doctor's analysis is necessary. For more specific or complex tasks, the model will use a huge memory. Because the model should use more parameters. Bigger memory that model used, more difficult to implement in the device with limited storage. By reducing the number of parameters and removing redundant connections, it is possible to improve model performance while saving computational load significantly. This research applies Knowledge Distillation (KD), which is a model compression method where knowledge from the teacher model is transferred to the student model. KD aims to teach student models to generalize based on the teacher model's knowledge. This research uses the U-Net architecture as the teacher model and the CFPNet-M architecture as the student model. The KD model performs better IoU than the student model, 0.872 or 3% higher. Judging from the resulting model file size, the KD model file is 131.4MB or 33.33 % smaller than the teacher model file size.