ViTZU-LoRA: A Diagnostic Research Method for Tongue Image

Tongue diagnosis is based on medical theory. By observing the changes in the tongue appearance, it can detect abnormalities in the body’s qi and blood as well as the functions of internal organs. It plays an irreplaceable role in the early warning of diseases and the dynamic monitoring of the condition, and is an important reference basis for medical clinical diagnosis and disease prevention. However, in the traditional medical methods of “observation, auscultation and olfaction, inquiry, and palpation”, symptom assessment can have an impact on the diagnostic results due to the subjective differences of doctors. This paper proposes a tongue image diagnosis method based on deep learning, the dualmode collaborative diagnosis method (ViTZU-LoRA), which realizes multi-scale feature extraction of tongue images through the improved U-Net architecture, including the dual convolutional units with residual connections and the gated dynamic fusion mechanism. And combined with the dual-branch processing of Vision Transformer for HSV color statistics, histogram features and GLCM four-direction texture features for analysis and processing. The experimental results show that this method achieves a Dice coefficient of 0.9525 in the tongue body segmentation task, and the overall accuracy rate of the classification task reaches 0.9947, verifying the effectiveness of LoRA lightweight fine-tuning and knowledge injection of Zhongjing Medical’s large model.