Pseudo tongue data generation using statistical tongue model and TSC-GAN

Tongue diagnosis is one of the main observations in Traditional Chinese Medicine (TCM), which is widely used in clinical practice. The size and shape of the tongue are crucial characteristics for the diagnosis of the tongue. Recently, deep learning-based methods have been proposed for the classification and segmentation of tongue images. However, the limited annotated data may lead to overfitting problems. In this study, we propose a statistical model-based method for generating a tongue body mask generation method and an imageto-image translation and classification network- TSC-GAN. The idea is that using PCA to analyze the statistic of tongue body and build a deformable tongue body model, which can generate unlimited tongue body masks by adjusting the weights of the principle components. Then a multitask GAN-based network TSC-GAN is proposed to generate pseudo-tongue images and corresponding size and shape annotation from the tongue body mask generated by statistical tongue model (STM). Two tongue datasets, i.e. FJTCM-SZU and BioHit, are employed for evaluating the data augmentation performances. The experimental results show that employing STM and TSC-GAN for data augmentation significantly improves the performance of both tongue segmentation and size-and-shape classification, especially on small-sized datasets.