SkinSavvy2: Augmented Skin Lesion Diagnosis and Personalized Medical Consultation System

The shortage of medical personnel and the busy lives of modern people have increased the desire for the self-diagnosis of diseases, and the latest large-scale language models and image recognition technologies have the potential to meet this demand. In particular, skin-related diseases are one of the areas where symptoms are visually distinguishable, making self-diagnosis and care possible. In this paper, we propose a system that classifies diseases through images of skin diseases and combines them with individual conditions such as age, skin type, and gender for self-diagnosis. First, we design the latest deep learning model-based skin disease classifier that can classify six types of skin diseases using the HAM10000 dataset and generate prompts by combining the personal information input. By utilizing the Generative Pre-trained Transformer (GPT) model, the system generates personalized care methods based on these prompts. We measured the accuracy of the classification model of the proposed system and validated the effectiveness of the proposed method through user evaluations.