Multimodal Dermatological Disorder Assessment via Conversational Bot Integration for Textual and Visual Diagnostic Interpretation

The skin of a person is a good sign of their health because it often shows the first signs of problems with their internal organs. It is very important to notice these signs early on so that you can get a diagnosis and treatment. People sometimes forget how important the skin is as a defence mechanism for the body. The goal of this research is to make a multimodal skin disease classification system that uses a Telegram chatbot to combine ensemble transfer learning models (DenseNet169, ResNet50, EfficientNetV2, and Swing Transformer) with Natural Language Processing (NLP). The main goal is to make the chatbot better at giving personalized and correct skin-related diagnoses based on information from users, such as their skin type, chemical exposure, and past treatments. The system makes diagnoses more accurate and personalised by combining both image-based analysis and textual contextual information. EfficientNetV2 helps speed up computations and extract high-resolution features, while Swing Transformer uses hierarchical vision transformers to learn both global and local features so that it can work with different skin conditions. In addition, geospatial mapping (MAP Integration) shows the number and location of skin disease cases so that epidemiological analysis and public health information can be gathered. The chatbot can learn on its own, which means it can change how it responds to users based on how they interact with it. This keeps users interested over time. The hybrid strategy makes the most of fine-grained features and understanding the context, which leads to great classification performance. We tested 11,747 images, of which 7,930 were used for training and validation and 3,817 for testing. The proposed model achieved 77.07% accuracy and an AUC of 96.72% for image classification, whereas the NLP model attained 93.62% accuracy, delivering a comprehensive and tailored diagnostic experience.