Multimodal Deep Learning with XAI for Transparent and Accurate Doctor Prescription Recognition

This paper presents a deep learning framework to automate recognition of medical prescriptions by using handwritten content, feature extraction from CNNs like VGG16, Alex Net, and MobileNetV2, and sequential data utilizing LSTMs. A multi-modal approach captures even the spatial and temporal details in a prescription, which assists in an accurate interpretation of complex handwriting. Specifically, key pre- processing like grayscale conversion, normalization and edge detection improving the quality of the image, reducing noise and bringing forth the prominent features that aid in the improvement of recognition accuracy and also implementation of XAI concepts to further enhance the interpretation. This model reduces transcriptions, making it easier for the patients as well as the practitioners to understand the handwriting through the computer screen. The method therefore assists in safer patient-centered healthcare by automating prescription processing thus triggering clearer communication towards more informed healthcare decisions.