FedEmo: A Federated Learning Framework for Privacy-Preserving Emotion Detection From Handwriting on Consumer IoMT Devices

Emotion detection from handwriting and drawing presents a promising yet underexplored avenue for scalable mental health monitoring. This is particularly relevant within consumer-centric Internet of Medical Things (IoMT) ecosystems, where privacy and cross-institutional data sharing remain critical challenges. This paper proposes FedEmo, a privacy-preserving federated learning framework that leverages an attention-based transformer model to analyze handwriting and drawing samples on edge devices, while adhering to Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR) regulations. The model processes stroke-level features (e.g., pen pressure, speed, and direction during handwriting or drawing tasks), which are key indicators of emotional states, through self-attention mechanisms, achieving 92.64% accuracy on the EMOTHAW dataset under centralized training. A federated protocol enables distributed model refinement without sharing raw data, maintaining 87.3% accuracy in simulated non-Independent and Identically Distributed (non-IID) settings, consistent with existing federated learning benchmarks. The framework introduces a hybrid cloud-edge deployment that reduces communication bandwidth by 58% through local embedding computation, and supports a clinician alert system with a modeled end-to-end latency of 620ms. Experimental results confirm the system’s robustness under typical IoMT constraints, including 15% packet loss and 100kbps bandwidth. FedEmo offers a scalable, privacy-compliant solution for real-time emotion recognition and remote mental health diagnostics using consumer-grade IoMT devices, with potential applications in telepsychiatry and early screening for depression and Parkinson’s disease.