Evaluating dual-path temporal fusion strategies for multi-modal hand gesture recognition under limb-position variability

<i>Objective.</i>Wearable biosignal-based hand gesture recognition is a key enabling technology for prosthetic hand control, but its reliability is often affected by limb-position variability and other real-world confounding factors. This study aims to systematically compare dual-path temporal fusion architectures for co-located surface electromyography (sEMG) and pressure-based force myography (pFMG), with emphasis on robustness, computational efficiency, and interpretability under conditions relevant to practical prosthetic use.<i>Approach.</i>Three dual-path fusion temporal convolutional network (DFF-TCN) architectures were structured and investigated to integrate sEMG and pFMG using different fusion strategies: (1) a baseline concatenation-based model combining decision-level (DL) and feature-level (FL) fusion, (2) a DL cross-attention variant, and (3) a FL cross-attention variant. All models were evaluated under identical training and testing protocols using a custom dataset collected from ten participants performing nine functional hand gestures across multiple static and dynamic arm positions.<i>Main results.</i>Across all evaluated conditions, the concatenation-based DFF-TCN achieved balanced performance, with a mean classification accuracy of 95.88%, while the attention-based variants achieved accuracies of 90.65% and 94.02%, respectively. Computational profiling showed that the concatenation-based model also achieved the lowest inference latency (1.70 ms), indicating suitability for real-time deployment. Explainable artificial intelligence analysis using integrated gradients revealed complementary contributions from sEMG (54.08%) and pFMG (45.92%), with contribution patterns varying across gestures and subjects.<i>Significance.</i>The results demonstrate that different fusion strategies offer distinct trade-offs between recognition performance, computational cost, and robustness. In particular, the concatenation-based model provides a favorable balance for real-time prosthetic hand control, while attention-based variants offer additional modeling flexibility. These findings provide practical guidance for selecting multi-modal fusion architectures in wearable human-machine interface systems and support the continued use of co-located sEMG-pFMG sensing in prosthetic and rehabilitation applications.

Authors

Institutions

Publication Details

Journal
Journal of Neural Engineering
Published
2026-02-01
DOI
https://doi.org/10.1088/1741-2552/ae4653
Primary Topic
Muscle activation and electromyography studies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Evaluating dual-path temporal fusion strategies for multi-modal hand gesture recognition under limb-position variability

Gürsel Alıcı, Rayane Tchantchane, Shen Zhang, Hao Zhou
Journal of Neural Engineering
Muscle activation and electromyography studies
article

Evaluating dual-path temporal fusion strategies for multi-modal hand gesture recognition under limb-position variability

Gürsel Alıcı, Rayane Tchantchane, Shen Zhang, Hao Zhou
article en

Abstract

<i>Objective.</i>Wearable biosignal-based hand gesture recognition is a key enabling technology for prosthetic hand control, but its reliability is often affected by limb-position variability and other real-world confounding factors. This study aims to systematically compare dual-path temporal fusion architectures for co-located surface electromyography (sEMG) and pressure-based force myography (pFMG), with emphasis on robustness, computational efficiency, and interpretability under conditions relevant to practical prosthetic use.<i>Approach.</i>Three dual-path fusion temporal convolutional network (DFF-TCN) architectures were structured and investigated to integrate sEMG and pFMG using different fusion strategies: (1) a baseline concatenation-based model combining decision-level (DL) and feature-level (FL) fusion, (2) a DL cross-attention variant, and (3) a FL cross-attention variant. All models were evaluated under identical training and testing protocols using a custom dataset collected from ten participants performing nine functional hand gestures across multiple static and dynamic arm positions.<i>Main results.</i>Across all evaluated conditions, the concatenation-based DFF-TCN achieved balanced performance, with a mean classification accuracy of 95.88%, while the attention-based variants achieved accuracies of 90.65% and 94.02%, respectively. Computational profiling showed that the concatenation-based model also achieved the lowest inference latency (1.70 ms), indicating suitability for real-time deployment. Explainable artificial intelligence analysis using integrated gradients revealed complementary contributions from sEMG (54.08%) and pFMG (45.92%), with contribution patterns varying across gestures and subjects.<i>Significance.</i>The results demonstrate that different fusion strategies offer distinct trade-offs between recognition performance, computational cost, and robustness. In particular, the concatenation-based model provides a favorable balance for real-time prosthetic hand control, while attention-based variants offer additional modeling flexibility. These findings provide practical guidance for selecting multi-modal fusion architectures in wearable human-machine interface systems and support the continued use of co-located sEMG-pFMG sensing in prosthetic and rehabilitation applications.

Journal of Neural EngineeringVol. 23(1)
University of Wollongong (AU)
Openalex Percentile: Top 55%
Muscle activation and electromyography studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.