Multi-Task Learning for Hierarchical Professional Gesture Recognition: State-Space Modeling for Task Temporal Dependencies

Human gesture recognition plays an important role in professional environments and applications such as industrial automation and human-machine collaboration. Standard Single-Task Learning (STL) architectures, while effective for gesture recognition, appear robust only in curated settings, lacking the ability to generalize in real-world scenarios due to the inherent variability of human movement. Multi-Task Learning (MTL) appears as method able to enhance generalization by exchanging knowledge among tasks, without taking into consideration the temporal dependencies and stochasticity of human movement. This work addresses these limitations, by introducing a hierarchical structure that decomposes professional movements into three levels and an Autoregressive State-Space Loss (ASSL) function that introduces stochasticity to the model. Experiments on real-world datasets demonstrate that MTL with both the hierarchical structure and the ASSL function provide stability and pave the way for new learning approaches.