Integrating Soft Computing and Multi-Agent for Action Recognition: Basics, Challenging and Future Directions

Abstract Human activity recognition (HAR) represents a significant area of research within the domain of computer vision, which has been extensively explored, yet it faces significant challenges including real-world variability, fine-grained discrimination, computational efficiency, and robust multi-modal data fusion. Traditional “hard computing” techniques frequently find it difficult to cope with the intrinsic imprecision, uncertainty, and ever-changing aspects of human behavior. This study commences with a broad overview of the HAR framework, detailing the distribution of machine learning (ML) and deep learning (DL) in HAR, as well as providing an outline of the recent HAR datasets. Further, the study offers a comprehensive overview of the synergistic combination of Soft Computing (SC) paradigms and Multi-Agent Systems (MAS) as a robust strategy to overcome these challenges in HAR. Further, the study presents a new problem-oriented taxonomy that categorizes HAR challenges into three distinct groups: sensing challenges, recognition challenges, and scalability & robustness challenges. Moreover, the study primarily investigates the integration of these two domains and how they yield innovative solutions to challenges in HAR. The final section outlines the existing challenges within this integrated research domain and highlights potential future directions, which encompass sophisticated neuro-fuzzy fusion techniques, self-organizing multi-agent learning for HAR, and the creation of explainable and resilient HAR systems.