Activity Recognition for Mechanical Systems with Complex Kinematics

Activity recognition is one of the major tasks in computer science and engineering that aims to recognize and understand the actions of one or more agents from a series of observations. It assists different sectors in ensuring safety in human-machine interaction. This paper investigates and tests an encoder classifier based on an LSTM model with various attention mechanism structures over a synthetic time-series motion dataset generated with the NVIDIA Isaac simulator. The simulator simulates the motion of an excavator as an example of a complex kinematic system in a construction environment, gathering data from position sensors connected to the excavator’s main joints. The results show that fewer sensors can be used for certain types of motion classification with a large accuracy range between 66.7% and 83.3%. The encoder-LSTM model with scaled dot-product attention gives the most accurate results compared to other attention mechanism types, with around 33.4% when using data from only two sensors and around 22.2% for the four main sensors. These results are in comparison to models that used other attention mechanisms.