Workload Estimation for a Military Ground Vehicle Crew using Supervised Machine Learning of FACS Action Unit Intensity Data

<div class="section abstract"><div class="htmlview paragraph">The proliferation of intelligent technologies in the future battlefield necessitates an exploration of crew workload balancing strategies for human-machine integrated formations. Many current techniques to measure cognitive workload, through qualitative surveys or wearable sensors, are too brittle for the harsh, austere operational environments found in military settings. Non-invasive workload estimation techniques, such as those that analyze physiological effects from video feeds of the crew, present a way forward for workload-aware Soldier-machine interfaces that could trigger events – such as task reallocation – if limits on crew or individual workload are exceeded. One such technique that is being explored is the use of facial expression analysis for workload estimation. We present the performance results of regression and classification models developed from supervised machine learning algorithms that predict pNN50, a common heart rate variability metric used as a physiological measure for workload, from action unit intensity data of the Facial Action Coding System (FACS). Drawing from these results, we propose implementation recommendations for leveraging facial expressions to inform crew workload in workload-aware Soldier-machine interfaces. We conclude with a discussion on open challenges and areas of exploration for non-invasive workload estimation in military vehicle applications.</div></div>