Using Microwave Simulations for the Generation of Radar Data for Gesture and Activity Recognition: Potential and Challenges for Real-World Applications

In recent years, algorithms have been successfully developed to recognize a wide range of activities and gestures with millimeter-wave radar sensors. Applications range from patient monitoring in health care <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> to contactless gesture control of smart devices <xref ref-type="bibr" rid="ref2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[2]</xref> and traffic gesture recognition by autonomous vehicles <xref ref-type="bibr" rid="ref3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[3]</xref>. All these approaches involve machine learning (ML) models that first extract valuable information from the radar data and then use it to make a prediction about the observed human motion. The design of the involved neural networks often requires the acquisition of an application-specific training dataset. Since the acquisition of large amounts of gesture or activity observations is very laborious, typical radar datasets have a few hundred to a few thousand samples per class <xref ref-type="bibr" rid="ref4" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[4]</xref>, <xref ref-type="bibr" rid="ref5" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[5]</xref>, <xref ref-type="bibr" rid="ref6" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[6]</xref>, <xref ref-type="bibr" rid="ref7" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[7]</xref>. The small number of samples per class prevents the use of very deep neural networks with many layers and millions of trainable parameters. If the neural network becomes too powerful for a given dataset size, it will start to simply memorize the data from training and be unable to correctly classify new samples, a problem known as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">overfitting</i>. The authors of <xref ref-type="bibr" rid="ref8" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[8]</xref> and <xref ref-type="bibr" rid="ref9" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[9]</xref> train a convolutional neural network (CNN) with increasing depth on a small activity dataset and observe severe overfitting as the model becomes too powerful. This prevents the further increase in accuracy that would be expected from deeper neural networks.