I Sense You Fast: Simultaneous Action and Identity Inference by Slimming Multi-Branch RadarNet

With the increasing connection between internet and human society, millimeter-wave radar based action recognition and user authentication exhibit remarkable prospects in security scenarios. Existing solutions usually focus on one of the tasks and mainly emphasize accuracy without reducing the inference time. In this paper, we propose a dual-task based Polymorphic Lightweight (PolyLite) RadarNet framework, in which the shared features are fed into two split streams for different tasks under joint supervision. The polymorphic concept here means that the trained network with parallel designs can be slimmed as a single-branch structure for inference. By this design strategy, we can not only efficiently extract spatial-temporal features during the training stage but also largely improve the response speed for simultaneously testing human activities and identities. Specifically, we design triple-view (TRIview) video-like data as the input by successively concatenating the range-velocity and range-angle matrices. Then a PolyLite module with linear and lightweight designs in each branch is integrated into our RadarNet framework to learn discriminative representations. Experimental results demonstrate that our approach is able to reach the accuracy over 98<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\%}$</tex-math></inline-formula> within 0.21ms inference time. Especially, untrained intruders can also be successfully identified by a simple matching computation. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MagicalLiHua/PolyLite-RadarNet</uri>.