Few-shot generative adversarial network for radar micro-doppler spectrogram synthesis with application to human activity recognition

Generative adversarial networks (GANs) have been recently introduced to synthesize radar micro-Doppler spectrograms for data augmentation. However, traditional GANs easily overfit with limited training data and they are hard to ensure that the synthetic spectrograms have high kinematic fidelity (e.g., continuous activity is suddenly interrupted in the time dimension). To address these issues, we propose a physics-aware few-shot generative adversarial network (PFGAN) to synthesize high-quality spectrograms with limited radar data. Our main contributions are two-fold. First, we design an attention ranking-based local fusion module (ARLFM). ARLFM learns to select important local features for matching and replacement, which considers the distribution characteristics of micro Doppler signatures. Second, we improve the kinematic fidelity of synthetic spectrograms by designing a multi-scale envelope-extraction module (MEM) where three types of spectrogram envelopes are extracted at different resolutions to reflect physical motion information. Experiments show that PFGAN can generate diverse spectrograms using few samples with high kinematic fidelity. The effectiveness of synthetic spectrograms is also demonstrated for human activity recognition tasks.