A Calibrated Multi-Dimensional Evaluation Framework for Diffusion-Based Radio Frequency Signal Generation

Evaluation of generative models for RF (Radio Frequency) signals remains largely ad hoc, with existing approaches relying on uncalibrated metrics imported from computer vision without systematic justification or baseline establishment. We present a multi-dimensional, distribution-level evaluation framework comprising ten metrics across six layers, calibrated against real-real baselines. A real-real baseline is computed by splitting authentic signals into two independent subsets and measuring the same metric between them; the resulting value sets the achievable ceiling for that metric, against which generated-vs-real scores are normalized. The framework is grounded in four design principles: distribution-level aggregation, real-real baseline normalization, signal-to-noise ratio (SNR)-modulation stratification, and multi-dimensional coverage. Application of the framework reveals two previously unreported phenomena. First, linear Short-Time Fourier Transform preprocessing creates a gradient imbalance between frequency-domain and time-domain loss components that causes catastrophic generation failure for analog amplitude modulation; logarithmic compression resolves this. Second, the widely adopted noise-prediction training objective exhibits systematic gradient suppression for amplitude-modulated signals due to SNR-dependent implicit loss weighting; switching to signal-prediction substantially improves temporal structure fidelity for amplitude-modulated signals, with only modest trade-offs for digital communication signals. The framework and calibration methodology establish reproducible standards for comparative assessment of RF generative models.