Ellipse fitting meets AI-RAN: Robust non-parametric compensation of joint transmitter and receiver in-phase and quadrature imbalance

In-phase and Quadrature (IQ) modulation is a cornerstone of modern wireless communication systems, underpinning high-order modulation formats and spectrally efficient transmission. However, inevitable analog imperfections in RF front-ends introduce amplitude mismatches and phase deviations, leading to constellation distortion and severe degradation in demodulation performance. The challenge becomes particularly acute when IQ imbalances arise simultaneously at both the transmitter (Tx) and receiver (Rx), as conventional receiver-side compensation schemes are unable to cope with the compounded impairments. This problem is of growing significance in the emerging paradigm of AI-driven Radio Access Networks (AI-RAN), where reliable physical-layer signals form the foundation for data-driven control and optimization in RAN Intelligent Controllers (RICs). To address these issues, we propose an efficient correction framework based on ellipse fitting and affine transformation, which avoids explicit estimation of imbalance parameters. The distorted constellation is geometrically fitted to an ellipse, transformed to restore circular symmetry, and refined through phase correction. Simulation results with 16-QAM modulation demonstrate that the proposed method achieves performance comparable to conventional schemes in receiver-only scenarios while exhibiting superior robustness to joint Tx/Rx distortions. By delivering clean and interpretable signal constellations with low computational complexity, the approach not only preserves physical-layer fidelity but also enhances the reliability of AI-RAN workflows, positioning impairment correction as a lightweight yet critical enabler for next-generation intelligent wireless networks.