Learning to Design Constellation of MLC With Geometric Shaping

This letter proposes a constellation design of geometric shaping (GS) for multilevel coding (MLC) and bit-interleaved coded modulation (BICM) systems with high spectral efficiency. In order to optimize the GS of constellation, our proposed autoencoder based on deep learning is composed of parallel-connected encoders and serial-connected decoders, fully learning the characteristics of MLC. Then the optimized constellations are applied in the MLC scheme and compared to quadrature amplitude modulation (QAM) with square constellations (SQCs) and ATSC 3.0 non-uniform constellations (NUCs). It is shown that the decoding performance of MLC scheme using our GS optimized constellation is close to that of the NUC-based BICM with nearly half of the decoding complexity. Moreover, our simulation results demonstrate that the MLC scheme with trained constellation can achieve better gain for higher order modulations, such as 2<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">10</sup>-QAM.