Safe image transmission with hybrid quantum key-genetic encryption via spatial modulation assisted NOMA

This manuscript presents a secure and spectrally efficient Spatial Modulation (SM)-based Non-Orthogonal Multiple Access (NOMA) system for two-user downlink 5G wireless communication. The proposed SM-NOMA shows significant improvements in spectral efficiency and energy savings by activating only one transmit antenna per symbol duration and all other remaining antennas remain silent, reducing hardware complexity and RF chain usage compared to traditional Multiple Input-Multiple Output (MIMO) systems. In terms of detection strategy, the weak user employs Maximum Ratio Combining (MRC) to directly decode its signal, as it receives higher power and does not need interference cancellation. Conversely, the strong user first applies MRC to detect the weak user's signal, performs Successive Interference Cancellation (SIC) to remove the weak user's contribution, and then applies MRC again to decode its own signal. This approach balances complexity and performance, offering practical implementation benefits. Additionally, the system includes optimized power allocation to manage interference and maintain user fairness. The proposed SM-NOMA system performance is evaluated using Bit Error Rate (BER), outage probability, and the spectral efficiency under imperfect CSI and imperfect SIC conditions. SM-NOMA outperforms conventional NOMA by significantly reducing BER, lowering outage probability and enhancing spectral efficiency. At a transmit power of 25 dBm, in the case of User 1 the BER is improved from 10^-2 (NOMA) to 10^-3 (SM-NOMA). Similarly, for User 2, there is an improvement in BER from 10^-2 (NOMA) to 10^-4 (SM-NOMA). Also, the outage probability values for both the users improved significantly with User 1 dropping from the previous value of 10^-1 (NOMA) to 10^-5 (SM-NOMA) at 20 dBm transmit power and for User 2 it drops from 10^-4 to 8.39233 × 10^-5. The overall spectral efficiency improved from 2.57 bps/Hz (NOMA) to 2.74 bps/Hz (SM-NOMA). Wireless systems are easily prone to attacks by intruders. To enhance transmission security, the system integrates Quantum-Enhanced Genetic Encryption (QGE²) a hybrid encryption framework combining quantum gate-generated keys and genetic algorithm-based encryption. The sequence generated using quantum gates is used as a seed value for the genetic algorithm. It is used to encrypt a 256 × 256 8-bit grayscale image. The randomness of the proposed quantum key is tested through the NIST test. The proposed encryption scheme passed all 17 NIST randomness evaluations (15 core + 2 subsections) with p-values ≥ 0.01. Additional metrics such as correlation, entropy, and histogram analysis validate the strength of the encryption against both classical and quantum threats. The integration of spatial modulation NOMA with QGE² ensures the safe and secure transfer of information through the Rayleigh faded wireless channel.