Advancing Image Cipher: Quadratic Residue on Elliptic Curve With <scp>DNA</scp> Encryption

ABSTRACT Recent research on image encryption has predominantly focused on pixel‐swapping techniques. However, such approaches often lack sufficient complexity and remain susceptible to known‐plaintext and chosen‐plaintext attacks. Another critical challenge lies in securely sharing the hash values of plain images in asymmetric cryptographic settings. Addressing these gaps, this study proposes a novel and efficient image encryption scheme integrating elliptic curve cryptography (ECC) with DNA computing. The proposed method operates in three phases. In the first phase, encryption keys and the hash of the plain image are generated and securely shared with the authorized party. The second phase introduces a pixel‐switching mechanism based on quadratic residues (QR) and quadratic non‐residues (QNR) derived from the ‐coordinates of elliptic curve points, ensuring high structural complexity. In the final phase, DNA operations are applied for the diffusion process, where randomly chosen rows of the image and a mask image are processed using dynamically selected DNA rules. Row and rule selection are governed by the intertwining logistic map, a chaotic map known for its extreme sensitivity to initial conditions, ensuring unpredictable and non‐repetitive encryption behavior. This unpredictability significantly complicates any attempt to reverse‐engineer the encryption process. By incorporating the Diffie–Hellman (DH) key exchange into both the confusion and dynamic diffusion stages, the scheme achieves robust security against various cryptanalytic attacks. Experimental results demonstrate that the proposed approach effectively addresses the challenges of secure hash sharing, confusion, and diffusion while offering high efficiency, strong resistance to cryptanalysis, and practical feasibility for real‐world image encryption applications.