Enhanced image encryption via a newly modified hyperchaotic lorenz system and extended DNA computing

Abstract The growing threat of cyberattacks targeting digital images calls for robust, efficient encryption schemes to secure sensitive visual data. This paper proposes an enhanced RGB image encryption scheme that integrates a newly modified five-dimensional (5D) hyperchaotic Lorenz system featuring two positive Lyapunov eigenvalues and a Kaplan-Yorke dimension $$D_{KY}=3.809$$ D KY = 3.809 with an extended eight-base DNA computing framework and a hash-based dynamic key generation mechanism. The chaotic sequences drive a three-stage encryption pipeline: DNA encoding and arithmetic operations for pixel-level confusion, adaptive circular permutation for spatial scrambling, and feedback-driven XOR diffusion for greater pixel dependency. Experimental evaluation on standard USC-SIPI benchmark images (512 $$\times $$ × 512) demonstrated high security, achieving an average information entropy of 7.9993, NPCR of 99.62%, and UACI of 33.46 %, all within optimal thresholds. The proposed scheme features a key space exceeding $$2^{465}$$ 2 465 , far exceeding the recommended $$2^{100}$$ 2 100 threshold and successfully passing all 15 NIST 800-22 randomness tests. Encryption and decryption are completed in 1.24 s and 1.28 s, respectively, confirming practical computational effectiveness. Comparative evaluations prove robustness against statistical, differential, occlusion, cropping, known-plaintext, and chosen-plaintext attacks, confirming its suitability for secure image communication.