Generalized triangle group based S-box construction for secure image encryption

Cyber image security is an important issue in current communication networks, as secret images are often communicated through insecure channels. Most existing image encryption algorithms based on conventional S-box designs and chaotic maps are mostly subject to weak cryptanalysis resistance, poor efficiency, or poor flexibility. In order to overcome such limitations, this paper introduces a new substitution box (S-box) design method based on the parametric action of Generalized Triangle Groups (GTG), represented as [Formula: see text]. The new scheme employs GTG-based substitution and permutation operations for RGB image encryption. The research process entails: (i) rigorous mathematical parameterization of GTG to design nonlinear and bijective S-boxes, (ii) thorough analysis of their cryptographic attributes, such as nonlinearity, differential uniformity, strict avalanche criterion, and linear/differential attack resistance, and (iii) integration of the S-boxes in an RGB image encryption system. Experimental testing proves that the suggested S-box has excellent nonlinearity 112, well-balanced Strict Avalanche Criterion ≈0.50, and minimal BIC values, showing very good resistance towards linear attacks, also entropy levels approaching the ideal 8.0, NPCR of more than 99.8%, and UACI of ≈ 33.3%, better than a number of state-of-the-art approaches. In comparison to state-of-the-art S-box and image encryption algorithms, the new scheme has larger nonlinearity, enhanced diffusion/confusion characteristics, and greater cryptanalysis resistance with comparable computational efficiency.