Cryptoanalysis on the optical image cryptosystem based on interference and unequal modulus decomposition

This paper offers a security analysis of an optical image cryptosystem that integrates interference and unequal modulus decomposition (EMD). In contrast to the conventional interference-based cryptosystem, which utilizes a single phase-only mask (POM) derived from the encryption process as the private key, the enhanced cryptosystem employs two random phase masks (RPMs) and two POMs derived from the phase-reserved (PR) structure as private keys. This modification addresses the silhouette problem inherent in the conventional interference-based cryptosystem by expanding the key space, thereby enhancing security strength. However, it is determined that the RPM <i>R</i>₂(<i>u</i>,<i>v</i>) and the POM <i>P</i>₂(<i>x</i>,<i>y</i>) exhibited low key sensitivity, thereby limiting their impact on the security enhancement. Additionally, due to the independence from the plaintext, the RPMs employed as private keys can be recovered using a designed known-plaintext attack (KPA). Once the RPMs are recovered, the number of unknown keys is reduced to two. This enables the recovery of plaintext information from its ciphertexts via a subsequent ciphertext-only attack (COA) without any prior knowledge of the POMs. In light of these findings, a novel hybrid attack strategy, to our knowledge, is proposed, integrating four cascaded KPAs and a COA, to effectively compromise the advanced interference and EMD-based cryptosystem. To validate the feasibility and effectiveness of the proposed hybrid attack, numerical simulations have been conducted.