Design of public key cryptosystem based on the underlying mathematical complexity

This study aims to present a new cryptographic system based on the principles of the Discrete Logarithm Problem (DLP) applied to automorphisms and the non-commutative nature of rectangular matrices. The suggested system undergoes significant adjustments in encryption, decryption, and key generation algorithms to ensure its strength and effectiveness. Empirical results indicate the system’s ability to function within a non-abelian group, thus strengthening it to resist various attacks. Additionally, the integration of a private matrix key adds an extra layer of complexity. Overall, the proposed cryptosystem demonstrates commendable functionality and provides reasonable security.