Comparing Generative Artificial Intelligence (GenAI) Models’ Capabilities in Solving Cryptographic Problems and Algorithms

This study aims to evaluate the capabilities of four large language models (LLMs) that include ChatGPT 4o, Claude 3.5 Sonnet, Copilot, and Gemini 1.5 Flash in different cryptographic tasks and algorithms, mainly solving foundation mathematical problems and classical algorithms in cryptography, while also generating code and performing cryptoanalysis. Using a structured methodology, this study assesses the performance of these models in such tasks. The results of this research vary between models, as some are excelling in simpler tasks (e.g Caesar Cipher), but struggle when it comes to more complex algorithms (e.g AES, DES) and large number factorizations and computations.