A novel deep learning-based statistical randomness evaluation test methodology for cryptographic applications

The security of cryptographic systems is directly linked to the statistical randomness properties of the random numbers used. Traditional statistical randomness tests can be limited in evaluating the properties of these numbers and can require long processing times. While widely used test suites such as the National Institute of Standards and Technology (NIST) Special Publication (SP) 800–22 play a crucial role in assessing data randomness, they are slow on large data sets, can only evaluate certain statistical properties, and fail to detect complex data patterns and dependency structures. In this paper, we propose a new deep learning (DL) based method to overcome these limitations. In the study, bit sequences produced by an FPGA-based true random number generator (TRNG) and different pseudo-random number generators (PRNGs) were converted into image format, and classification experiments were carried out on the AlexNet, ResNet50, and EfficientNetB0 architectures. The results showed that AlexNet, with $$87\boldsymbol{\%}$$ accuracy and $$99\boldsymbol{\%}$$ recall, was by far the most successful method compared to the other models. Furthermore, the ablation analysis revealed that the components of data augmentation, early stopping, and cross-validation play a critical role in the model’s stability and generalizability. Providing a reliable randomness evaluation with high accuracy, this new method, as an alternative to traditional statistical test suites, demonstrates that AI-based solutions can enhance the effectiveness and accuracy of randomness assessment in cryptographic applications. The success of DL methods in complex data structures offers significant potential for improving the security of modern cryptographic systems.