A Novel Optimized Approach to Detect Complex and Unknown Malware using Deep Neural Networks

Since new generations of malware become more diverse and complex nowadays the conventional approaches and methods of malware detection and categorization do not always prove to be efficient. In this work, the combined and optimized CNN, and RNN architecture for the Firefly Algorithm is suggested to accurately classify and detect complex and unseen malware. To detect subtle patterns depicting the behavior of the malware, the framework utilizes a comprehensive feature extraction process while reducing the complexity as much as possible. In our future work, we aim to expand the proposed idea to use a deeper model utilizing CNN in combination with GANs, which we would fine-tune with firefly algorithm, on the similar dataset to bolster the detection performance. The optimization algorithm enhances effectiveness and decreases the consumption of resources while retaining high detection capability. Experimental results demonstrate that the model achieves a training and validation accuracy exceeding 99% after a few epochs, showcasing its suitability for real-world, large-scale cybersecurity applications. This novel approach provides a high-accuracy solution for malware detection, incorporating innovative features for both whitelist and blacklist classification.