An Efficient Malware Classification Method Using Novel Deep Learning Model

Malware attacks getting increased due to the increased complexity in their structures have become a key threat to cybersecurity and require better and more efficient means of detection. Signature and heuristic methods of detecting malware do not perform well due to slow developments in this field and thus current detection uses machine learning and deep learning approaches. However, it is seen that high dimensionality and the complexity of malware data are major problems in terms of existing solutions, such as computational burden and overfitting. The presented work in this thesis aims to design a new malware detection framework using ResNet50 deep neural networks fine-tuned with a new wrapper-based feature selection technique operated by the GOA. The supporting framework also takes advantage of the transfer learning feature in ResNet50, a robust convolutional neural network, for feature extraction from malware data. Every slight hint related to malware is learnt by the model through training using ResNet50 on malware datasets. In addition to this, the GOA-based feature selection approach is used to help define the most important features as input to the neural network as well as to relieve the computational load. To assess the effectiveness of the proposed approach, the benchmark datasets of malware were used, and their results were compared to the traditional and recent methods. The findings affirm that the proposed ResNet50-GOA framework for fine-tuning outperforms the competitors by a significant margin in terms of the detection rate and by improved accuracy, precision, recall, area under the precision-recall curve, and F1-score, which illustrates high robustness and fewer false positive cases and complex computation. In addition, the proposed framework is immune to issues like class imbalance and discovers new patterns of emerging malware. This paper fulfills the following gaps in existing literature: It proposes a new approach for detecting malware that is more efficient and scalable than deep learning and metaheuristic optimization algorithms. The results speak to the promise of a combination of techniques in addressing multi-faceted cybersecurity issues, which opens further possibilities for the improvement of automated threat identification systems in the future