A Deep Learning Model Leveraging Time-Series System Call Data to Detect Malware Attacks in Virtual Machines

Abstract A Tenant Virtual Machine (TVM) user in the cloud may misuse its computing power to launch malware attack against other tenant VMs, Host OS, Hypervisor, or any other computing devices/resources inside the cloud environment of a Cloud Service Provider. The security solutions deployed within the TVM may not be reliable, as malware can disable them or remain undetected due to its hidden nature. Therefore, security solutions deployed outside the virtual machine are necessary. This research proposes deploying an Intrusion Detection System (IDS) at the Hypervisor layer, utilizing time series system call data and employing a Convolutional Neural Network (CNN) model to accurately detect the presence of malicious (malware) computer programs within virtual machines. The raw VMM system call traces are transformed into novel Time Series System Call patterns and utilized by a deep learning algorithm for training and building the classifier model. A deep learning model, CNN, is used to build the classifier model for detecting intrusions with high accuracy. It is capable of detecting both known and unknown malware. The CNN model is compared with machine learning algorithms for the results and discussions, and it outperforms ML algorithms in terms of intrusion detection accuracy when utilizing novel time series system call data..