New Approach to Shorten Feature Set via TF-IDF for Machine Learning-Based Webshell Detection

The existence of malicious webshells poses a significant threat to the security infrastructure of computer systems, smart devices, and applications. In our work, prevalent forms of malicious webshell scripts, such as PHP, ASP, ASPX, JSP and Powershell have been identified. Machine learning techniques have been proved to be a valuable tool for detecting webshells. Feature reduction has played an important role to overcome excessive features of the dataset in the feature reduction phase, which helps reducing computational costs while still keeping the generalization of machine learning model. This study introduces an innovative approach in feature reduction research by leveraging regular expressions to filter functions or words in webshell files. Subsequently, through the calculation of Term Frequency-Inverse Document Frequency (TF-IDF) values and the establishment of a cut-off point, common and rare features lacking distinguishing value between benign and malicious activities are eliminated. Then, this work extends its scope to perform webshell detection of five types (PHP, ASP, ASPX, JSP, Powershell). Besides, we utilize five distinct machine learning models: Random Forest (RF), Extreme Gradient Boosting (XGB), Multi-layer Perceptron (MLP), Support Vector Machine (SVM), and Convolutional Neural Network (CNN). Computational metrics including Accuracy, F1-score and Training time are examined to comprehensively assess the efficiency of each methodology. Overall results shows that the proposed approach not only accelerates computation time but also enhances the classification accuracy of machine learning models. The outcome of this research underscores the efficiency of the proposed methodology with the highest accuracy of 99.61% when utilizing RF and a cut-off point of 200 (1548 features).