Latent Semantic Analysis for Feature Selection: A Proposed Approach for Anomaly Detection in Network Traffic

In recent times, there has been a paradigm shift in technological advancement that has brought about a revolution in every aspect of our lives. Advancements in technologies such as the Internet of Things (IoT), cloud computing and emerging wireless connectivity are now an integral part of our lives and have become an essential national infrastructure component. However, with the advancements in these technologies, cyber attacks have become more sophisticated and stealthier, making it increasingly challenging to detect them using typical layers of defence such as firewalls a nd an tivirus $s o$ ftware, w h ich are predominantly rule-based. To complement these layers of defence, Intrusion Detection Systems (IDSs) have been developed. This work proposes an adaptive feature selection approach as part of IDSs. Its effectiveness in detecting anomalies in network traffic has been demonstrated, and it can complement traditional layers of defence such as firewalls and antivirus software. The prop osed approach involves a two-step process: first, pr eprocessing and feature selection, and second, the training and deployment of a Machine Learning (ML) model for the detection of anomalies in network traffic. The p roposed a pproach u tilises a n unsupervised learning technique called Latent Semantic Analysis (LSA) for feature selection. To evaluate the effectiveness of the proposed approach, a publicly available dataset is used, which yields an accuracy, True Positive Rate (TPR), and F-Score of 99.89%, 100%, and 99.94% respectively. Furthermore, the proposed approach yields a lowest False Positive Rate (FPR) of 0.68%.