Predictive AI for Identifying Undiscovered Cyber Threats: A Proactive Security Model for Big Data

As technology advances, cybercriminals adopt more sophisticated strategies to attack weaknesses in individual computers, organisational networks, and nation-states. Organisations systematically gather substantial quantities of security-relevant data, including log events from individuals, networks, and software applications, for further forensic analysis. Conventional security analysis methods are inadequate for handling huge data volumes and may generate excessive false alarms, particularly when organisations transition to cloud architectures and accumulate more data. Furthermore, the identification of current and more complex assaults, such as persistent and advanced threats (APTs), requires ongoing monitoring and analysis of extensive security-related data, with precision and speed. Big Data analytics is actively used in several domains, including financial transactions, healthcare, and industrial applications, among others. It has recently garnered the interest of the information security community because to its purported capability to correlate security-related data and derive insights effectively at an unprecedented scale. In this study, we examine the limitations of conventional technology/systems and SIEM tools in handling massive amounts of data and complex, advanced threats. We further examine the prerequisites for the effective use of Big Data analytics in the domains of cyber threat intelligence and cybersecurity to address extensive data volumes and complex threats. Ultimately, we emphasise the issues arising from this adoption and provide solutions to address these challenges in future study.