IMPROVED HYBRID FUZZING TECHNIQUES FOR CONCURRENT SOFTWARE VULNERABILITY DETECTION

This thesis presents an Enhanced Hybrid Fuzzing Framework designed for testing and identifying vulnerabilities in concurrent software systems by integrating fuzzy testing, machine learning, model checking, and concurrency testing techniques. Traditional fuzzing methods often fall short in detecting subtle bugs, particularly those arising in concurrent environments such as race conditions and deadlocks. This hybrid framework addresses these limitations by incorporating a Machine Learning Module that predicts the likelihood of software crashes based on patterns from previous tests, and a Model Checking system that verifies software correctness across different states and multi-threaded executions. The framework’s fuzzing engine generates random or semi-random inputs to test various software behaviors, while the machine learning component prioritizes high-likelihood crash inputs for more focused testing. The Model Checking Module evaluates state transitions and thread interactions, allowing the detection of complex concurrency-related issues. In addition, Error Detection and Reporting mechanisms capture detailed logs of crashes, stack traces, and anomalies, facilitating deeper analysis and efficient debugging. The framework was implemented using Python and C++ programming languages, selected for their flexibility in handling machine learning algorithms, concurrency testing, and low-level memory operations required for fuzzing. Python was employed for the machine learning and data handling components, while C++ was used for the fuzzing engine and model checking due to its performance and system-level capabilities. The results demonstrate the framework's capability to increase the detection of vulnerabilities in complex software systems, reduce false positives, and improve efficiency in concurrent software testing. By leveraging the power of machine learning and model checking, this hybrid approach enhances the software testing process, contributing to more reliable and secure software development. This abstract summarizes the key objectives, techniques, and results of the Enhanced Hybrid Fuzzing Framework, highlighting its implementation in Python and C++ for optimal performance in concurrent software environments.