Falska larm, riktiga problem : En utvärdering av statiska kodanalysverktyg

Static analysis tools are widely used to detect vulnerabilities in C/C++ code, yet their effectiveness across realistic and synthetic datasets remains underexplored. This thesis systematically evaluates five prominent static analysis tools: Cppcheck, Flawfinder, Semgrep, Clang-Tidy, and Snyk Code. The evaluation was done using two contrasting datasets: the Juliet Test Suite, offering synthetic examples with clearly labeled vulnerabilities, and PrimeVul, providing real-world vulnerable functions paired with their corresponding patches. An evaluation system was implemented, including automated build support across diverse open-source projects, tool output normalization, and metric-based classification of results. Performance was assessed using many metrics, such as Precision, Recall, F1, pair-wise comparative measures, and the Matthews Correlation Coefficient. The results reveal that all tools perform relatively poorly on the synthetic dataset and really poorly on the realistic dataset. This highlights the need for tool improvement and dataset improvements, as well as the importance of dataset diversity in benchmarking static analysis tools.