Beyond VirusTotal: A semantic approach to building reliable and up-to-date android malware datasets from threat reports

The effective development and evaluation of machine learning (ML)-based Android malware detection systems is hindered by their critical dependence on high-quality, up-to-date training datasets. Current dataset construction methods face several limitations: manual collection using threat intelligence reports ensures reliability but lacks scalability and novelty, whereas automated collection via anti-virus scanning offers scale but suffers from labelling inconsistency and unreliability. This presents a significant software engineering challenge for maintaining robust malware detection performance. This study aims to improve the software engineering practice of building security datasets by designing and implementing AMARSTR, an automated system for constructing reliable, large-scale, and continuously updated Android malware datasets. The key objectives include leveraging the reliability of expert-verified threat reports through automation and developing a novel semantic-based algorithm for accurate malware family labelling directly from the report content. We developed AMARSTR, a system that automatically gathers and parses threat intelligence reports from leading security vendors. It applies a novel semantic-based algorithm to extract Indicators of Compromise (IoCs) and reliably determines malware family labels from the report content. Subsequently, the system downloads the corresponding malicious APK samples using the identified IoCs. We deployed the AMARSTR to generate a new dataset. Our deployment yielded the PubAndMal dataset, which contains 2,894 malicious APKs from 132 families identified in 186 distinct threat reports (2013-2024). Comprehensive experiments demonstrated the effectiveness of PubAndMal for training and evaluating diverse ML-based malware detection models, thereby validating the utility of our method. AMARSTR provides an automated and scalable method for generating reliable and up-to-date Android malware datasets from threat intelligence reports. This contributes to a practical solution to the software engineering challenges of dataset construction in malware detection research and development. The PubAndMal dataset and associated codes are publicly available for further research.