Multi-HMM Embeddings for Dynamic Malware Classification Using API Call Sequences

Classifying malware into families and categories remains difficult because modern threats are complex and polymorphic. This study couples Hidden Markov Model (HMM)-based representations of Application Programming Interface (API) call sequences with supervised learning. A class-specific HMM is trained for each malware class, and four descriptors (state occupancy, transition, observation compatibility, and likelihood), extracted from the posterior quantities of every HMM, are concatenated into a fixed-length Multi-HMM embedding, classified with Random Forest (RF), Support Vector Machine (SVM), and k-Nearest Neighbors (k-NN). The number of hidden states is fixed to K=4 (selecting training data only does not change the conclusions). On the benchmark dataset of Aggarwal and Di Troia, RF reaches 91.1% (family) and 69.2% (category) accuracy on the main split, and about 90% and 70% averaged over 10 splits. The class-specific construction improves accuracy by about 6 percentage points over a single pooled HMM and by between 11.8 and 17.7 points over a maximum-likelihood HMM classifier. On a second, larger public dataset (APIMDS), without re-tuning, the latter advantage holds, while the gain over a pooled HMM is smaller than on the first dataset. These results indicate that the Multi-HMM representation is effective on both datasets.