TCN-BiGRU Model for Malware Detection based on API Call Sequences

The Detection of malware based on API call sequences remains an open cybersecurity challenge because of evasion techniques such as obfuscation, polymorphism, and run-time manipulation. RNNs and CNNs are weak in modeling bidirectional contextual relationships and long-range dependencies, which compromises their robustness against advanced attacks. To overcome these weaknesses, we propose TCN-BiGRU, a deep-learning method that integrates Temporal Convolutional Networks (TCNs) and Bidirectional Gated Recurrent Units (BiGRUs). The TCN module hierarchically models long-range temporal dependencies in API call sequences using dilated convolutions, and the BiGRU layer detects bidirectional contextual patterns to detect evasion techniques such as reverse-order payload decryption. Evaluated on MalBehavD-V1, a benchmark dataset of balanced benign and malicious API call sequences, the model achieved state-of-the-art performance with AUC-ROC values of 0.94, 0.95, 0.89, and 0.99 AUC-ROC, outperforming CNN, GRU, and Transformer baselines. TCN-BiGRU also reduced the training time by 70% compared to the transformer models. The results validated the potential of hybrid architectures in achieving computational efficiency, temporal pattern discovery, and evasion robustness, with great potential for real-time endpoint security products.