Real-time Network Intrusion Detection via Importance Sampled Decision Transformers

Many real-time cybersecurity problems, like network intrusion detection from packet sequences, can be abstracted as sequence modeling issues. Traditional approaches such as reinforcement learning may not suit these problems due to the non-Markovian nature and often unobservable network states. We propose framing real-time network intrusion detection as causal sequence modeling, leveraging transformer architecture for decision-making. Our framework uses past trajectories (rewards, packets, detection decisions) to generate future detection actions. We enhance this by integrating offline reinforcement learning, using past environment-action records to learn a policy without real-time data. This combination addresses policy bias through Double Policy Estimation (DPE), employing importance sampling for variance reduction in biased datasets, like those for malicious packet detection. Evaluated on public datasets, our solution surpasses baseline algorithms in detection accuracy and timeliness, demonstrating performance improvements and the benefits of double policy estimation in sequence-modeled reinforcement learning.