Towards Open-World DoH Tunnel Detection: A Dual-View Contrastive Learning Framework with Adaptive Feature Boundaries

The emergence of DNS-over-HTTPS (DoH) tunnels poses significant challenges to network security, particularly when encountering unknown traffic patterns not seen during training. Existing approaches struggle to effectively identify novel DoH tunnel variants while maintaining accurate classi-fication of known traffic patterns. In this paper, we propose DualConBound, a novel framework that combines dual-view contrastive learning with dynamic feature boundary estimation to address open-set network traffic classification. Our approach leverages complementary traffic representations and adaptive decision boundaries to better distinguish between known and unknown traffic patterns. Through extensive experiments on real-world network traffic datasets, we demonstrate that our framework achieves 91 % accuracy on known traffic patterns and 71 % accuracy on unknown variants, outperforming other methods by 6% in open-set scenarios. Our work provides a robust solution for identifying emerging DoH tunnel threats in real-world network environments and establishes a new paradigm for open-set network traffic classification.