Encrypted-Traffic Detection in the TLS 1.3 Era: A Comprehensive Review and Future Research Directions

Transport Layer Security (TLS) 1.3 strengthens Internet privacy by encrypting protocol metadata increasingly used by network monitoring and intrusion-detection systems, while Encrypted ClientHello (ECH) further reduces visibility into connection establishment. This paper presents a systematic and technically grounded review of encrypted-traffic detection approaches under TLS 1.3 and ECH, with an emphasis on their observable features, analytical formulations, and practical limitations. Unlike previous reviews that primarily classify detection techniques, this study explicitly examines the methodological assumptions and mathematical foundations underlying representative approaches, including Random Forest aggregation, Kullback–Leibler divergence, Discrete Fourier Transform (DFT), and Shannon entropy. The literature is systematically organised into machine learning, statistical/rule-based, and behavioural/flow-level approaches and assessed against feature dependency, interpretability, reproducibility, scalability, deployment feasibility, and resilience to reduced visibility. The review identifies continued dependence on TLS-specific or handshake-derived features and highlights persistent challenges in dataset representativeness, cross-environment generalisation, explainability, and adversarial robustness. In contrast, residual observables—including packet timing, size distributions, directional asymmetry, flow dynamics, frequency-domain characteristics, and burst behaviour—remain potentially useful without inspecting encrypted payloads or concealed protocol fields. The synthesis identifies behavioural–statistical fusion as a promising research direction; however, its effectiveness remains empirically unvalidated as an integrated framework. Future research should therefore prioritise reproducible datasets, cross-environment and adversarial evaluation, and lightweight, interpretable detection mechanisms capable of operating under progressively restricted network visibility.