TNT: A Large-Scale P2P Botnet Detection Framework via Communication Topology and Network Traffic

With the booming development of embedded systems and mobile networks, the attack surfaces of botnets are broadened and amplified. Especially recent adversaries tend to leverage peer-to-peer (P2P) manner propagation to construct large-scale botnets because P2P-based schemes eliminate single points of failure. Over the past few decades, the research and industry communities have proposed a variety of solutions to detect botnets, which mainly involve communication topology identification and network traffic analysis. Yet, coping with the large-scale P2P botnets, the former suffer topology indistinguishability, and the latter struggles under massive background traffic. In this paper, we present <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {TNT}$</tex-math> </inline-formula>, a large-scale P2P botnet detection framework via communication topology and network traffic. As its core, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {TNT}$</tex-math> </inline-formula> is powered by three tightly-coupled components: <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {(i)}$</tex-math> </inline-formula> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {tScouter}$</tex-math> </inline-formula> is responsible for profiling the communication topology; <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {(ii)}$</tex-math> </inline-formula> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {tCommander}$</tex-math> </inline-formula> plans the strategy for node inspection; and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {(iii)}$</tex-math> </inline-formula> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {tPatroller}$</tex-math> </inline-formula> investigates the traffic of the corresponding node. Taken together, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {TNT}$</tex-math> </inline-formula> advances the trade-off between detection accuracy (enhance topology-based results via traffic analysis) and overhead (only check part of node traffic according to the planning). Based on 42 groups of combinations involving 6 types of botnets and 7 legitimate P2P traffic, we perform extensive evaluation and demonstrate that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textsf {TNT}$</tex-math> </inline-formula> realizes outstanding detection performance, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.,</i> after checking ~20K nodes, achieve ~99.9% accuracy for a communication graph (including >140K nodes). In addition, we develop the expansion experiments in terms of heterogeneous nodes and accuracy loss, as well as provide deep insights into interpretability from the aspect of the attribution matrix.