Packet Field Tree: a Hybrid Approach for Automated Protocol Reverse-Engineering

Information and communications technology (ICT) systems exchange data through agreed network protocols. Hence, to decode the communicated data, these protocols must be thoroughly understood. However, proprietary protocol specifications often exist in either verbose documents, or require manually reverse-engineering its software implementation. Automated protocol reverse-engineering (APRE) attempts to do so without any access to documentation, nor access to the software binary. In this paper, we introduce Packet Field Tree (PFT), the first hybrid approach combining the advantages of heuristic and supervised-learning methods. We apply it to one aspect of the APRE problem, that of discerning the boundaries between packet fields, and it does so with a superior mean perfection, an APRE-specific quasi-accuracy measure, of 0.27 over a variety of 21 protocols. This is an improvement of 0.11 compared to the next best APRE method. PFT also classifies the syntax of every discerned field. To measure this output we propose the Field Tree Score, a starting point for holistic evaluation of APRE-inferred specifications. We make the PFT and our benchmark dataset of 21 protocols available, with both designed to ease integration of community contributions.