A DistilBERT-based hierarchical text classification for traffic analysis

Hierarchical multilabel text classification (HMTC) is constrained by the relationships between labels, often represented by nontrivial data structures such as directed acyclic graphs (DAGs). HMTC is widely utilized in real-world applications such as e-commerce and traffic condition classification for the intelligent transportation systems (ITS). However, the scarcity of training datasets and the strict constraints between labels make the HMTC one of the most challenging text classification problems. Recent advances in language modeling and transfer learning have enabled breakthroughs such as the pretraining of large-scale transformer models, vastly improving inference performance while lowering training requirements for task-specific models. This paper proposes a novel model, namely the DistilBERT branching hierarchical classification network (DB-BHCN), to maximize the utilization of advanced language comprehension capabilities in the DistilBERT. We also propose a combination with the adjacency wrapping matrix (AWX) layer, DB-BHCN+AWX, which is capable of full hierarchical compliance. We perform a comprehensive experimental analysis and find that both the proposed methods outperform the state-of-the-art Distil-BERT-based models. These results reveal the effectiveness and efficiency of the proposed mechanisms, showing their possibility to be applied in real-world applications, specifically in the ITS. • DistilBERT Branching Hierarchical Classification Network (DB-BHCN) is proposed. • DB-BHCN significantly improves inference performance compared to the existing models. • The Adjacency Wrapping Matrix (DB-BHCN+AWX) achieves an even larger accuracy. • Both DB-BHCN variants achieve about 18.7% accuracy over state-of-the-art classifiers.