Simplicial Convolutional Networks for Inductive Short Text Classification

Text classification is a fundamental task in Natural Language Processing (NLP). Recently, short texts have gained much attention, given their substantial growth from social media, search snippets, and news feeds. However, short texts with limited contextual information pose challenges to the current models with texts’ sparsity in words and semantics. Graph models have been actively explored, given their capabilities in modelling global structures and their potential to incorporate external auxiliary information. Despite their success, the current graph models are limited in capturing higher-order information, such as group-wise interactions expressed in terms of phrases, and the additional information, with higher computational resources required, might introduce unnecessary noise. In this work, we propose a novel simplicial complex construction based on text data and a higher-order message-passing mechanism on simplexes. We develop a document simplicial complex representation for text sentences based on the directed word co-occurrence. Novel features are proposed for 0-simplex (word), 1-simplex (word-pair), and 2-simplex (three consecutive words) to characterise intrinsic higher-order information among words. The proposed framework, Simplicial Convolutional Networks (SCN), leverages the expressive power of graph neural networks, models higher-order information beyond pair-wise relations and inductively learns their features. Experimental results demonstrate the capability of SCN in short-text classification to outperform benchmark datasets, with generalisability in unseen scenarios.