Can BERT predict fillers for construction elements?

Abstract The starting point of this paper is a central problem in constructicography: on the one hand, constructicon projects aim at describing a broad spectrum of constructions based on usage data; on the other hand, the established method for determining salient fillers of construction elements (slots) requires comprehensive annotations of entire corpora. In this paper, we propose a new method to tackle this problem. Using the pre-trained language model BERT and a limited number of example sentences, the method predicts typical fillers of slots, so-called collo-profiles. A critical evaluation of the method reveals not only its potentials and limitations but also differences between such a prediction-based method and classical count-based methods.