To NER or Not to NER? A Case Study of Low-Resource Deontic Modalities in EU Legislation

Deontic modality (obligation, permission, prohibition) in legal documents can convey critical information, and identification of deontic modalities is often performed using Natural Language Processing (NLP) techniques as a 'Deontic Modality Classification' (DMC) text classification task. As deontic modalities in legal text are not mutually exclusive, a key challenge with DMC is that it classifies the provided text into a single modality while in reality it might have multiple deontic modalities. To address this, this study analyzes the feasibility of performing deontic modality identification as a Named Entity Recognition (NER) task over DMC task approaches in a low-resource data setting with EU legislation. Low-resource NLP approaches can offer solutions to tackle the problem of scarce data. In this paper, we use a rule-based approach with modal verbs and a Decision Tree classifier for DMC task. For NER, we utilize Conditional Random Fields (CRFs) in a low-resource setting and report on the reliability and precision for identification of deontic modality. Our experiments reveal that simpler models, like decision trees, out perform larger models in the low-resource setting of DMC obtaining macro-F1 score of 0.83. For the NER task, the CRF models show consistent performance for 'obligation' labels with an F1-score of 0.51 but have wavering results for other classes with a max F1-score of 0.26 for 'permission', and 0.08 for 'prohibition'.