ProMALex: Progressive Modular Adapters for Multi-Jurisdictional Legal Language Modeling

This paper addresses the challenge of adapting language models to the jurisdictionspecific nature of legal corpora.Existing approaches-training separate models for each jurisdiction or using a single shared model-either fail to leverage common legal principles beneficial for low-resource settings or risk negative interference from conflicting jurisdictional interpretations.To overcome these limitations, we propose a parameter-efficient framework ProMALex, that first derives hierarchical relationships across jurisdictions and progressively inserts adapter modules across model layers based on jurisdictional similarity.This design allows modules in lower layers to be shared across jurisdictions, capturing common legal principles, while higher layers specialize through jurisdiction-specific adapters.Experimental results on two legal language modeling benchmarks demonstrate that Pro-MALex outperforms both fully shared and jurisdiction-specific models.