Towards discovering linguistic indicators for misalignment in language models

Large language models have been shown to exhibit misalignment: unexpected and unin- tended behaviors that emerge across a wide range of domains despite finetuning on benign datasets. We propose identifying relational lin- guistic structures, such as mechanisms that bind entities and attributes conditionally based on whether the model is misaligned or not. As an example, the binding mechanism binds colors "green" or "red" to an object, like "mushroom," when the model is provided the context that one of them is harmful and the other is not. Using mean ablation interventions, we isolate the binding mechanisms that allow models to produce aligned and misaligned responses. Our comparisons of the two mechanisms show that they are distinct, suggesting that the linguistic structures that govern misalignment are sepa- rate and uniquely identifiable. Our approach serves as a latent indicator for misalignment.