Controlling Language and Style of Multi-lingual Generative Language Models with Control Vectors

Control vectors have recently gained popularity as a method for steering transformer-based generative language models. This paper contributes to this path of research by evaluating the robustness of these control vectors in multi- and cross-lingual question-answering settings mimicking the real-world deployment scenario, where models are expected to generate answers to challenging questions. We present a set of experiments to demonstrate that a control vector approach can be used to shift the output of a generative language model from one language to another, and to exercise stylistic control of the output across languages. Overall, we find that the control vector approach offers a relatively lightweight and effective path for developing methods to control the output of multilingual language models with multiple design choices affecting the real-world control performance.