Cross-lingual Representation Learning via Centroid Intervention Fusion

arXiv:2608.26357 · cs.CL · Submitted 2026-08-26 · Read on arXiv

cs.CL

Submitted: 2026-08-26

Updated: 2026-08-26

Comments: EMNLP 2026 (Main)

Code: https://github.com/VRCMF/CIF

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages.

Terminology

Abstract

Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages. Inference-time intervention offers a lightweight way to improve cross-lingual transfer by modifying the hidden states produced by the LLMs during the forward pass, without updating model parameters. However, existing cross-lingual intervention methods typically learn separate projections from source to target languages, which limits scalability and prevents knowledge sharing across languages. We propose Centroid Intervention Fusion (CIF), a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator. Across multilingual commonsense reasoning, natural language inference, factual editing, and machine translation benchmarks, CIF outperforms the strongest prior pairwise intervention baseline by up to +3.378 pp on average across four model backbones, while supporting performance gains for low resource languages. The code is available at https://github.com/VRCMF/CIF.git.

Sources

Related papers