Abstract
Cross-lingual alignment, the meaningful similarity of representations acrosslanguages in multilingual language models, has been an active field of researchin recent years. We survey the literature of techniques to improvecross-lingual alignment, providing a taxonomy of methods and summarisinginsights from throughout the field. We present different understandings ofcross-lingual alignment and their limitations. We provide a qualitative summaryof results from a large number of surveyed papers. Finally, we discuss howthese insights may be applied not only to encoder models, where this topic hasbeen heavily studied, but also to encoder-decoder or even decoder-only models,and argue that an effective trade-off between language-neutral andlanguage-specific information is key.