8. 🫶 Huge thanks to my co-author @hjhan.bsky.social , my advisor @marinecarpuat.bsky.social, and @navitagoyal.bsky.social for all the insightful discussions!

7. 🚀 While it might be tempting from aggregate correlations to conclude that x-ling alignment affects multilingual performance, we find that by analyzing samples within a language, x-ling alignment with En is not the sole driver of multilingual capability and is task-dependent.

6. 📖✍️ The only exception is Belebele, a reading comprehension benchmark where x-ling alignment is higher in correct samples than incorrect samples. This suggests that LM-s resort to other mechanisms when solving multilingual NLU tasks focused on reasoning.

5. 🤖 Despite high correlations at the language level, we find that x-ling alignment metrics do not differ between samples LLM answers correctly in En and XX (EC-XC) and samples LLM answers correctly in En and wrongly in XX (EC-XW) for 2/3 NLU tasks.

4. 🔍 Asymmetry in translation quality: We believe that the superior English generation capabilities of LLM override the x-ling alignment, as we observe many LR languages with poor x-ling alignment scores but high XX-En COMET scores.

3. 🔍 Alignment at an aggregate level: x-ling alignment (i.e., the percentage of aligned samples) for a given language correlates with accuracy across three NLU tasks. Also, we find x-ling alignment correlates with En-XX translation quality (r=0.87 in FLORES) over XX-En (0.68)

2. We introduce novel measures of x-ling alignment by extracting the embeddings of parallel XX and En text across the layers of the transformer: Discriminative Alignment Index (DALI) and its stronger variant (DALI.S)

1. 🌎 Question: Pretraining data is dominated by English (En), but LLMs exhibit surprising multilingual capabilities. Does this come from LLM's ability to align representations for text in non-English (XX) with its corresponding parallel text in En?
Spoiler: It is complicated!