Grateful for @wired.com 's coverage of our work on Generative Echo Chambers.

📰 Read the full article:
www.wired.it/article/chat...

📚 Explore our research:
🔗 aclanthology.org/2025.naacl-l...
🔗 dl.acm.org/doi/10.1145/...

#AI #LLMs #EchoChambers #ResponsibleAI #AIEthics

#NAACL2025 30 April - 2pm, Hall 3, Special Theme
"Faux Polyglots: A study on Information Disparity in Multilingual Large Language Models".

Come visit and learn about how multilingual RALMs fail to handle multilingual information conflicts.

Teaser: youtu.be/aPS2Ntav1FE

#LLM #AI #NLProc

🚨 You are only evaluating a slice of your test-time scaling model's performance! 🚨

📈 We consider how models’ confidence in their answers changes as test-time compute increases. Reasoning longer helps models answer more confidently!

📝: arxiv.org/abs/2502.13962

This joint work with @kentonmurray.bsky.social
@ziangxiao.bsky.social. We would love to hear your thoughts and feedback on our research. We are open to future collaborations, feel free to reach out to us!

We discuss the long-term consequences of such findings. With the rise in the militarization of information and its capability to shape narratives, these results concern information parity, which can lead to polarization and hinder constructive communication across different communities.

We find LLMs are “Faux Polyglots”. Not only do current LLMs prefer documents that share the language of the query, but when there is no relevant document in the language of the query, the current LLMs prefer documents in high-resource languages, highlighting the effects of linguistic disparity.

We ask if LLM's multilingual capabilities are backfiring by pushing us into linguistic information cocoons, by analyzing: 1) query language, 2) query type, and 3) document language in relation to query language during information retrieval and generation during multilingual knowledge conflict.

Are multilingual LLMs ready for the challenges of real-world information-seeking where diverse perspectives and facts are represented in different languages?

Unfortunately, No.

We find LLMs are faux polyglots.

📢Preprint: tinyurl.com/fdunz3dz

#LLMs #NLProc

Thrilled that our paper Faux Polyglot has been accepted to #NAACL2025 main! 🚀
We show that multilingual RAG creates language-specific information cocoons and amplifies perspectives and facts in the dominant language, especially when handling knowledge conflicts.
📜 arxiv.org/abs/2407.05502

Putting together a JHU Center for Language and Speech Processing starter pack!

Please reply or DM me if you're doing research at CLSP and would like to be added - I'm still trying to find out which of us are on here so far.

go.bsky.app/JtWKca2

What do you think? How should misinformation be defined and addressed?
Would love to hear your thoughts!

Then, there are cases without clear consensus. Take aliens, for example:
- "Aliens exist."
- "Aliens don’t exist."
Which of these, if either, qualifies as misinformation?
It’s easier when scientific consensus clearly opposes a claim, but what about gray areas? Emerging evidence? Speculative ideas?

If misinformation depends on consensus, is it always relative and period-dependent? Claims accepted as truth today might have been condemned as misinformation in the past. Another recent example of this is COVID 19.

Some argue misinformation is anything that goes against societal consensus or norms. But what about paradigm shifts? When Galileo said the Earth revolves around the Sun, that was against the consensus then. Would it have been classified as "misinformation" at the time?

Is misinformation just about what is said, or does how it's said play a role? For example:
- "I've heard X is doing Y and it might cause Z."
- "X is doing Y, and it WILL cause Z. They must be stopped."
Does tone make a difference?

Had a good conversation about "What exactly is misinformation?" with
@williamjurayj.bsky.social

Thread below