Great collaboration with Dmitry Nikolaev, @dominsta.bsky.social and @deboranozza.bsky.social ☺️

- Finally, and for me, most interestingly, our analysis suggests that political biases are already encoded during the pre-training stage.

Taken these evidences together, we highlight important implications these results play on data processing in the development of fairer LLMs.

- There's a strong correlation (Pearson r=0.90) between the predominant stances in the training data and the models’ behavior when probed for political bias on eight policy issues (e.g., environmental protection, migration, etc).

- Source domains of pre-training documents differ significantly, with right-leaning content containing twice as many blog posts and left-leaning content 3 times as many news outlets.

- The framing of political topics varies considerably: right-leaning labeled documents prioritize stability, sovereignty, and cautious reform via technology or deregulation, while left-leaning documents emphasize urgent, science-led mobilization for systemic transformation and equity.

- left-leaning documents consistently outnumber right-leaning ones by a factor of 3 to 12 across training datasets.
- pre-training corpora contains about 4 times more politically engaged content than post-training data.

We have the answers of these questions here : arxiv.org/pdf/2509.22367

We analyze the political content of the training data from OLMO2, the largest fully open-source model.
🕵️‍♀️ We run an analysis in all the datasets (2 pre- and 2 post-training) used to train the models. Here are our findings:

📣 New Preprint!
Have you ever wondered what the political content in LLM's training data is? What are the political opinions expressed? What is the proportion of left- vs right-leaning documents in the pre- and post-training data? Do they correlate with the political biases reflected in models?

Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach, Debora Nozza: What Is The Political Content in LLMs' Pre- and Post-Training Data? https://arxiv.org/abs/2509.22367 https://arxiv.org/pdf/2509.22367 https://arxiv.org/html/2509.22367

Thanks SoftwareCampus for supporting Multiview, the organizers of INRA, and Sourabh Dattawad and @agnesedaff.bsky.social for the great collaboration!

Our evaluation with normative metrics shows that this approach does not diversify only frames in user's history, but also sentiment and news categories. These findings demonstrate that framing acts as a control lever for enhancing normative diversity.

In this paper, we propose introduce media frames as a device for diversifying perspectives in news recommenders. Our results show an improvement in exposure to previously unclicked frames up to 50%.

Today Sourabh Dattawad presented our work "Leveraging Media Frames to Improve Normative Diversity in News Recommendations" at INRA (International Workshop on News Recommendation and Analytics) co-located with RecSys 2025 in Prague.
arxiv.org/pdf/2509.02266

🚨 New paper alert 🚨 Using LLMs as data annotators, you can produce any scientific result you want. We call this **LLM Hacking**.

Paper: arxiv.org/pdf/2509.08825

Last week we held our 1st MilaNLP retreat by beautiful Lago Maggiore! ⛰️🌊
We shared research ideas, stories (academic & beyond), and amazing food. It was a great time to connect outside of the usual lab working days, and most importantly, strengthen our bonds as a team. #ResearchLife #NLProc

🔍 Stiamo studiando come l'AI viene usata in Italia e per farlo abbiamo costruito un sondaggio!

👉 bit.ly/sondaggio_ai...

(è anonimo, richiede ~10 minuti, e se partecipi o lo fai girare ci aiuti un sacco🙏)

Ci interessa anche raggiungere persone che non si occupano e non sono esperte di AI!

Reminder for the importance of evaluating political biases robustly. :)

We (w/ @diyiyang.bsky.social, @zhuhao.me, & Bodhisattwa Prasad Majumder) are excited to present our #NAACL25 tutorial on Social Intelligence in the Age of LLMs!
It will highlight long-standing and emerging challenges of AI interacting w humans, society & the world.
⏰ May 3, 2:00pm-5:30pm Room Pecos

Join us in an hour at 17:00 (CEST) for @taniseceron.bsky.social's talk on "Evaluating Political Bias: Insights into Robustness and Multilinguality“. Access to Zoom at join.slack.com/t/tadapolisc... or send me a ✉️

Sure, it's here: github.com/tceron/eval_...
The code mapping is in the readme file. :)

🥁 It's the second half of our 🌱 speaker series (tada.cool) this term, and we couldn't be more excited! Next week (Wednesday, April 30 at 5pm CET), we have the pleasure of welcoming @taniseceron.bsky.social to share insights on "Facilitating Information Access Through Language Models". More details ⬇️

liberal society... For example, there's no clear stance in the issues of migration.
More on: direct.mit.edu/tacl/article...

I would expect Llama3.1 released last year to have similar political views to what we found in Llama-2.

change the political worldviews of models. In our study, we find that the previous version (Llama-2) consistently reflects more left-leaning views. However, it does depend on the policy issue as we found clear stances of the models only towards social state welfare, environment protection and [2/3]

I agree 100% that we need to understand what they're measuring, and specifically, how they're aligning the models to be hold certain types of political worldviews. However, I find your results rather puzzling because Llama3.1 was released much before they started announcing their strategy to [1/3]