✨ Shared task alert! ✨

We are organizing FadeIT, the first shared task on fallacy detection accounting for genuine disagreement. FadeIT is part of EVALITA, whose workshop will be held in beautiful Bari, Italy in February 2026.

Learn more from our website!
🌐 sites.google.com/fbk.eu/fadei...

Stiamo vivendo una rivoluzione tecnologica, ma quanto conosciamo realmente l'uso dell'IA nella nostra quotidianità? Aiuta a scoprirlo compilando questo breve questionario (10 min): bit.ly/sondaggio_ai...

🔍 Stiamo studiando come l'AI viene usata in Italia e per farlo abbiamo costruito un sondaggio!

👉 bit.ly/sondaggio_ai...

(è anonimo, richiede ~10 minuti, e se partecipi o lo fai girare ci aiuti un sacco🙏)

Ci interessa anche raggiungere persone che non si occupano e non sono esperte di AI!

Further information:
🌐 Website: noisy-text.github.io/2025/
📕 Proceedings: aclanthology.org/volumes/2025...

Organizers: JinYeong Bak, Rob van der Goot, Hyeju Jang, Weerayut Buaphet, Alan Ramponi, Wei Xu, Alan Ritter

See you tomorrow!

Keynote talk 2️⃣

🗣️ Su Lin Blodgett (Microsoft Research Montréal)
🕤 May 3rd, 16:00 (UTC-6 time)
✨ What Can We Learn from Perspectives on Noisy
User-Generated Text?

Keynote talk 1️⃣

🗣️ Verena Blaschke (LMU Munich & MCML)
🕤 May 3rd, 09:30 (UTC-6 time)
✨ Beyond “noisy” text: How (and why) to process dialect data

📣 Join us tomorrow May 3rd for the 10th Workshop on Noisy and User-generated Text #W-NUT at #NAACL2025 (📍 Room Navajo/Nambe)!

The workshop features 16 paper presentations and 2 exciting keynote talks by @verenablaschke.bsky.social and Su Lin Blodgett (titles+abstracts below)! #NLProc #NAACL

👇

I'll be presenting our work today 🕓 Apr 30th, 16:15 (UTC-6 time) at #NAACL2025 during the R&E.2 oral session (Ballroom A)! Come say hi 😊

📝 aclanthology.org/2025.naacl-l...

#NAACL #NLProc #NLP

👋 great initiative!

We release data, code, and the full annotation guidelines to encourage extensions to cover new languages, topics, and additional perspectives 🗣️

See you in Albuquerque! 🏜️

A manual analysis of LLMs’ outputs unveils and quantifies different types of issues that call for future research to make generated responses less brittle in complex setups such as ours 🕵️‍♀️

Check the paper for full results, analyses, discussion and insights! 📝

8/8

Our results show that fallacy detection, which involves capturing lexical, semantic, and even pragmatic aspects of communication, is still far from being addressed with LLMs in a zero-shot setup, especially if we aim at embracing human label variation

7/🧵

We design multi-task fallacy detection baselines and assess LLMs in a zero-shot setting in four fallacy detection setups of increasing complexity: at the post- or the span-level, and using either fallacy macro-categories or the full inventory

6/🧵

In the paper, we provide in-depth analyses and insights into the full annotation process 📝

We also conducted experiments by simultaneously accounting for multiple test sets (beyond “single ground truth”), partial span matches, overlaps, and the varying severity of labeling errors

5/🧵

Due to the complexity of the task, we avoided crowdsourcing and instead devised multiple rounds of annotation and discussion among two expert annotators. We minimize annotation errors whilst keeping signals of human label variation on the whole dataset

⚠️ Natural disagreement is not noise!

4/🧵

Faina covers public discourse on 🔄 migration, 🌱 climate change, and 🏥 public health over a ⌛️ 4-year time frame (2019-22). It opens opportunities for modeling multiple ground truths at a the fine-grained level of text segments and benchmarking fallacy detection methods across topics and time

3/🧵

We introduce Faina, the first fallacy detection dataset that embraces multiple plausible answers and natural disagreement. Faina includes >11K human-labeled span annotations with overlaps across 20 fallacy types on social media posts in Italian

*Faina (en: “beech marten”) 🙂

2/🧵

Happy to share that “Fine-grained Fallacy Detection with Human Label Variation” with @agnesedaff.bsky.social and @satonelli.bsky.social was accepted to #NAACL2025 main conference 🎉

📝 arxiv.org/abs/2502.13853

#NLProc #NLP #NAACL

1/🧵