Dirk Hovy
@dirkhovy.bsky.social
Professor @milanlp.bsky.social for #NLProc, compsocsci, #ML
Also at http://dirkhovy.com/
Also at http://dirkhovy.com/
Pinned
GitHub - dirkhovy/MACE: Multi-Annotator Competence Estimation tool
Multi-Annotator Competence Estimation tool. Contribute to dirkhovy/MACE development by creating an account on GitHub.
github.com
🚨(Software) Update:
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
Reposted by Dirk Hovy
What are the main issues discussed in a set of documents?
We’ve just released a step-by-step BERTopic tutorial.
We also launch a new page, gathering various NLP tutorials for social scientists.
👉 www.css.cnrs.fr/tutorials-an...
We’ve just released a step-by-step BERTopic tutorial.
We also launch a new page, gathering various NLP tutorials for social scientists.
👉 www.css.cnrs.fr/tutorials-an...
Tutorials and Resources – CSS @ IP-Paris
Site web de l'axe sciences sociales computationnelles du CREST-CNRS. Cours et tutoriels pour l'analyse des données numériques en sciences sociales.
www.css.cnrs.fr
January 27, 2026 at 3:16 PM
What are the main issues discussed in a set of documents?
We’ve just released a step-by-step BERTopic tutorial.
We also launch a new page, gathering various NLP tutorials for social scientists.
👉 www.css.cnrs.fr/tutorials-an...
We’ve just released a step-by-step BERTopic tutorial.
We also launch a new page, gathering various NLP tutorials for social scientists.
👉 www.css.cnrs.fr/tutorials-an...
Reposted by Dirk Hovy
Citation is the foundation of academic promotion. It’s noisy, sure, but its integrity is worth fighting for. Hallucinated citations should be a desk reject.
NEW: NeurIPS,one of the world’s top academic AI conferences, accepted research papers with 100+ AI-hallucinated citations, new report claims
fortune.com/2026/01/21/n...
fortune.com/2026/01/21/n...
NeurIPS papers contained 100+ AI-hallucinated citations, new report claims | Fortune
An analysis of NeurIPS 2025 papers by startup GPTZero reveals how AI-generated citations are slipping into elite academic research.
fortune.com
January 22, 2026 at 1:16 AM
Citation is the foundation of academic promotion. It’s noisy, sure, but its integrity is worth fighting for. Hallucinated citations should be a desk reject.
Reposted by Dirk Hovy
The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted more—which of these new ideas work, and which work for SLMs? jurgens.people.si.umich.edu/CSE598-004/
CSE 598-004 - Building Small Language Models
jurgens.people.si.umich.edu
January 19, 2026 at 9:29 PM
The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted more—which of these new ideas work, and which work for SLMs? jurgens.people.si.umich.edu/CSE598-004/
Reposted by Dirk Hovy
January 20, 2026 at 11:15 AM
Reposted by Dirk Hovy
⏳ Deadline approaching! We’re hiring 2 fully funded postdocs in #NLP.
Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD)
🔗 Details + how to apply: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD)
🔗 Details + how to apply: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
January 19, 2026 at 5:24 PM
⏳ Deadline approaching! We’re hiring 2 fully funded postdocs in #NLP.
Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD)
🔗 Details + how to apply: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD)
🔗 Details + how to apply: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
🚨(Software) Update:
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
GitHub - dirkhovy/MACE: Multi-Annotator Competence Estimation tool
Multi-Annotator Competence Estimation tool. Contribute to dirkhovy/MACE development by creating an account on GitHub.
github.com
January 20, 2026 at 10:12 AM
🚨(Software) Update:
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easy–but often wrong.
1/N
New year, new job? If that is your current mantra, check the open postdoc positions with Debora Nozza and me at our lab. Deadline is January 31st.
milanlproc.github.io/open_positio...
milanlproc.github.io/open_positio...
Postdoctoral Researcher – NLP (2 positions) | MilaNLP Lab @ Bocconi University
Two Postdoctoral Researcher positions – Deadline January 31st, 2026
milanlproc.github.io
January 19, 2026 at 4:13 PM
New year, new job? If that is your current mantra, check the open postdoc positions with Debora Nozza and me at our lab. Deadline is January 31st.
milanlproc.github.io/open_positio...
milanlproc.github.io/open_positio...
Reposted by Dirk Hovy
🚀 We’re opening 2 fully funded postdoc positions in #NLP!
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
December 18, 2025 at 3:29 PM
🚀 We’re opening 2 fully funded postdoc positions in #NLP!
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Happy to have contributed to this
Thrilled to announce the Handbook of Computational Social Science is officially out! 956 pages, 118 authors, and truly global, interdisciplinary perspectives. Deep thanks to the contributors and anonymous reviewers who shaped this over 4 years. Buy your copy now!
@elgarpublishing.bsky.social
@elgarpublishing.bsky.social
December 23, 2025 at 1:55 PM
Happy to have contributed to this
Reposted by Dirk Hovy
#MemoryModay #NLProc Countering Hateful and Offensive Speech Online - Open Challenges" by Plaza-Del-Arco, @debora_nozza, Guerini, Sorensen, Zampieri, 2024 is a tutorial on the challenges and solutions for detecting and mitigating hate speech.
Countering Hateful and Offensive Speech Online - Open Challenges
Flor Miriam Plaza-del-Arco, Debora Nozza, Marco Guerini, Jeffrey Sorensen, Marcos Zampieri. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts.…
aclanthology.org
December 22, 2025 at 4:03 PM
#MemoryModay #NLProc Countering Hateful and Offensive Speech Online - Open Challenges" by Plaza-Del-Arco, @debora_nozza, Guerini, Sorensen, Zampieri, 2024 is a tutorial on the challenges and solutions for detecting and mitigating hate speech.
Reposted by Dirk Hovy
#MemoryModay #NLProc Uma, A. N. et al. examine AI model training in 'Learning from Disagreement: A Survey'. Disagreement-handling methods' performance is shaped by evaluation methods & dataset traits.
jair.org
December 15, 2025 at 4:02 PM
#MemoryModay #NLProc Uma, A. N. et al. examine AI model training in 'Learning from Disagreement: A Survey'. Disagreement-handling methods' performance is shaped by evaluation methods & dataset traits.
Reposted by Dirk Hovy
#TBT #NLProc #MachineLearning #SafetyFirst 'Safety-Tuned LLaMAs: Improving LLMs Safety' by Bianchi et al. explores training LLMs for safe refusals, warns of over-tuning.
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large...
Training large language models to follow instructions makes them perform better on a wide range of tasks and generally become more helpful. However, a perfectly helpful model will follow even the most...
arxiv.org
December 18, 2025 at 4:02 PM
#TBT #NLProc #MachineLearning #SafetyFirst 'Safety-Tuned LLaMAs: Improving LLMs Safety' by Bianchi et al. explores training LLMs for safe refusals, warns of over-tuning.
Come work with @deboranozza.bsky.social, me, and the lab in Milan!
🚀 We’re opening 2 fully funded postdoc positions in #NLP!
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
Join the MilaNLP team and contribute to our upcoming research projects.
🔗 More details: milanlproc.github.io/open_positio...
⏰ Deadline: Jan 31, 2026
December 19, 2025 at 10:58 AM
Come work with @deboranozza.bsky.social, me, and the lab in Milan!
Reposted by Dirk Hovy
We don't actually trust AI.
We trust the companies behind it.
As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.
We trust the companies behind it.
As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.
December 15, 2025 at 5:04 PM
We don't actually trust AI.
We trust the companies behind it.
As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.
We trust the companies behind it.
As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.
Reposted by Dirk Hovy
We're hiring interns in the Computational Social Science group at Microsoft Research NYC!
If you're interested in designing AI‑based systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026: apply.careers.microsoft.com/careers/job/...
If you're interested in designing AI‑based systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026: apply.careers.microsoft.com/careers/job/...
Research Intern - Computational Social Science | Microsoft Careers
Research Interns put inquiry and theory into practice. Alongside fellow doctoral candidates and some of the world's best researchers, Research Interns learn, collaborate, and network for life. Researc...
apply.careers.microsoft.com
December 15, 2025 at 4:33 PM
We're hiring interns in the Computational Social Science group at Microsoft Research NYC!
If you're interested in designing AI‑based systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026: apply.careers.microsoft.com/careers/job/...
If you're interested in designing AI‑based systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026: apply.careers.microsoft.com/careers/job/...
After I shared “How to professor” last year, some people asked for a similar post on writing. Now I finally got around to typing up our lab's writing workshop slides.
It covers basic advice for research papers and grant applications.
Curious? Read it here: dirkhovy.com/post/2025_11...
It covers basic advice for research papers and grant applications.
Curious? Read it here: dirkhovy.com/post/2025_11...
How to Write Gooder | Dirk Hovy
After publishing “ How to professor”, several people said they found it helpful, and asked whether I had a similar post on writing. Luckily, we have held an annual writing workshop in the lab for the last few years, so there already was a presentation.
dirkhovy.com
December 12, 2025 at 11:49 AM
After I shared “How to professor” last year, some people asked for a similar post on writing. Now I finally got around to typing up our lab's writing workshop slides.
It covers basic advice for research papers and grant applications.
Curious? Read it here: dirkhovy.com/post/2025_11...
It covers basic advice for research papers and grant applications.
Curious? Read it here: dirkhovy.com/post/2025_11...
Reposted by Dirk Hovy
#TBT #NLProc 'Respectful or Toxic?' by Plaza-del-Arco, @debora & @dirkhovy.bsky.social (2023) explores zero-shot learning for multilingual hate speech detection. Highlights prompt & model choice for accuracy. #AI #LanguageModels #HateSpeechDetection
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Flor Miriam Plaza-del-arco, Debora Nozza, Dirk Hovy. The 7th Workshop on Online Abuse and Harms (WOAH). 2023.
aclanthology.org
December 11, 2025 at 4:03 PM
#TBT #NLProc 'Respectful or Toxic?' by Plaza-del-Arco, @debora & @dirkhovy.bsky.social (2023) explores zero-shot learning for multilingual hate speech detection. Highlights prompt & model choice for accuracy. #AI #LanguageModels #HateSpeechDetection
Reposted by Dirk Hovy
#MemoryModay #NLProc 'Leveraging Social Interactions to Detect Misinformation on Social Media' by Fornaciari et al. (2023) uses combined text and network analysis to spot unreliable threads.
arxiv.org
December 8, 2025 at 4:03 PM
#MemoryModay #NLProc 'Leveraging Social Interactions to Detect Misinformation on Social Media' by Fornaciari et al. (2023) uses combined text and network analysis to spot unreliable threads.
Reposted by Dirk Hovy
The Center for Information Technology Policy at Princeton invites applications for a Postdoctoral Fellow to work with Andy Guess (Politics/SPIA), Brandon Stewart (Sociology), and me (CS).
puwebp.princeton.edu/AcadHire/app...
Please apply before Sunday, the 13th of December!
puwebp.princeton.edu/AcadHire/app...
Please apply before Sunday, the 13th of December!
December 9, 2025 at 8:51 PM
The Center for Information Technology Policy at Princeton invites applications for a Postdoctoral Fellow to work with Andy Guess (Politics/SPIA), Brandon Stewart (Sociology), and me (CS).
puwebp.princeton.edu/AcadHire/app...
Please apply before Sunday, the 13th of December!
puwebp.princeton.edu/AcadHire/app...
Please apply before Sunday, the 13th of December!
Reposted by Dirk Hovy
#MemoryModay #NLProc 'Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models' by @paul-rottger.bsky.social et al. (2022). A suite of tests for 10 languages.
Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
Paul Röttger, Haitham Seelawi, Debora Nozza, Zeerak Talat, Bertie Vidgen. Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH). 2022.
aclanthology.org
December 1, 2025 at 4:03 PM
#MemoryModay #NLProc 'Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models' by @paul-rottger.bsky.social et al. (2022). A suite of tests for 10 languages.
Reposted by Dirk Hovy
#TBT #NLProc 'Compromesso! Italian Many-Shot Jailbreaks Undermine LLM Safety' by Pernisi, @dirkhovy.bsky.social, @paul-rottger.bsky.social (2024). Paper highlights LLM vulnerability through Italian demos, more demos = more attack chances.
Compromesso! Italian Many-Shot Jailbreaks undermine the safety of Large Language Models
Fabio Pernisi, Dirk Hovy, Paul Röttger. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop). 2024.
aclanthology.org
December 4, 2025 at 4:02 PM
#TBT #NLProc 'Compromesso! Italian Many-Shot Jailbreaks Undermine LLM Safety' by Pernisi, @dirkhovy.bsky.social, @paul-rottger.bsky.social (2024). Paper highlights LLM vulnerability through Italian demos, more demos = more attack chances.
Reposted by Dirk Hovy
📢 Postdoc position 📢
I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1.
(Different from NYU Faculty Fellows, which are also great but less connected to my lab.)
Link in 🧵
I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1.
(Different from NYU Faculty Fellows, which are also great but less connected to my lab.)
Link in 🧵
December 2, 2025 at 4:04 PM
📢 Postdoc position 📢
I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1.
(Different from NYU Faculty Fellows, which are also great but less connected to my lab.)
Link in 🧵
I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1.
(Different from NYU Faculty Fellows, which are also great but less connected to my lab.)
Link in 🧵
Reposted by Dirk Hovy
I will be @euripsconf.bsky.social this week to present our paper as non-archival at the PAIG workshop (Beyong Regulation:
Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions!
If you are at #EurIPS and want to chat about LLM's training data. Reach out!
Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions!
If you are at #EurIPS and want to chat about LLM's training data. Reach out!
📣 New Preprint!
Have you ever wondered what the political content in LLM's training data is? What are the political opinions expressed? What is the proportion of left- vs right-leaning documents in the pre- and post-training data? Do they correlate with the political biases reflected in models?
Have you ever wondered what the political content in LLM's training data is? What are the political opinions expressed? What is the proportion of left- vs right-leaning documents in the pre- and post-training data? Do they correlate with the political biases reflected in models?
December 2, 2025 at 9:47 PM
I will be @euripsconf.bsky.social this week to present our paper as non-archival at the PAIG workshop (Beyong Regulation:
Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions!
If you are at #EurIPS and want to chat about LLM's training data. Reach out!
Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions!
If you are at #EurIPS and want to chat about LLM's training data. Reach out!
Reposted by Dirk Hovy
Another exhausting day in the lab… conducting very rigorous panettone analysis. Pandoro was evaluated too, because we believe in fair experimental design.
November 27, 2025 at 4:06 PM
Another exhausting day in the lab… conducting very rigorous panettone analysis. Pandoro was evaluated too, because we believe in fair experimental design.
Reposted by Dirk Hovy
#TBT #NLProc '@donyarn.bsky.social & @dirkhovy.bsky.social's 2024 paper, 'Conversations as a Source for Teaching Scientific Concepts' turns video dialogues into effective teaching tools.'
arxiv.org
November 27, 2025 at 4:03 PM
#TBT #NLProc '@donyarn.bsky.social & @dirkhovy.bsky.social's 2024 paper, 'Conversations as a Source for Teaching Scientific Concepts' turns video dialogues into effective teaching tools.'