Lightnews — Scholar-powered news

Reposted by Desmond Elliott

Philipp Mondorf @pmondorf.bsky.social · Jul 18

📄 [ACL 2025 main] LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks (doi.org/10.48550/arX...)

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

There is an increasing trend towards evaluating NLP models with LLMs instead of human judgments, raising questions about the validity of these evaluations, as well as their reproducibility in the case...

doi.org

1 4 9

Reposted by Desmond Elliott

Tokenization Workshop (TokShop) @ICML2025 @tokshop.bsky.social · Jul 16

Three invited speakers will share their insights at TokShop! Hear from Yuval Pinter @uvp.bsky.social, Desmond Elliott @delliott.bsky.social, and Adrian Łańcuck on cutting-edge tokenization research. Don't miss these keynote presentations! #ICML2025 tokenization-workshop.github.io/speakers

2 9

Reposted by Desmond Elliott

Iyad Rahwan | إياد رهوان @iyadrahwan.bsky.social · Jul 7

It is with great pleasure that I share MAXMINDS 2.0, a new Max Planck program to support scholars in danger of displacement by war or natural disasters, and who have limited access to resources and institutional support.

If you know affected scholars, please share.

www.maxminds.mpg.de

MAXMINDS 2.0 Homepage

MAXMINDS 2.0

www.maxminds.mpg.de

22 24

Desmond Elliott @delliott.bsky.social · Jul 7

📢I am hiring a Postdoc to work on post-training methods for low-resource languages. Apply by August 15 employment.ku.dk/faculty/?sho....
Let's talk at #ACL2025NLP in Vienna if you want to know more about the position and life in Denmark.

Postdoc in Natural Language Processing

employment.ku.dk

12 23

Reposted by Desmond Elliott

Valentina Pyatkin @valentinapy.bsky.social · Jul 3

💡Beyond math/code, instruction following with verifiable constraints is suitable to be learned with RLVR.
But the set of constraints and verifier functions is limited and most models overfit on IFEval.
We introduce IFBench to measure model generalization to unseen constraints.

1 5 29

Desmond Elliott @delliott.bsky.social · Jun 26

📣 I am happy to support Ph.D applications to the Danish Advanced Research Academy. My main areas of research include multimodal learning and tokenization-free language processing. Feel free to reach out if you have similar interests! Applications due August 29 www.daracademy.dk/fellowship/f...

Dara

www.daracademy.dk

1 4

Reposted by Desmond Elliott

#ICCV2025 @iccv.bsky.social · Jun 25

Following #CVPR2025, #ICCV2025 implemented a new policy targeting accountability and integrity. PCs identified 25 highly irresponsible reviewers, resulting in the desk rejection of 29 associated papers, including 12 submissions that otherwise would have been accepted.

9 21

Desmond Elliott @delliott.bsky.social · Jun 23

The participants brought a lot of energy, enthusiasm, and great posters to highlight their research: @antoniakrm.bsky.social and @saravera.bsky.social pictured.

Finally, I want to think the Danish Data Science Academy, Carlsberg Foundation, and the Villum Foundation for supporting the event!

Sara presenting her poster on reasoning with DeepSeek-R1

Antonia presenting her poster (not visible in the image)

4

Desmond Elliott @delliott.bsky.social · Jun 23

Huge thanks to everyone that attended the Copenhagen NLP Symposium last week. Thanks for our wonderful speakers @kylelo.bsky.social, @najoung.bsky.social, Yohei Oseki, @mziizm.bsky.social, and @loubnabnl.hf.co! @mariaa.bsky.social did a great job of summarizing the talks in these liveposts (quoted).

People finding their seats before the event started

1 8

Desmond Elliott @delliott.bsky.social · Jun 22

Thought-provoking interview with Meg about “AGI”

Margaret Mitchell @mmitchell.bsky.social · Jun 20

🙀Have we reached AGI?? 🙀
💛 Really grateful to @financialtimes.com and the incredible journalist @melissahei.bsky.social, who gave me space to talk about “AGI” (vs AI, vs ML) and where we’re headed.
Link here!!
www.ft.com/content/7089...

Margaret Mitchell: artificial general intelligence is ‘just vibes and snake oil’

One of the pioneers of AI ethics explains why human needs should be the central driver in the development of the technology

www.ft.com

1 2

Desmond Elliott @delliott.bsky.social · Jun 20

No, we didn’t record anything but there was an excellent live-poster!

1 2

Reposted by Desmond Elliott

Stella Frank @scfrank.bsky.social · Jun 13

📯 Best Paper Award at CVPR workshop on Visual concepts for our (@doneata.bsky.social + @delliott.bsky.social) paper on probing vision/lang/ vision+lang models for semantic norms!

TLDR: SSL vision models (swinV2, dinoV2) are surprisingly similar to LLM & VLMs even w/o lang 👀
arxiv.org/abs/2506.03994

1 4 12

Desmond Elliott @delliott.bsky.social · Jun 11

Your workshop is so popular that someone managing the door on a one-in one-out basis.

2

Desmond Elliott @delliott.bsky.social · Jun 11

I am looking forward to meeting people working on multimodality at #CVPR2025. You can find me hopping between the @vlms4all.bsky.social and Visual Concepts Workshops on Thursday. Feel free to reach out if you want to grab a coffee ☕ or a beer 🍻 during the week!

Where and when to find me at #CVPR2025 this week

1

Reposted by Desmond Elliott

Ilker Kesen @ilkerkesen.bsky.social · Jun 4

Announcing our recent work “Multilingual Pretraining for Pixel Language Models”! We introduce PIXEL-M4, a pixel language model pretrained on four visually & linguistically diverse scripts: English, Hindi, Ukrainian & Simplified Chinese. #NLProc

1 1 1

Reposted by Desmond Elliott

Srishti @srishtiy.bsky.social · Jun 2

I am excited to announce our latest work 🎉 "Cultural Evaluations of Vision-Language Models Have a Lot to Learn from Cultural Theory". We review recent works on culture in VLMs and argue for deeper grounding in cultural theory to enable more inclusive evaluations.

Paper 🔗: arxiv.org/pdf/2505.22793

Paper title "Cultural Evaluations of Vision-Language Models
Have a Lot to Learn from Cultural Theory"

3 18 57

Reposted by Desmond Elliott

Anna Rogers @annarogers.bsky.social · May 26

📢 The Copenhagen NLP Symposium on June 20th!

- Invited talks by @loubnabnl.hf.co (HF) @mziizm.bsky.social (Cohere) @najoung.bsky.social (BU) @kylelo.bsky.social (AI2) Yohei Oseki (UTokyo)
- Exciting posters by other participants

Register to attend and/or present your poster at cphnlp.github.io /1

Copenhagen NLP Symposium 2025

symposium website

cphnlp.github.io

1 12 35

Reposted by Desmond Elliott

Miryam de Lhoneux @mdlhx.bsky.social · May 16

Interested in multilingual tokenization in #NLP? Lisa Beinborn and I are hiring!

PhD candidate position in Göttingen, Germany: www.uni-goettingen.de/de/644546.ht...

PostDoc position in Leuven, Belgium:
www.kuleuven.be/personeel/jo...

Deadline 6th of June

Stellen OBP - Georg-August-Universität Göttingen

Webseiten der Georg-August-Universität Göttingen

www.uni-goettingen.de

2 13 25

Reposted by Desmond Elliott

Maria Antoniak @mariaa.bsky.social · May 9

Has anyone written anything about *scraping and text processing* for internet pretraining data? Practical details, which tools are used, which webpage elements are considered, how HTML to text conversion is done?

(I know about work on quality filters, relevant but not quite what I'm looking for)

11 10 40

Desmond Elliott @delliott.bsky.social · May 8

Thanks for sharing! I'm looking forward to reading this because I enjoyed reading your lecture notes on Natural Language Understanding with Distributed Representation back in the day.

2

Reposted by Desmond Elliott

Andrew Lampinen @lampinen.bsky.social · May 1

Had fun talking at the Spurious Correlations & Shortcut Learning at ICLR! One example I brought up, which I think provides an uncommon perspective: a case where spurious shortcuts can improve generalization... even to out-of-distribution sets where the spurious feature doesn't generalize! Thread:

1 2 18

Desmond Elliott @delliott.bsky.social · Apr 14

What would you do if someone has rolled your dataset into their benchmark (cool!) but marked it as being available under a much more permissive license (not so cool)?

4

Desmond Elliott @delliott.bsky.social · Apr 12

I'm recruiting a postdoc on an 18-month contract candidate.hr-manager.net/ApplicationI.... The position is about deploying LLMs in the Danish public sector. This is an interdisciplinary project that touches on technical, ethical, and legal aspects of LLM usage. Apply by 1 May 2025.

Postdoctoral Researcher in Natural Language Processing

Postdoc in Natural Language Processing, Department of Computer Science, Faculty of Science, University of Copenhagen The Natural Language Process

candidate.hr-manager.net

2 5

Reposted by Desmond Elliott

Marzieh Fadaee @mziizm.bsky.social · Apr 10

Very excited to release Kaleidoscope—a multilingual, multimodal evaluation set for VLMs, built as part of our open-science initiative!

🌍 18 languages (high-, mid-, low-)
📚 21k questions (55% require image understanding)
🧪 STEM, social science, reasoning, and practical skills

1 4 10

Reposted by Desmond Elliott

Cohere Labs @cohereforai.bsky.social · Apr 10

🚀 We are excited to introduce Kaleidoscope, the largest culturally-authentic exam benchmark.

📌 Most VLM benchmarks are English-centric or rely on translations—missing linguistic & cultural nuance. Kaleidoscope expands in-language multilingual 🌎 & multimodal 👀 VLMs evaluation

1 7 18