eunsol.bsky.social
Eunsol Choi
@eunsol.bsky.social
2.4K followers 46 following 1 posts
assistant professor in computer science / data science at NYU. studying natural language processing and machine learning.
Posts Media Videos Starter Packs
eunsol.bsky.social
Eunsol Choi @eunsol.bsky.social · Mar 6
When using LLM-as-a-judge, practitioners often use greedy decoding to get the most likely judgment. But we found that deriving a score from the judgment distribution (like taking the mean) works better!
❌LLM-as-a-judge with greedy decoding
😎Using the distribution of the judge’s labels
victorwang37.bsky.social
Victor Wang @victorwang37.bsky.social · Mar 6
LLM judges have become ubiquitous, but valuable signal is often ignored at inference.

We analyze design decisions for leveraging judgment distributions from LLM-as-a-judge: 🧵

(w/ Michael J.Q. Zhang, @eunsol.bsky.social)
1 4 28
Reposted by Eunsol Choi
colmweb.org
Conference on Language Modeling @colmweb.org · Dec 17
Announcement #1: our call for papers is up! 🎉
colmweb.org/cfp.html
And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social
24 66
Reposted by Eunsol Choi
thomlake.bsky.social
Thom Lake @thomlake.bsky.social · Dec 11
I'm at #Neurips2024 this week!

My work (arxiv.org/abs/2406.17692) w/ @gregdnlp.bsky.social & @eunsol.bsky.social exploring the connection between LLM alignment and response pluralism will be at pluralistic-alignment.github.io Saturday. Drop by to learn more!
6 28