gneubig.bsky.social
Graham Neubig
@gneubig.bsky.social
800 followers 30 following 2 posts
Associate professor at CMU, studying natural language processing and machine learning. Co-founder All Hands AI
Posts Media Videos Starter Packs
gneubig.bsky.social
Graham Neubig @gneubig.bsky.social · Jun 9
Where does one language model outperform the other?

We examine this from first principles, performing unsupervised discovery of "abilities" that one model has and the other does not.

Results show interesting differences between model classes, sizes and pre-/post-training.
lindiatjuatja.bsky.social
Lindia Tjuatja @lindiatjuatja.bsky.social · Jun 9
When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs:

🧵1/9
2 5
Reposted by Graham Neubig
ramon-astudillo.bsky.social
Ramon Astudillo @ramon-astudillo.bsky.social · Feb 8
Nice contribution to the understanding of Long CoT induction arxiv.org/abs/2502.03373 by Edward Yeo and colleagues (advised by @gneubig.bsky.social and @xiangyue96.bsky.social ). Its hard not to see this as mostly a negative result on induction on the 8B scale. 👇
Demystifying Long Chain-of-Thought Reasoning in LLMs
Scaling inference compute enhances reasoning in large language models (LLMs), with long chains-of-thought (CoTs) enabling strategies like backtracking and error correction. Reinforcement learning (RL)...
arxiv.org
1 2 7
Reposted by Graham Neubig
nlpxuhui.bsky.social
Xuhui Zhou @nlpxuhui.bsky.social · Feb 19
LLM agents can code—but can they ask clarifying questions? 🤖💬
Tired of coding agents wasting time and API credits, only to output broken code? What if they asked first instead of guessing? 🚀

(New work led by Sanidhya Vijay: www.linkedin.com/in/sanidhya-...)
1 3 7
gneubig.bsky.social
Graham Neubig @gneubig.bsky.social · Nov 27
We are now done with all classes for CMU CS11-711 Advanced NLP!

Slides: phontron.com/class/anlp-f...
Videos: youtube.com/playlist?lis...

Hope this is useful to people 😀
Schedule
The weekly event schedule.
phontron.com
6 51
Reposted by Graham Neubig
akariasai.bsky.social
Akari Asai @akariasai.bsky.social · Nov 19
1/ Introducing ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge 📚
@uwnlp.bsky.social & Ai2
With open models & 45M-paper datastores, it outperforms proprietary systems & match human experts.
Try out our demo!
openscholar.allen.ai
6 39 160
Reposted by Graham Neubig
lindiatjuatja.bsky.social
Lindia Tjuatja @lindiatjuatja.bsky.social · Nov 20
💬 Have you or a loved one compared LM probabilities to human linguistic acceptability judgments? You may be overcompensating for the effect of frequency and length!
🌟 In our new paper, we rethink how we should be controlling for these factors 🧵:
Screenshot of the paper title "What Goes Into a LM Acceptability Judgment? Rethinking the Impact of Frequency and Length"
1 19 84