Lightnews — Scholar-powered news

Will Held

@williamheld.com

2.1K followers 450 following 100 posts

Modeling Linguistic Variation to expand ownership of NLP tools Views my own, but affiliations that might influence them: ML PhD Student under Prof. Diyi Yang 2x RS Intern🦙 Pretraining Alum NYU Abu Dhabi Burqueño he/him

Posts Media Videos Starter Packs

Pinned

Will Held @williamheld.com · Jan 22

Balancing data across domains is key to training the best generalist LLMs!

In my summer work on the Meta Llama team, we introduce UtiliMax and MEDU, new methods to estimate data utility and optimize data mixes efficiently.

HF Blog: huggingface.co/blog/WillHel...
ArXiv: arxiv.org/abs/2501.11747

1 5

Reposted by Will Held

Dan Jurafsky @jurafsky.bsky.social · Aug 24

Now that school is starting for lots of folks, it's time for a new release of Speech and Language Processing! Jim and I added all sorts of material for the August 2025 release! With slides to match! Check it out here: web.stanford.edu/~jurafsky/sl...

Speech and Language Processing

web.stanford.edu

2 59 150

Will Held @williamheld.com · Aug 11

"GPT-5 shows scaling laws are coming to an end"

Reposted by Will Held

George Pearkes @peark.es · Aug 5

We’ve discovered a literal miracle with almost unlimited potential and it’s being scrapped for *no reason whatsoever*. This isn’t even nihilism, it’s outright worship of death and human suffering.

Jen Bendery @jbendery.bsky.social · Aug 5

"The U.S. Department of Health and Human Services (HHS) today announced the beginning of a coordinated wind-down of its mRNA vaccine development activities...."

cc: Sen. Bill Cassidy

49 3.3K 10K

Will Held @williamheld.com · Aug 6

Really great pointer from Hao Zhang on the other site in relation to GPT OSS use of attention sinks.

If I were to guess, the attention sink is what allows them to omit QK-Norm which has become otherwise standard.

www.evanmiller.org/attention-is...

Attention Is Off By One

Let’s fix these pesky Transformer outliers using Softmax One and QuietAttention.

www.evanmiller.org

Will Held @williamheld.com · Jul 28

The SALT Lab is at #ACL2025 with our genius leader @diyiyang.bsky.social.

Come see work from
@yanzhe.bsky.social,
@dorazhao.bsky.social @oshaikh.bsky.social,
@michaelryan207.bsky.social, and myself at any of the talks and posters below!

Alt Text:

Conference schedule for July 28th (Monday) and July 29th (Tuesday), listing talk titles, locations, times, and authors:

July 28th, Monday:

1. Attacking Vision-Language Computer Agents via Pop-ups
Location: Hall 4/5, Time: 11:00–12:30
Authors: Yanzhe Zhang, Tao Yu, Diyi Yang

2. SPHERE: An Evaluation Card for Human-AI Systems
Location: Hall 4/5, Time: 18:00–19:30
Authors: Dora Zhao*, Qianou Ma*, Xinran Zhao, Chenglei Si, Chenyang Yang, Ryan Louie, Ehud Reiter, Diyi Yang*, Tongshuang Wu*
(asterisk denotes equal contribution)

July 29th, Tuesday:

1. SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs
Location: Hall 4/5, Time: 10:30–12:00
Authors: Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Barr Held, Diyi Yang

2. Distilling an End-to-End Voice Assistant Without Instruction Training Data
Location: Room 1.61, Time: 14:12 (Second Talk)
Authors: William Barr Held, Yanzhe Zhang, Weiyan Shi, Minzhi Li, Michael J Ryan, Diyi Yang

3. Mind the Gap: Static and Interactive Evaluations of Large Audio Models
Location: Room 1.61 (implied), follows previous talk
Authors: Minzhi Li*, William Barr Held*, Michael J Ryan, Kunat Pipatanakul, Potsawee Manakul, Hao Zhu, Diyi Yang
(asterisk denotes equal contribution)

4. EgoNormia: Benchmarking Physical Social Norm Understanding
Location: Hall 4/5, Time: 16:00–17:30
Authors: MohammadHossein Rezaei*, Yicheng Fu*, Phil Cuvin*, Caleb Ziems, Yanzhe Zhang, Hao Zhu, Diyi Yang
(asterisk denotes equal contribution)

Will Held @williamheld.com · Jul 28

Paper: aclanthology.org/2025.acl-lon...

Will Held @williamheld.com · Jul 28

I'm in Vienna for #ACL2025!

My work is all presented tomorrow, but today you'll find me today at the poster session from 11-12:30 evangelizing
my labmate Yanzhe Zhang's work on his behalf.

If you're interested in the risks traditional pop-up attacks present for AI agents, come chat!

1 4

Will Held @williamheld.com · Jul 10

It seems (at a minimum) like they post-trained on the virulently racist content from this thread. Musk framed this as a request for training data... and the top post is eugenics. Seems unlikely to be coincidence that the post uses the same phrasing as the prompt they later removed...