LightNews
ab-carrell.bsky.social
Annabelle Michael Carrell
@ab-carrell.bsky.social
23 followers 4 following 3 posts
PhD student in ML
Posts Media Videos Starter Packs
ab-carrell.bsky.social
Annabelle Michael Carrell @ab-carrell.bsky.social · Jul 19
So you want to skip our thinning proofs—but you’d still like our out-of-the-box attention speedups? I’ll be presenting the Thinformer at two ICML workshop posters tomorrow!

Catch me at Es-FoMo (1-2:30, East hall A) and at LCFM (10:45-11:30 & 3:30-4:30, West 202-204)
ab-carrell.bsky.social
Annabelle Michael Carrell @ab-carrell.bsky.social · Jul 14
Your data is low-rank, so stop wasting compute! In our new paper on low-rank thinning, we share one weird trick to speed up Transformer inference, SGD training, and hypothesis testing at scale. Come by ICML poster W-1012 Tuesday at 4:30!
lestermackey.bsky.social
Lester Mackey @lestermackey.bsky.social · Feb 18
New guarantees for approximating attention, accelerating SGD, and testing sample quality in near-linear time
4 5
ab-carrell.bsky.social
Annabelle Michael Carrell @ab-carrell.bsky.social · Jul 14
If you’re not at ICML, you can still read our work. Our new theoretically principled algorithms beat recent baselines across multiple tasks—including Transformer approximation!
Low-Rank Thinning
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match the quality of unifor...
arxiv.org
5
ab-carrell.bsky.social
Annabelle Michael Carrell @ab-carrell.bsky.social · Jul 14
Your data is low-rank, so stop wasting compute! In our new paper on low-rank thinning, we share one weird trick to speed up Transformer inference, SGD training, and hypothesis testing at scale. Come by ICML poster W-1012 Tuesday at 4:30!
lestermackey.bsky.social
Lester Mackey @lestermackey.bsky.social · Feb 18
New guarantees for approximating attention, accelerating SGD, and testing sample quality in near-linear time
https://arxiv.org/abs/2502.12063
2 7 25
Reposted by Annabelle Michael Carrell
inference.vc
Ferenc Huszár @inference.vc · Jul 12
Off to ICML next week?

Check out my student Annabelle’s paper in collaboration with @lestermackey.bsky.social and colleagues on low-rank thinning!

New theory, dataset compression, efficient attention and more:

arxiv.org/abs/2502.12063
Low-Rank Thinning
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match the quality of unifor...
arxiv.org
5 11