Lightnews — Scholar-powered news

Dylan Foster 🐢 @djfoster.bsky.social · 13h

Lots of interesting directions here! We think there is a lot more to do building on the connection to the discrete sampling/TCS literature and algos from this space, as well as moving beyond autoregressive generation.

Dylan Foster 🐢 @djfoster.bsky.social · 13h

Empirically, we have only tried this w/ small-ish scale so far, but find consistently that VGB outperforms textbook algos on either (1) accuracy; or (2) diversity when compute-normalized. Ex: for Dyck language, VGB escapes the accuracy-diversity frontier for baselines algos.

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

Main guarantee:
- As long as you have exact/verifiable outcome rewards, always converges to optimal distribution.
- Runtime depends on process verifier quality, gracefully degrading as quality gets worse.

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

VGB generalizes the Sinclair–Jerrum '89 random walk (people.eecs.berkeley.edu/~sinclair/ap...)
from TCS (used to prove equivalence of apx. counting & sampling for self-reducible problems), linking test-time RL/alignment with discrete sampling theory. We are super excited about this connection.

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

We give a new algo, Value-Guided Backtracking (VGB), where the idea is to view autoregressive generation as a random walk on the tree of partial outputs, and add a *stochastic backtracking* step—occasionally erasing tokens in a principled way—to counter error amplification.

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

Test-time guidance with learned process verifiers has untapped potential to enhance language model reasoning, but one of the issues with getting this to actually work is that small verifier mistakes are amplified by textbook algos (e.g., block-wise BoN), w/ errors compounding as length increases.

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

With amazing team: Dhruv Rohatgi, Abhishek Shetty, Donya Saless, Yuchen Li, Ankur Moitra, and Andrej Risteski (andrejristeski.bsky.social).

Short thread below.

Andrej Risteski (@andrejristeski.bsky.social)

Machine learning researcher. Professor in ML department at CMU.

andrejristeski.bsky.social

1

Dylan Foster 🐢 @djfoster.bsky.social · 13h

Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking.

A totally new framework based on ~backtracking~ for using process verifiers to guide inference, w/ connections to approximate counting/sampling in theoretical CS.

Paper: www.arxiv.org/abs/2510.03149

1 1

Dylan Foster 🐢 @djfoster.bsky.social · 10d

MSR NYC is hiring spring and summer interns in AI/ML/RL!

Apply here: jobs.careers.microsoft.com/global/en/jo...

Microsoft Research Lab - New York City - Microsoft Research

Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.

www.microsoft.com

7 20

Reposted by Dylan Foster 🐢

Miro Dudik @mdudik.bsky.social · 24d

🚨Microsoft Research NYC is hiring🚨

We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis.

Links to apply: aka.ms/msrnyc-jobs

Microsoft Research Lab - New York City - Microsoft Research

Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.

aka.ms

7 16

Dylan Foster 🐢 @djfoster.bsky.social · Sep 12

For details, see the following links:

Empirical ML/AI: jobs.careers.microsoft.com/global/en/jo...

Theoretical ML/AI: jobs.careers.microsoft.com/global/en/jo...

Search Jobs | Microsoft Careers

jobs.careers.microsoft.com

1 1

Dylan Foster 🐢 @djfoster.bsky.social · Sep 12

Microsoft Research New York City (www.microsoft.com/en-us/resear...) is seeking applicants for multiple Postdoctoral Researcher positions in ML/AI!

These are positions for up to 2 years, starting in July 2026.

Application deadline: October 22, 2025

Microsoft Research Lab - New York City - Microsoft Research

Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.

www.microsoft.com

1 1 4

Dylan Foster 🐢 @djfoster.bsky.social · Aug 27

Quick reminder: The deadline for our workshop on Foundations of Reasoning in Language Models (FoRLM) at NeurIPS 2025 is next Wednesday, Sept 3!

Dylan Foster 🐢 @djfoster.bsky.social · Aug 11

Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025!

📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.

📆 Deadline: Sept 3, 2025

Dylan Foster 🐢 @djfoster.bsky.social · Aug 11

Help us understand how reasoning emerges, where it fails, and how it can be systematically improved.

Website (& CFP/instructions): reasoning-workshop.github.io

Submission link: openreview.net/group?id=Neu...

See you in San Diego!

FoRLM @ NeurIPS'25

NeurIPS 2025 Workshop -- San Diego, California, USA

reasoning-workshop.github.io

Dylan Foster 🐢 @djfoster.bsky.social · Aug 11

Led by Audrey Huang (ahahaudrey.bsky.social), with co-organizers Adam Block, Sadhika Malladi (sadhika.bsky.social), Will Merrill (lambdaviking.bsky.social), Pavel Izmailov, Akshay Krishnamurthy (akshaykr.bsky.social), Tatsunori Hashimoto, and myself.

Bluesky

ahaaudrey.bsky.social

1

Dylan Foster 🐢 @djfoster.bsky.social · Aug 11

Featuring amazing speakers Alekh Agarwal, Yejin Choi (yejinchoinka.bsky.social), Michael Hahn (m-hahn.bsky.social), and Nathan Lambert (natolambert.bsky.social).

yejinchoinka.bsky.social

1

Dylan Foster 🐢 @djfoster.bsky.social · Aug 11

Announcing the first workshop on Foundations of Language Model Reasoning (FoRLM) at NeurIPS 2025!

📝Soliciting abstracts that advance foundational understanding of reasoning in language models, from theoretical analyses to rigorous empirical studies.

📆 Deadline: Sept 3, 2025

1 3 11

Dylan Foster 🐢 @djfoster.bsky.social · Jul 15

For those at ICML, Audrey will be presenting this paper at the 4:30pm poster session this afternoon! West Exhibition Hall B2-B3 W-1009

3

Reposted by Dylan Foster 🐢

Gautam Kamath @gautamkamath.com · Jun 30

ICML's election for their board of directors has begun. I've thrown my hat in the ring. Please consider voting for Gautam Kamath.

I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.

5 30

Reposted by Dylan Foster 🐢

Tom Silver @tomssilver.bsky.social · Jun 29

This week's #PaperILike is "The Power of Resets in Online Reinforcement Learning" (Mhammedi et al., 2024).

If you're doing RL in sim, why not use the sim to its full potential? Reset to any state! (gym.Env.reset() is not all we need.)

PDF: arxiv.org/abs/2404.15417

The Power of Resets in Online Reinforcement Learning

Simulators are a pervasive tool in reinforcement learning, but most existing algorithms cannot efficiently exploit simulator access -- particularly in high-dimensional domains that require general fun...

arxiv.org

2 5

Reposted by Dylan Foster 🐢

let-all.com @let-all.com · Jun 24

📣Join us at COLT 2025 in Lyon for a community event!
📅When: Mon, June 30 | 16:00 CET
What: Fireside chat w/ Peter Bartlett & Vitaly Feldman on communicating a research agenda, followed by mentorship roundtable to practice elevator pitches & mingle w/ COLT community!
let-all.com/colt25.html

7 16

Reposted by Dylan Foster 🐢

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Jun 21

Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please!

Full posting to come in a bit.

Robust Autonomy Emerges from Self-Play

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic drivi...

arxiv.org

4 24 60

Reposted by Dylan Foster 🐢

Jason Hartline @jasonhartline.bsky.social · Jun 9

At the IDEAL annual meeting and saw this paper presented. Basically: reducing length of chain of thought LLM computations by deleting intermediate computations, more like classical functional programming where only function call and return values are important.

arxiv.org/abs/2503.14337

PENCIL: Long Thoughts with Short Memory

While recent works (e.g. o1, DeepSeek R1) have demonstrated great promise of using long Chain-of-Thought (CoT) to improve reasoning capabilities of language models, scaling it up during test-time is c...

arxiv.org

1 3

Reposted by Dylan Foster 🐢

Clément Canonne @ccanonne.github.io · Jun 4

RADEMACHER CHAOS 🤘

1 3

Dylan Foster 🐢 @djfoster.bsky.social · May 26

Link: sites.google.com/view/rltheor...

RL theory seminars - Next Seminar

May 27th 2025, 6 pm UTC Speaker: Dhruv Rohatgi (MIT) Title: Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification Pape...

sites.google.com