Lightnews — Scholar-powered news

Quanquan Gu

@quanquangu.bsky.social

1.8K followers 560 following 70 posts

Professor @UCLA, Research Scientist @ByteDance | Recent work: SPIN, SPPO, DPLM 1/2, GPM, MARS | Opinions are my own

Posts Media Videos Starter Packs

Reposted by Quanquan Gu

Nathaniel Blalock @nathanielblalock.bsky.social · Dec 20

Papers #2-3: arxiv.org/abs/2402.10210 and arxiv.org/abs/2405.00675 from the incredible
@quanquangu.bsky.social. I really like how they explore new techniques for RLHF

Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation

Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI), especially when compared with the remarkable progress made in fine-tuning Large Language M...

arxiv.org

1 3 3

Quanquan Gu @quanquangu.bsky.social · Dec 14

Pretraining will only end once we find the optimal scaling law.

1 6

Quanquan Gu @quanquangu.bsky.social · Dec 5

To better interpret the plot, draw a horizontal line representing a specific target validation loss. Find the points where this line intersects the curves for AdamW and MARS, which will allow you to determine how much speedup, in terms of training tokens, MARS achieves compared to AdamW.

Quanquan Gu @quanquangu.bsky.social · Dec 3

Just added you.

Quanquan Gu @quanquangu.bsky.social · Dec 3

With the delivery of MARS complete, the focus now shifts to delivering new architectures.

GitHub - AGI-Arena/MARS: The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models - AGI-Arena/MARS

buff.ly

2 3

Quanquan Gu @quanquangu.bsky.social · Dec 3

Just added you! Welcome!

Quanquan Gu @quanquangu.bsky.social · Dec 2

Just added you.

Quanquan Gu @quanquangu.bsky.social · Dec 1

Just added you.

Quanquan Gu @quanquangu.bsky.social · Nov 30

Just added you!

Quanquan Gu @quanquangu.bsky.social · Nov 29

Just added you!

1 1

Quanquan Gu @quanquangu.bsky.social · Nov 29

Just added you.

1 1

Quanquan Gu @quanquangu.bsky.social · Nov 29

This Thanksgiving, I want to express my heartfelt gratitude to all the students, colleagues, and collaborators who have contributed to the success of SPIN, SPPO, DPLM, GPM, MARS, and many other projects. Your hard work and dedication continue to be truly inspiring.