🧵 [6/n] For more details, see:
Project Website: shuhaibm.github.io/refed/
Paper: arxiv.org/abs/2502.04511

Thanks to my incredible co-authors Xiusi Chen, Heng Ji, @dilekh.bsky.social!

🧵 [5/n] Our experiments speak for themselves:
📊 We demonstrate consistent improvement across both base and instruct variants of different model architectures
📊 Analysis of filtering strategies reveals dataset variants that maintain strong performance while reducing costs

🧵 [4/n] Our experiments speak for themselves:
📊 Llama-3.1-8B-Instruct + REFED achieves SOTA among SFT-based 8B parameter models on AlpacaEval 2.0
📊 Comparisons and ablation studies validate every component of our framework and show advantages over traditional feedback

🧵 [3/n] 📚Our data synthesis framework uses reference-level feedback to guide the synthesis of new instructions as well as improve their corresponding responses. We present REFED, a dataset consisting of 10K samples synthesized using our framework.

🧵 [2/n] Our key insight 🎯 We extract valuable feedback from high-quality reference samples to guide data synthesis. This effectively leverages seed datasets, propagating desirable qualities to newly synthesized data.

💡 Introducing Reference-Level Feedback: A new paradigm for using feedback to improve synthetic data!
🌐 shuhaibm.github.io/refed/
🧵 [1/n]