Lightnews — Scholar-powered news

Emily Byun @yewonbyun.bsky.social · 1d

14/ This work will be presented as a spotlight talk today at #COLM2025 SocialSim workshop and at NeurIPS 2025.

Paper: arxiv.org/abs/2508.06635
Code: github.com/lasilab/valid-synth-inference

1 2

Emily Byun @yewonbyun.bsky.social · 1d

13/ I really enjoyed working on this project with the brilliant and kindest @shantanug.bsky.social and great mentors @zacharylipton.bsky.social @donskerclass.bsky.social @brwilder.bsky.social

1

Emily Byun @yewonbyun.bsky.social · 1d

12/ This framework provides a foundation for easily extensible estimation methods that can safely incorporate the growing variety and quality of synthetic data sources.

1 1

Emily Byun @yewonbyun.bsky.social · 1d

11/ At a fundamental level, this work takes a step towards understanding how synthetic data from foundation models can be used to support valid inference. As the usage and promise of FMs continue to grow, so too will the complexity of pipelines that incorporate their outputs.

1 2

Emily Byun @yewonbyun.bsky.social · 1d

10/ Empirically, we observe large gains in estimation performance (lower MSE + tighter confidence intervals with valid coverage) across diverse computational social science tasks, with benefits most pronounced in low label regimes.

1 1

Emily Byun @yewonbyun.bsky.social · 1d

9/ In other words, in the worst case where synthetic data is *completely* uninformative (bad quality), including it does not hurt, at least asymptotically.

1 1

Emily Byun @yewonbyun.bsky.social · 1d

8/ When they are independent from each other, the variance reduces to the optimal variance based only on the real data.

1 2

Emily Byun @yewonbyun.bsky.social · 1d

7/ Precisely: The GMM measures the cross-correlations between the synthetic and real data, producing a combination of these moments that reduces the variance of the real data moments if there is information from the synthetic data moments.

1 2

Emily Byun @yewonbyun.bsky.social · 1d

6/ Why and when does synthetic data help? We found that the incorporation of synthetic data leads to more precise estimation and tighter confidence intervals when its moments are predictive of the real data moments

1 2

Emily Byun @yewonbyun.bsky.social · 1d

5/ Prospectively, it was not intuitive whether the incorporation of additional moments based solely on synthetic data (defined in terms of a separate parameter from the target) would yield any benefits (or even affect) the estimation of the target parameter of the real data.

1 1

Emily Byun @yewonbyun.bsky.social · 1d

4/ We propose a solution via a new estimator based on generalized method of moments (GMM) that allows us to incorporate these multiple sources of information by adding moments.

1 2

Emily Byun @yewonbyun.bsky.social · 1d

3/ Problem: Naively aggregating these different sources of information leads to highly biased estimates, due to differences in the underlying distribution

1 1

Emily Byun @yewonbyun.bsky.social · 1d

2/ In limited labeled regimes, LLMs provide practitioners a cheap alternative to attain imperfect labels and even generate entirely new synthetic samples

1 3

Emily Byun @yewonbyun.bsky.social · 1d

💡Can we trust synthetic data for statistical inference?

We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data

2 7 30

Reposted by Emily Byun

Bryan Wilder @brwilder.bsky.social · May 26

Should LLMs be used to review papers? AAAI is piloting LLM-generated reviews this year. I wrote a blog post arguing that using LLMs as reviewers can have bad downstream consequences for science by centralizing judgments about what constitutes good research.

bryanwilder.github.io/files/llmrev...

Equilibrium effects of LLM reviewing

bryanwilder.github.io

11 31 120

Reposted by Emily Byun

Michael Oberst @moberst.bsky.social · Nov 27

I'm recruiting PhD students for Fall 2025! CS PhD Deadline: Dec. 15th.

I work on safe/reliable ML and causal inference, motivated by healthcare applications.

Beyond myself, Johns Hopkins has a rich community of folks doing similar work. Come join us!

1 7 19

Emily Byun @yewonbyun.bsky.social · Nov 19

would love to join!

1 3