Bryan Wilder
@brwilder.bsky.social
Assistant Professor at Carnegie Mellon. Machine Learning and social impact. https://bryanwilder.github.io/
Paper here: arxiv.org/abs/2602.06286. Led by my excellent PhD student Khurram Yamin
Do LLMs Act Like Rational Agents? Measuring Belief Coherence in Probabilistic Decision Making
Large language models (LLMs) are increasingly deployed as agents in high-stakes domains where optimal actions depend on both uncertainty about the world and consideration of utilities of different out...
arxiv.org
February 9, 2026 at 10:10 PM
Paper here: arxiv.org/abs/2602.06286. Led by my excellent PhD student Khurram Yamin
In applications based on medical diagnosis, the answer is...sometimes! In some settings, we can prove that no rational agent could hold beliefs expressed by the model. But in others, particularly for stronger models, outputs are close to consistent with rational belief
February 9, 2026 at 10:10 PM
In applications based on medical diagnosis, the answer is...sometimes! In some settings, we can prove that no rational agent could hold beliefs expressed by the model. But in others, particularly for stronger models, outputs are close to consistent with rational belief
We give a framework to test whether the model's stated belief functions *as if it were* a rational agent's subjective probability by comparing with its decisions. We give empirically checkable conditions that don't require any assumptions about the model's "utility function".
February 9, 2026 at 10:10 PM
We give a framework to test whether the model's stated belief functions *as if it were* a rational agent's subjective probability by comparing with its decisions. We give empirically checkable conditions that don't require any assumptions about the model's "utility function".
You might think that models don't have coherent beliefs at all. Or, you might think that they don't report truthfully in response to any given prompt. How could we possibly tell?
February 9, 2026 at 10:10 PM
You might think that models don't have coherent beliefs at all. Or, you might think that they don't report truthfully in response to any given prompt. How could we possibly tell?
LLMs are increasingly used as agents for decisions under uncertainty, e.g. medical diagnosis. But do they act like rational agents with coherent beliefs and preferences? Much of the difficulty is telling whether a model's response to.a prompt ("What is the probability of X?") is a "real" belief.
February 9, 2026 at 10:10 PM
LLMs are increasingly used as agents for decisions under uncertainty, e.g. medical diagnosis. But do they act like rational agents with coherent beliefs and preferences? Much of the difficulty is telling whether a model's response to.a prompt ("What is the probability of X?") is a "real" belief.
Totally agree! I think the fundamental distinction is more between people using AI in their own work vs AI being in a decision-making role that everyone is subject to
December 5, 2025 at 10:07 PM
Totally agree! I think the fundamental distinction is more between people using AI in their own work vs AI being in a decision-making role that everyone is subject to
Reposted by Bryan Wilder
As UKRI explores using LLMs to review grants, it's a good time to revisit Bryan Wilder's excellent blog post.
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
Should LLMs be used to review papers? AAAI is piloting LLM-generated reviews this year. I wrote a blog post arguing that using LLMs as reviewers can have bad downstream consequences for science by centralizing judgments about what constitutes good research.
bryanwilder.github.io/files/llmrev...
bryanwilder.github.io/files/llmrev...
Equilibrium effects of LLM reviewing
Equilibrium effects of LLM reviewing
bryanwilder.github.io
December 5, 2025 at 3:25 PM
As UKRI explores using LLMs to review grants, it's a good time to revisit Bryan Wilder's excellent blog post.
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
Come talk to me and Angela at NeurIPS on Friday! We argue that "AI for social impact" needs to get more rigorous about evaluating deployments of AI, but also that there are many other forms of impact that get overlooked right now
with @brwilder.bsky.social
Position Paper: Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards
arxiv.org/abs/2510.18238
But we don't know how to do a poster presentation for a position paper 😅
Position Paper: Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards
arxiv.org/abs/2510.18238
But we don't know how to do a poster presentation for a position paper 😅
Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards
There has been increasing research interest in AI/ML for social impact, and correspondingly more publication venues have refined review criteria for practice-driven AI/ML research. However, these…
arxiv.org
December 1, 2025 at 7:41 PM
Come talk to me and Angela at NeurIPS on Friday! We argue that "AI for social impact" needs to get more rigorous about evaluating deployments of AI, but also that there are many other forms of impact that get overlooked right now
Based on joint work led by @yewonbyun.bsky.social, with @donskerclass.bsky.social. See our NeurIPS paper, arxiv.org/abs/2508.06635, for more!
Valid Inference with Imperfect Synthetic Data
Predictions and generations from large language models are increasingly being explored as an aid in limited data regimes, such as in computational social science and human subjects research. While pri...
arxiv.org
November 14, 2025 at 7:02 PM
Based on joint work led by @yewonbyun.bsky.social, with @donskerclass.bsky.social. See our NeurIPS paper, arxiv.org/abs/2508.06635, for more!
I gave talks at MIT and Harvard this week about "Science with synthetic data". How can generative models help us learn about the actual world (e.g., social systems) in a principled way? Lots of interesting conversations -- more convinced than ever that there's nuanced issues to navigate here.
November 14, 2025 at 7:02 PM
I gave talks at MIT and Harvard this week about "Science with synthetic data". How can generative models help us learn about the actual world (e.g., social systems) in a principled way? Lots of interesting conversations -- more convinced than ever that there's nuanced issues to navigate here.
Reposted by Bryan Wilder
I’m recruiting students this upcoming cycle at UIUC! I’m excited about Qs on societal impact of AI, especially human-AI collaboration, multi-agent interactions, incentives in data sharing, and AI policy/regulation (all from both a theoretical and applied lens). Apply through CS & select my name!
November 6, 2025 at 6:52 PM
I’m recruiting students this upcoming cycle at UIUC! I’m excited about Qs on societal impact of AI, especially human-AI collaboration, multi-agent interactions, incentives in data sharing, and AI policy/regulation (all from both a theoretical and applied lens). Apply through CS & select my name!
We're in the process of selecting the location for next year's ACM EAAMO conference! If you're interested in bringing the EAAMO community to your institution, please check out the open call here and get in touch. conference.eaamo.org/call_for_loc...
Call for Proposals: Host the 2026 ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization!
EAAMO is seeking proposals from universities, institutes and other appropriate venues interested in hosting the 2026 ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (AC...
conference.eaamo.org
October 28, 2025 at 2:58 PM
We're in the process of selecting the location for next year's ACM EAAMO conference! If you're interested in bringing the EAAMO community to your institution, please check out the open call here and get in touch. conference.eaamo.org/call_for_loc...
How can synthetic data from LLMs be used, e.g. for social science, in a principled way? Check out Emily's thread on our NeurIPS paper! Generating paired real-synthetic samples and using both in a method-of-moments framework enables valid inference that benefits when synthetic data is informative.
💡Can we trust synthetic data for statistical inference?
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
October 10, 2025 at 4:39 PM
How can synthetic data from LLMs be used, e.g. for social science, in a principled way? Check out Emily's thread on our NeurIPS paper! Generating paired real-synthetic samples and using both in a method-of-moments framework enables valid inference that benefits when synthetic data is informative.
Reposted by Bryan Wilder
Are you a researcher using computational methods to understand cities?
@mfranchi.bsky.social @jennahgosciak.bsky.social and I organize an EAAMO Bridges working group on Urban Data Science and we are looking for new members!
Fill the interest form on our page: urban-data-science-eaamo.github.io
@mfranchi.bsky.social @jennahgosciak.bsky.social and I organize an EAAMO Bridges working group on Urban Data Science and we are looking for new members!
Fill the interest form on our page: urban-data-science-eaamo.github.io
Urban Data Science & Equitable Cities | EAAMO Bridges
EAAMO Bridges Urban Data Science & Equitable Cities working group: biweekly talks, paper studies, and workshops on computational urban data analysis to explore and address inequities.
urban-data-science-eaamo.github.io
September 3, 2025 at 3:05 PM
Are you a researcher using computational methods to understand cities?
@mfranchi.bsky.social @jennahgosciak.bsky.social and I organize an EAAMO Bridges working group on Urban Data Science and we are looking for new members!
Fill the interest form on our page: urban-data-science-eaamo.github.io
@mfranchi.bsky.social @jennahgosciak.bsky.social and I organize an EAAMO Bridges working group on Urban Data Science and we are looking for new members!
Fill the interest form on our page: urban-data-science-eaamo.github.io
Reposted by Bryan Wilder
New piece, out in the Sigecom Exchanges! It's my first solo-author piece, and the closest thing I've written to being my "manifesto." #econsky #ecsky
arxiv.org/abs/2507.03600
arxiv.org/abs/2507.03600
August 11, 2025 at 1:25 PM
New piece, out in the Sigecom Exchanges! It's my first solo-author piece, and the closest thing I've written to being my "manifesto." #econsky #ecsky
arxiv.org/abs/2507.03600
arxiv.org/abs/2507.03600
Submit an abstract to present a poster at EAAMO, deadline July 25! EAAMO is one of my favorite conferences, and a great place for anyone working on ML/algorithms/optimization in social settings. The conference is in Pittsburgh this November.
conference.eaamo.org/cfp/call_for...
conference.eaamo.org/cfp/call_for...
Call for Posters
We seek poster contributions from different fields that offer insights into the intersectional design and impacts of algorithms, optimization, and mechanism design with a grounding in the social scien...
conference.eaamo.org
July 16, 2025 at 2:57 PM
Submit an abstract to present a poster at EAAMO, deadline July 25! EAAMO is one of my favorite conferences, and a great place for anyone working on ML/algorithms/optimization in social settings. The conference is in Pittsburgh this November.
conference.eaamo.org/cfp/call_for...
conference.eaamo.org/cfp/call_for...
Reposted by Bryan Wilder
ACM EAAMO, which is coming to Pitt this Fall, has two events for students: a doctoral consortium and a poster session, both of which are due July 25th
- poster session conference.eaamo.org/cfp/call_for...
- doctoral consortium
conference.eaamo.org/cfp/call_for...
- poster session conference.eaamo.org/cfp/call_for...
- doctoral consortium
conference.eaamo.org/cfp/call_for...
Call for Posters
We seek poster contributions from different fields that offer insights into the intersectional design and impacts of algorithms, optimization, and mechanism design with a grounding in the social scien...
conference.eaamo.org
July 15, 2025 at 2:09 PM
ACM EAAMO, which is coming to Pitt this Fall, has two events for students: a doctoral consortium and a poster session, both of which are due July 25th
- poster session conference.eaamo.org/cfp/call_for...
- doctoral consortium
conference.eaamo.org/cfp/call_for...
- poster session conference.eaamo.org/cfp/call_for...
- doctoral consortium
conference.eaamo.org/cfp/call_for...
My takeaway is that algorithm designers should think more broadly about the goals for algorithms in policy settings. It's tempting to just train ML models to maximize predictive performance, but services might be improved a lot with even modest alterations for other goals.
July 8, 2025 at 2:59 PM
My takeaway is that algorithm designers should think more broadly about the goals for algorithms in policy settings. It's tempting to just train ML models to maximize predictive performance, but services might be improved a lot with even modest alterations for other goals.
Using historical data from human services, we then look at how severe learning-targeting tradeoffs really are. It turns out, not that bad! We get most of the possible targeting performance while giving up only a little bit of learning compared to the ideal RCT.
July 8, 2025 at 2:59 PM
Using historical data from human services, we then look at how severe learning-targeting tradeoffs really are. It turns out, not that bad! We get most of the possible targeting performance while giving up only a little bit of learning compared to the ideal RCT.
We introduce a framework for designing allocation policies that optimally trade off between targeting high-need people and learning a treatment effect as accurately as possible. We give efficient algorithms and finite-sample guarantees using a duality-based characterization of the optimal policy.
July 8, 2025 at 2:59 PM
We introduce a framework for designing allocation policies that optimally trade off between targeting high-need people and learning a treatment effect as accurately as possible. We give efficient algorithms and finite-sample guarantees using a duality-based characterization of the optimal policy.
A big factor is that randomizing conflicts with the targeting goal: running a RCT means that people with high predicted risk won't get prioritized for treatment. We wanted to know how sharp the tradeoff really is: does learning treatment effects require giving up on targeting entirely?
July 8, 2025 at 2:59 PM
A big factor is that randomizing conflicts with the targeting goal: running a RCT means that people with high predicted risk won't get prioritized for treatment. We wanted to know how sharp the tradeoff really is: does learning treatment effects require giving up on targeting entirely?
These days, public services are often targeted with predictive algorithms. Targeting helps prioritize people who might be most in need. But, we don't typically have good causal evidence about whether the program we're targeting actually improves outcomes. Why not run RCTs?
July 8, 2025 at 2:59 PM
These days, public services are often targeted with predictive algorithms. Targeting helps prioritize people who might be most in need. But, we don't typically have good causal evidence about whether the program we're targeting actually improves outcomes. Why not run RCTs?
Excited to share that our paper "Learning treatment effects while treating those in need" received the exemplary paper award for AI at EC 2025! This paper grew out collaborations with Allegheny County's human services department and my co-author Pim Welle (at ACDHS).
arxiv.org/abs/2407.07596
arxiv.org/abs/2407.07596
Learning treatment effects while treating those in need
Many social programs attempt to allocate scarce resources to people with the greatest need. Indeed, public services increasingly use algorithmic risk assessments motivated by this goal. However, targe...
arxiv.org
July 8, 2025 at 2:59 PM
Excited to share that our paper "Learning treatment effects while treating those in need" received the exemplary paper award for AI at EC 2025! This paper grew out collaborations with Allegheny County's human services department and my co-author Pim Welle (at ACDHS).
arxiv.org/abs/2407.07596
arxiv.org/abs/2407.07596
CMU is hosting a workshop on Human-AI Complementarity for Decision Making this September! Abstract submissions due July 15, travel will be covered for accepted presenters.
www.cmu.edu/ai-sdm/resea...
www.cmu.edu/ai-sdm/resea...
Human-AI Complementarity Workshop - NSF AI Institute for Societal Decision Making - Carnegie Mellon University
Landing page that provides details for the annual AI-SDM workshop on Human-AI Complementarity for Decision Making
www.cmu.edu
July 7, 2025 at 3:33 PM
CMU is hosting a workshop on Human-AI Complementarity for Decision Making this September! Abstract submissions due July 15, travel will be covered for accepted presenters.
www.cmu.edu/ai-sdm/resea...
www.cmu.edu/ai-sdm/resea...
Reposted by Bryan Wilder
Excited to have this work out at ICML this year! Do LLMs make correlated errors? Yes, and those by the same company, and also more accurate/later generations are more correlated -- increasing algorithmic monoculture
arxiv.org/abs/2506.07962
arxiv.org/abs/2506.07962
July 3, 2025 at 1:06 PM
Excited to have this work out at ICML this year! Do LLMs make correlated errors? Yes, and those by the same company, and also more accurate/later generations are more correlated -- increasing algorithmic monoculture
arxiv.org/abs/2506.07962
arxiv.org/abs/2506.07962