Or, the response should be (using whichever method) let's spend more time learning about why effects look different in the field, so we can better project what we learn from surveys where it "matters"

Also the reply then should not be: let's do observational.

No, the reply then should be: let's do field experiments. ;)

I don't read the criticism as saying "survey experiments are bad", more, "this is an unbalanced diet"

(Okay maybe not _the_ most important, but at least equally important to the ATE)!

I agree but I think we're wrong to see it as an entirely technical question -> the compliance rate is often _the_ most important question in determining whether something "works" but we spend remarkably little time thinking about what determines it

I think this point is nicely stated by this quote in Kinder's critique of the framing literature

onlinelibrary.wiley.com/doi/full/10....

For me the concern that we don't spend enough time thinking carefully about how treatments & effects _would_ translate if applied in the field - people accept that a survey exp lacks ecological validity, but then stop there

I think Carnes and Henderson show an alternative approach quite neatly

Political scientists love their survey experiments.

Or you can also get in touch with some of our current PhD students who work on related questions (and are on BlueSky) @lenmetson.com @isoldehegemann.bsky.social @antonkoenneke.de @benghix.bsky.social

Congratulations 🎉🎊