Reposting appreciated since I'm new here and building my network!

Big thought 2
Replication opinions are post-publication review, which can be efficiently targeted at important studies. It can improve translation of research into knowledge. And making it common can improve incentives in academia to seek truth.
statmodeling.stat.columbia.edu/2016/12/16/a...

Big thought 1: Data & code sharing did more for quality assurance than peer review did.
6/X

BUT to be fair:
1. AEJ journals are leaders in requiring sharing of data & code. Here, that facilitated convergence to truth.
2. This is example of a larger problem. It’s about the system, not the individuals. 5/

To put it harshly, the journal went 0 for 2 on this one:
1. It peer-reviewed and published an article quickly demonstrated to have quality problems.
2. It published a back-and-forth that obscured the truth, even though it wasn’t actually a hard call to make and explain. 4/

The verdict this time was easy: I agree with the commenter. There is not compelling evidence that immigration judges grant asylum less on warmer days. Original result is best explained by publication bias or the like. 3/

Many times I’ve submitted comments to journals. It seems important to me that there are bugs, methodological problems, or missing data… The usual response: reject. Sometimes I’ve wished I could appeal to a judge to hear the case.

So I decided to “trial” being a judge. 2/

Last October
@novosad.bsky.social tweet-piqued about how econ journals handle replications, i.e., arguments about whether a particular study has problems 1/
x.com/paulnovosad/...