Thanks Dan

I agree with Ben: "fire alarms" better than "police patrols" (i.e. basic principal-agent theory!). So, we need to make fire alarms easier, e.g. via @i4replication.bsky.social et al. This is not about "gotcha" moments, but about credibility, which includes showing replications that confirm findings

Yes, journals signal quality. What I like about I4R is the culture it fosters: civil; engagement between original paper authors and replicators; open to younger scholars; publishing replications that show the original is sound, etc. So great for transparency and accountability.

Yep. And also platforms like Institute4Replication are a good innovation. Much faster and more flexible. I use it for the course I teach on “Replicating Research in Political Science”. i4replication.org

I agree. I’m sceptical that having an in house replication specialist would solve the issues we’re discussing. We know (eg. from principal-agent work) that, as a monitoring mechanism, “fire alarms” can be more effective than “police patrol”. So, in that sense, the system isn’t broken.

Sure. Are you willing to pay for that, e.g. by paying to submit papers? Almost no money from journal publishers to pay for anything like that, so journals rely a lot on temporary and usually voluntary labour.

I understand that. I know this stuff is far from easy. I’m just reporting what Dan and Stuart have been saying. Better to talk directly to them.

From what has been said (and that’s all I can judge), the replication and the response to the replication should ideally have been published online at the same time. Both sets of authors made other points, and they can address those themselves.

Yep. On the same page. We need to collectively work out how to do this stuff better.

Thanks Hugo. This debate is now on the curriculum of the course for next term :-)

Btw, it sounds like @apsrjournal.bsky.social didn’t handle this as well as they might have done. Lessons for the future, perhaps.

Thanks Dan. Didn’t mean to be snarky. Just thought I’d flag that, in case you weren’t aware. Good to know it’s in hand.

Hi Dan. This is fascinating. But (somewhat ironically!) the link to your replication files on Dataverse doesn't seem to work (doi.org/10.7910/DVN/...). Maybe fix that. Ta!