I think it's possible but would be a very hard thing to test empirically. It's a causal question about a thing that's difficult to manipulate experimentally, with a messy society-level outcome. Any paper that might give you a confident answer on this would be probably be bullshitting a tad!

Yikes. Even if they do realise, and describe it as such.... how many editors saying "wtf no" would it take before the data source named in the manuscript switches from AI to undergrad students or Prolific workers?

Congratulations John!! This is awesome news. Very well deserved!!

The motivation for calculations like these isn't to get p values for their own sake. It's to identify p values in journal articles that are inconsistent with other info provided in the articles. That in turn can signal mistakes in the analysis or writing, or fraud.

Did you guess "that paper does not actually exist"?

Did you also guess that NOT A SINGLE PAPER IN THEIR REFERENCES APPEARS TO EXIST? (Though I confess I only thoroughly checked for half of them, including databases for the journals.)

I haven't so far, but it's a good idea. I'll look into it!

100%. As an editor I've seen some eye-watering examples lately. And if you ask for open data before considering the paper further, the authors pull the 'Homer disappearing into the hedge' trick...

7. the kinda appealing, but substantively indefensible, idea that somehow AI is different to other technology, like calculators, in a pedagogical context — but we totally ban a great deal of technology in the classroom.

(Section 3.7 here doi.org/10.5281/zeno...)

9/n

Thank you for your excellent work as editor, David. AMPPS is such a great asset for the field.

JW: We developed a tool, INSPECT SR, which looks at various red flags, such as retractions, inconsistencies between preregistration and publication, image duplications, means vs SDs, outcome data, etc.
We do give training workshops.
bmjopen.bmj.com/content/14/3...
#IRICSydney