Lightnews — Scholar-powered news

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Aug 27

#GDRI_rep Update 8a: Major updates coming up over the next few weeks.

1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.

Additional studies are connected (see diagram)
🧵

1 13 17

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · May 29

GiveWell sought an independent check on the evidence supporting >$1 billion it has granted to its top charities. This fall, Alex Cohen asked whether we could obtain the datasets needed to reproduce the underlying RCTs. We said yes; charging only a modest fee to cover our time.

🧵 + 🐀 + blog post!

3 33 64

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 27

#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.

Short 🧵

1 16 27

Reposted by Essi Kujansuu

Niklas Jakobsson @nikljako.bsky.social · Mar 27

I am very happy that the editors took our report seriously. Hope more retractions will follow.

Institute for Replication @i4replication.bsky.social · Mar 27

#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.

Short 🧵

1 4 14

Essi Kujansuu @essieconomist.bsky.social · Mar 14

You have to respect a response to a re-response to a response to a report.

Institute for Replication @i4replication.bsky.social · Mar 14

Our full report and our codes are here: osf.io/5k4f7/files/.... This OSF page also includes the authors' response, our response to their response and a second authors response to our response to their response.

We thank the authors for taking the time to respond and this back and forth.

2

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 14

Our full report and our codes are here: osf.io/5k4f7/files/.... This OSF page also includes the authors' response, our response to their response and a second authors response to our response to their response.

We thank the authors for taking the time to respond and this back and forth.

1 2 4

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 14

#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.

1 13 19

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 9

#GDRI_rep Update 3: The paper "Improving Hygiene and Sanitation through Parental Skill Training" by Asadul Islam, Umair Khalil and Tabassum Rahman was conditionally accepted at Economic Development and Cultural Change. We requested a replication package. It has now been withdrawn

1 11 32

Reposted by Essi Kujansuu

Emilia Tjernstrom @econemilia.bsky.social · Mar 4

#OrphanPapers is apparently a thing now
#EconSky

Essi Kujansuu @essieconomist.bsky.social · Mar 3

We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.

Institute for Replication @i4replication.bsky.social · Mar 3

#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/

3 3 14

Essi Kujansuu @essieconomist.bsky.social · Mar 3

We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.

Institute for Replication @i4replication.bsky.social · Mar 3

#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/

7 10

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 3

The editor-in-chief at Food Policy (@_Food_Policy) contacted us because an author requested to withdraw their name for: “Determinants and Dynamics of Food Insecurity During COVID-19 in Rural Bangladesh”. We explained the situation and requested a replication package.

1 1 4

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 3

We completed a full report on “Food insecurity and mental health of women during COVID-19: Evidence from a developing country” PLOS One and shared it with the authors and journal. Two out of three authors have already requested to withdraw their names.

1 1 5

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 3

Here is a list of studies, with links to I4R reports, authors’ responses/statements, and emails sent to the editors and authors (with dates): osf.io/23x64/.

This excel document is being updated every day.

1 1 6

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 3

We are now aiming to reproduce about 20 published studies. This number keeps increasing as new GDRI studies are forthcoming or are now conditionally accepted.

1 1 5

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Mar 3

#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵

1 20 27

Reposted by Essi Kujansuu

Colin Camerer @cfcamerer.bsky.social · Feb 24

I4R doing the lord’s work. Hopefully these will be a collective wake up call to be more careful and transparent about complicated studies in applied micro (and elsewhere too)

Institute for Replication @i4replication.bsky.social · Feb 24

Let’s take a moment to thank all our amazing replicators: Jörg Ankel-Peters, Juan Pablo Aparicio, Gunther Bensch, Carl Bonander, Nikolai Cook, Lenka Fiala, Jack Fitzgerald, Olle Hammar, Felix Holzmeister, Niklas Jakobsson, Anders Kjelsrud, Andreas Kotsadam, Essi Kujansuu, ...

1 4 26

Reposted by Essi Kujansuu

Prashant Garg @prashantgarg.bsky.social · Feb 24

While the (quasi-) experimental methods increase credibility of causal results, there are two related concerns that make studies hard to replicate:
1. decline in reporting of null results
2. increase in use of private sector data

a mini thread 🧵based on work with @trfetzer.com

1 11 40

Reposted by Essi Kujansuu

Alexandra de Gendre @adegendre.bsky.social · Feb 24

Massive team work in replicating papers and cross checking samples and treatments across multiple papers — what a massive undertaking!
This is evidence that journals *must* require data and code sharing.
I’m wondering if anything in the peer review process could have prevented this

Institute for Replication @i4replication.bsky.social · Feb 24

After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵

1 5 15

Reposted by Essi Kujansuu

Mujtaba Rahman @mijrahman.bsky.social · Feb 24

If anything is clear from the last few days, it's surely that Ukraine belongs in the EU

8 110 510

Reposted by Essi Kujansuu

Enkelejda Havari @enkehavari.bsky.social · Feb 24

Wow. This is bigger than I thought.
We should be involved in the data collection as much as we are in the data analysis. Well done @i4replication.bsky.social!

Institute for Replication @i4replication.bsky.social · Feb 24

After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵

2 11

Reposted by Essi Kujansuu

Olle Hammar @ollehammar.bsky.social · Feb 24

Disregarding the sadness of this story, it’s been great working with this awesome team! @jrgptrs.bsky.social @fialalenka.bsky.social @jackfitzgerald.bsky.social @flxhlzmstr.bsky.social @nikljako.bsky.social @anderskjelsrud.bsky.social @kotsadam.bsky.social @essieconomist.bsky.social

Institute for Replication @i4replication.bsky.social · Feb 24

HUGE thanks to our 15+ replicators who have been working on this project for 3 months now. They are not paid, and this is all pro bono. Their work should get published as comments! It has been a pleasure working with you all. And somehow this feels like the beginning!

1 5 16

Reposted by Essi Kujansuu

Josh Merfeld @merfeld.bsky.social · Feb 24

The craziest thing about it is that if you want to believe the most “positive” interpretation, you have to agree that they’re bad at RCTs. Messy, don’t follow protocols, etc.

Yet they consistently get the biggest effect sizes!

It’s almost like a proof by contradiction.

2 4

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Feb 24

This effort was only possible because these journals require data and code repositories. This allowed us to compare data across papers and identify undisclosed connections between 15 studies (e.g., overlapping samples, overlapping interventions, reused treatment assignment).

1 6 41

Reposted by Essi Kujansuu

Jack Fitzgerald @jackfitzgerald.bsky.social · Feb 24

As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.

Institute for Replication @i4replication.bsky.social · Feb 24

After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵

6 28

Reposted by Essi Kujansuu

Institute for Replication @i4replication.bsky.social · Feb 24

After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵

3 61 140