Anne Scheel
@annemscheel.bsky.social
Assistant prof at Utrecht University, trying to make science as reproducible as non-scientists think it is. Blogs at @the100ci.
Reposted by Anne Scheel
Do you sometimes think "oh boy, I would really like to hear Julia ramble some more about the topics about which she doesn't stop talking to begin with?"
The wait is finally over! @guruspod.bsky.social and had a chat about open science, causal inference, and apparently birth order effects.
The wait is finally over! @guruspod.bsky.social and had a chat about open science, causal inference, and apparently birth order effects.
Open Science, Psychology, and the Art of Not Quite Claiming Causality with Julia Rohrer - Decoding the Gurus
In a rare departure from our usual diet of online weirdos, this episode features an academic who is very much not a guru. We’re joined by Julia Rohrer...
decoding-the-gurus.captivate.fm
January 30, 2026 at 11:06 AM
Do you sometimes think "oh boy, I would really like to hear Julia ramble some more about the topics about which she doesn't stop talking to begin with?"
The wait is finally over! @guruspod.bsky.social and had a chat about open science, causal inference, and apparently birth order effects.
The wait is finally over! @guruspod.bsky.social and had a chat about open science, causal inference, and apparently birth order effects.
Reposted by Anne Scheel
Really cool work!
New paper out:
Some people are systematically better at judging others’ intelligence.
Who are the best judges? People WHO are intelligent themselves, have good emotion-perception ability, and who are high in well-being.
www.sciencedirect.com/science/arti...
Some people are systematically better at judging others’ intelligence.
Who are the best judges? People WHO are intelligent themselves, have good emotion-perception ability, and who are high in well-being.
www.sciencedirect.com/science/arti...
The good judge of intelligence
Accurately judging others' intelligence is important, yet little is known about individual differences in this ability. In this study we investigated …
www.sciencedirect.com
January 31, 2026 at 8:37 AM
Really cool work!
Reposted by Anne Scheel
Great talk from @lhdjung.bsky.social at the SwissRN Research-On-Research day on his, our, and other folks' work to develop trustworthiness assessment tools.
Finally his {scrutiny} R package has a logo!
Finally his {scrutiny} R package has a logo!
January 29, 2026 at 4:09 PM
Great talk from @lhdjung.bsky.social at the SwissRN Research-On-Research day on his, our, and other folks' work to develop trustworthiness assessment tools.
Finally his {scrutiny} R package has a logo!
Finally his {scrutiny} R package has a logo!
Reposted by Anne Scheel
January 29, 2026 at 1:02 PM
Reposted by Anne Scheel
Just learned about this study looking at methodological trends in psych and econ over time: online.ucpress.edu/collabra/art....
Matches my perception well: Nobody in psych bothers to (explicitly) try causal inference unless they conducted an experiment, not a lot of theoretical work either.
Matches my perception well: Nobody in psych bothers to (explicitly) try causal inference unless they conducted an experiment, not a lot of theoretical work either.
January 29, 2026 at 9:41 AM
Just learned about this study looking at methodological trends in psych and econ over time: online.ucpress.edu/collabra/art....
Matches my perception well: Nobody in psych bothers to (explicitly) try causal inference unless they conducted an experiment, not a lot of theoretical work either.
Matches my perception well: Nobody in psych bothers to (explicitly) try causal inference unless they conducted an experiment, not a lot of theoretical work either.
Reposted by Anne Scheel
New blog post introducing Causion - a web app for causal inference teaching and learning: pedermisager.org/blog/causion....
Introducing Causion: A web app for playing with DAGs | Peder M. Isager
Personal website of Dr. Peder M. Isager
pedermisager.org
January 28, 2026 at 9:23 AM
New blog post introducing Causion - a web app for causal inference teaching and learning: pedermisager.org/blog/causion....
Reposted by Anne Scheel
In sum: all global prevalence rates about gaming addiction/disorder seem to be a partial combination of both gaming *and* gambling prevalence. The same applies to other studies, which use gaming measures in relevant languages. Chinese seems to be a major exception.
January 28, 2026 at 9:56 AM
In sum: all global prevalence rates about gaming addiction/disorder seem to be a partial combination of both gaming *and* gambling prevalence. The same applies to other studies, which use gaming measures in relevant languages. Chinese seems to be a major exception.
Reposted by Anne Scheel
behold, we found great variation in how people think! Many activities that we thought would be “gaming” weren't & vice versa, eg half of the participants interpreted ‘gambling’ to be ‘gaming’. Ergo: surveying ‘gaming’ without defining it creates data mess
January 28, 2026 at 9:56 AM
behold, we found great variation in how people think! Many activities that we thought would be “gaming” weren't & vice versa, eg half of the participants interpreted ‘gambling’ to be ‘gaming’. Ergo: surveying ‘gaming’ without defining it creates data mess
Reposted by Anne Scheel
..we then reviewed the next 358 studies to see if we could find more information in papers & public materials. Basically the only case where gambling was excluded was inherent language: ~30% of the studies used Chinese, where confusion shouldn’t exist. Field-wide measurement error is hence ~70%
January 28, 2026 at 9:56 AM
..we then reviewed the next 358 studies to see if we could find more information in papers & public materials. Basically the only case where gambling was excluded was inherent language: ~30% of the studies used Chinese, where confusion shouldn’t exist. Field-wide measurement error is hence ~70%
Reposted by Anne Scheel
📣 Our introduction to structural causal models in science studies is now published:
doi.org/10.1162/QSS....
@tklebel.bsky.social and I tried to make our introduction as accessible as possible. We illustrate the theory by three case studies based on a simulated model of Open Science. 🧵(1/6)
doi.org/10.1162/QSS....
@tklebel.bsky.social and I tried to make our introduction as accessible as possible. We illustrate the theory by three case studies based on a simulated model of Open Science. 🧵(1/6)
January 27, 2026 at 10:24 AM
📣 Our introduction to structural causal models in science studies is now published:
doi.org/10.1162/QSS....
@tklebel.bsky.social and I tried to make our introduction as accessible as possible. We illustrate the theory by three case studies based on a simulated model of Open Science. 🧵(1/6)
doi.org/10.1162/QSS....
@tklebel.bsky.social and I tried to make our introduction as accessible as possible. We illustrate the theory by three case studies based on a simulated model of Open Science. 🧵(1/6)
Reposted by Anne Scheel
We looked at 100 IGT studies and found abysmal reporting standards, and 244 different scoring techniques. Applied to a new dataset, the average correlation between these 244 techniques was 0. 5/15 IGT meta-analyses we looked at did not account for this hetereogenity.
January 26, 2026 at 8:28 AM
We looked at 100 IGT studies and found abysmal reporting standards, and 244 different scoring techniques. Applied to a new dataset, the average correlation between these 244 techniques was 0. 5/15 IGT meta-analyses we looked at did not account for this hetereogenity.
Reposted by Anne Scheel
Foody et al 2013 has over 130 citations (top 1%) as support for an element of Acceptance and Commitment Therapy's model, but it contains no evidence for that claim.
This is a case study in flawed original studies being cited favorably and uncritically until it becomes a common scientific belief.🧵
This is a case study in flawed original studies being cited favorably and uncritically until it becomes a common scientific belief.🧵
No evidence of differences between hierarchical versus distinction relations in self-based ACT exercises: A critical reanalysis of Foody et al. (2013) and Foody et al. (2015): https://osf.io/ab7de
January 26, 2026 at 12:47 PM
Foody et al 2013 has over 130 citations (top 1%) as support for an element of Acceptance and Commitment Therapy's model, but it contains no evidence for that claim.
This is a case study in flawed original studies being cited favorably and uncritically until it becomes a common scientific belief.🧵
This is a case study in flawed original studies being cited favorably and uncritically until it becomes a common scientific belief.🧵
Reposted by Anne Scheel
The raw data is now lost to the sands of time, but a multiverse analysis can still be conducted on the differences between the conditions at post by extracting data from the figures.
Results show no differences in means under any set of analytic choices.
Results show no differences in means under any set of analytic choices.
January 26, 2026 at 12:47 PM
The raw data is now lost to the sands of time, but a multiverse analysis can still be conducted on the differences between the conditions at post by extracting data from the figures.
Results show no differences in means under any set of analytic choices.
Results show no differences in means under any set of analytic choices.
Reposted by Anne Scheel
To understand the impact of this flexibility in scoring, we ran a multiverse analysis on an open IGT dataset using 205 of the identified scores. The correlations ranged from -0.942 to 0.998 with a median r = 0.022.
January 25, 2026 at 11:16 AM
To understand the impact of this flexibility in scoring, we ran a multiverse analysis on an open IGT dataset using 205 of the identified scores. The correlations ranged from -0.942 to 0.998 with a median r = 0.022.
Reposted by Anne Scheel
But the most striking finding was in the scoring: We found 244 distinct scores used to analyze IGT outcomes. 177 of those scores appeared only once in our sample, showing a massive fragmentation in how IGT data is analyzed.
January 25, 2026 at 11:16 AM
But the most striking finding was in the scoring: We found 244 distinct scores used to analyze IGT outcomes. 177 of those scores appeared only once in our sample, showing a massive fragmentation in how IGT data is analyzed.
Reposted by Anne Scheel
FYI to anyone whose "task queued" status never changed - there was a small bug preventing them from being processed, and this is now fixed. Your reports should be ready soon!
January 24, 2026 at 11:16 AM
FYI to anyone whose "task queued" status never changed - there was a small bug preventing them from being processed, and this is now fixed. Your reports should be ready soon!
Reposted by Anne Scheel
8/🧵
Finally, I fulfill something long-promised: a full, open-source release of the entire RegCheck codebase.
This comes with instructions on how to run RegCheck locally, both as a local implementation of the GUI, and as a standalone CLI application. Find the code here:
github.com/JamieCummins...
Finally, I fulfill something long-promised: a full, open-source release of the entire RegCheck codebase.
This comes with instructions on how to run RegCheck locally, both as a local implementation of the GUI, and as a standalone CLI application. Find the code here:
github.com/JamieCummins...
GitHub - JamieCummins/regcheck
Contribute to JamieCummins/regcheck development by creating an account on GitHub.
github.com
January 22, 2026 at 11:05 AM
8/🧵
Finally, I fulfill something long-promised: a full, open-source release of the entire RegCheck codebase.
This comes with instructions on how to run RegCheck locally, both as a local implementation of the GUI, and as a standalone CLI application. Find the code here:
github.com/JamieCummins...
Finally, I fulfill something long-promised: a full, open-source release of the entire RegCheck codebase.
This comes with instructions on how to run RegCheck locally, both as a local implementation of the GUI, and as a standalone CLI application. Find the code here:
github.com/JamieCummins...
Reposted by Anne Scheel
6/🧵
I have also added integration with the ClinicalTrials.gov API.
You can now provide a ClinicalTrials.gov identifier and RegCheck will automatically retrieve the registration.
I have also added integration with the ClinicalTrials.gov API.
You can now provide a ClinicalTrials.gov identifier and RegCheck will automatically retrieve the registration.
January 22, 2026 at 11:05 AM
6/🧵
I have also added integration with the ClinicalTrials.gov API.
You can now provide a ClinicalTrials.gov identifier and RegCheck will automatically retrieve the registration.
I have also added integration with the ClinicalTrials.gov API.
You can now provide a ClinicalTrials.gov identifier and RegCheck will automatically retrieve the registration.
Reposted by Anne Scheel
5/🧵
Crucially: quote extraction is NOT done by a generative LLM.
Instead, it uses embeddings-based comparisons between text chunks and user-defined comparison dimensions.
This means extraction is deterministic: run RegCheck with the same inputs, you'll get the same quotes as outputs.
Crucially: quote extraction is NOT done by a generative LLM.
Instead, it uses embeddings-based comparisons between text chunks and user-defined comparison dimensions.
This means extraction is deterministic: run RegCheck with the same inputs, you'll get the same quotes as outputs.
January 22, 2026 at 11:05 AM
5/🧵
Crucially: quote extraction is NOT done by a generative LLM.
Instead, it uses embeddings-based comparisons between text chunks and user-defined comparison dimensions.
This means extraction is deterministic: run RegCheck with the same inputs, you'll get the same quotes as outputs.
Crucially: quote extraction is NOT done by a generative LLM.
Instead, it uses embeddings-based comparisons between text chunks and user-defined comparison dimensions.
This means extraction is deterministic: run RegCheck with the same inputs, you'll get the same quotes as outputs.
Reposted by Anne Scheel
Comparing registrations to published papers is essential to research integrity - and almost no one does it routinely because it's slow, messy, and time-demanding.
RegCheck was built to help make this process easier.
Today, we launch RegCheck V2.
🧵
regcheck.app
RegCheck was built to help make this process easier.
Today, we launch RegCheck V2.
🧵
regcheck.app
RegCheck
RegCheck is an AI tool to compare preregistrations with papers instantly.
regcheck.app
January 22, 2026 at 11:05 AM
Comparing registrations to published papers is essential to research integrity - and almost no one does it routinely because it's slow, messy, and time-demanding.
RegCheck was built to help make this process easier.
Today, we launch RegCheck V2.
🧵
regcheck.app
RegCheck was built to help make this process easier.
Today, we launch RegCheck V2.
🧵
regcheck.app
Reposted by Anne Scheel
Journals that work hard to meet the needs of both authors and readers should be acknowledged publicly — encouraging others to follow suit, says researcher Jennifer A. Byrne
go.nature.com/4sgKWzg
go.nature.com/4sgKWzg
Put pressure on publishers to follow best practice — external regulation is the answer
Journals that work hard to meet the needs of both authors and readers should be acknowledged publicly — encouraging others to follow suit.
go.nature.com
December 31, 2025 at 11:10 AM
Journals that work hard to meet the needs of both authors and readers should be acknowledged publicly — encouraging others to follow suit, says researcher Jennifer A. Byrne
go.nature.com/4sgKWzg
go.nature.com/4sgKWzg
Reposted by Anne Scheel
Ultimately, we may care more about the specific mechanisms than about isolating period effects. After all, those are what we can intervene on! But we can't really make progress on them just looking at variables over time and suggesting that changes are attributable to one thing or another.
January 15, 2026 at 1:24 PM
Ultimately, we may care more about the specific mechanisms than about isolating period effects. After all, those are what we can intervene on! But we can't really make progress on them just looking at variables over time and suggesting that changes are attributable to one thing or another.
Reposted by Anne Scheel
The 0.5.0 version of my {brms} + {tidyverse} translation of McElreath's "Statistical Rethinking" (2nd ed) is up!
solomon.quarto.pub/sr2/
1/3
#rstats
solomon.quarto.pub/sr2/
1/3
#rstats
Statistical rethinking 2 with rstan and the tidyverse
solomon.quarto.pub
January 14, 2026 at 3:08 PM
The 0.5.0 version of my {brms} + {tidyverse} translation of McElreath's "Statistical Rethinking" (2nd ed) is up!
solomon.quarto.pub/sr2/
1/3
#rstats
solomon.quarto.pub/sr2/
1/3
#rstats
Reposted by Anne Scheel
There's still time to get your nominations in the UKRN Dorothy Bishop Prize to celebrate early career researchers!
Nominations close this Sunday, 18 January 2026.
Nominations close this Sunday, 18 January 2026.
Nominations for the UKRN Dorothy Bishop Prize 2026 have opened!
Named after @deevybee.bsky.social, the prize, first awarded in 2022, celebrates the contributions of early career researchers to research improvement.
Nominations close 18 January 2026.
#AcademicSky #Research
Named after @deevybee.bsky.social, the prize, first awarded in 2022, celebrates the contributions of early career researchers to research improvement.
Nominations close 18 January 2026.
#AcademicSky #Research
Dorothy Bishop Prize 2026
www.ukrn.org
January 14, 2026 at 11:10 AM
There's still time to get your nominations in the UKRN Dorothy Bishop Prize to celebrate early career researchers!
Nominations close this Sunday, 18 January 2026.
Nominations close this Sunday, 18 January 2026.
Reposted by Anne Scheel
> "There is no suggestion of malpractice"
The article's pubpeer thread would strongly disagree.
pubpeer.com/publications...
The article's pubpeer thread would strongly disagree.
pubpeer.com/publications...
January 14, 2026 at 9:29 AM
> "There is no suggestion of malpractice"
The article's pubpeer thread would strongly disagree.
pubpeer.com/publications...
The article's pubpeer thread would strongly disagree.
pubpeer.com/publications...