Lightnews — Scholar-powered news

Olmo van den Akker @denolmo.bsky.social · 7d

Hi GRIOS, is the contact form on your website functional? I sent a message just after Metascience25 but haven't heard back yet.

1

Olmo van den Akker @denolmo.bsky.social · 20d

@michelenuijten.bsky.social

1

Reposted by Olmo van den Akker

Research on Research Institute (RoRI) @rorinstitute.bsky.social · 29d

🔓 It's great to see authors sharing their experiences with publishing on MetaROR (MetaResearch Open Review) — our open review platform for metascience using the publish–review–curate model: www.openscience.nl/en/cases/the...

3 4

Olmo van den Akker @denolmo.bsky.social · 29d

Here's another conference that aims to bridge fields: errorsin.science/pse8/

In Leiden from 11-13 Feb 2026 (submission deadline 15 October)

Perspective on Scientific Error – 8th Perspectives on Scientific Error Workshop

errorsin.science

1

Reposted by Olmo van den Akker

Brian Nosek @briannosek.bsky.social · Sep 4

We are about a month away from releasing a complete refresh of the OSF user interface. The team has been working on this for a very long time, and we are very excited to be able to share it soon. A preview picture:

10 31 150

Olmo van den Akker @denolmo.bsky.social · Sep 4

- Journals should state what their aims and scope are from the outset and implement mechanisms to assess whether they achieve those aims. This could also be things like "we want to publish high risk research"
- Meta-research is necessary to find out which journals deserve prestige

#PRC10

2

Olmo van den Akker @denolmo.bsky.social · Sep 4

- "The replication crisis forced changes in transparency for the research itself, but not for the publication process"
- We need to raise our expectations for journals? How? Nullius in verba (don't take their word for it!)

#PRC10

1 2

Olmo van den Akker @denolmo.bsky.social · Sep 4

Many interesting tidbits in @simine.com's talk. A selection:
- Journal prestige depends on factors like aims and scope, selectivity, and impact factor, but changes in these factors do not always lead to changes in journal prestige - journal prestige is sticky

#PRC10

1 2

Olmo van den Akker @denolmo.bsky.social · Sep 4

There is also a publish-review-curate publishing platform specifically dedicated to meta-research: metaror.org

Send your studies on peer review there and be part of the future of science!

(CoI statement: I'm an ERC representative at MetaROR)

#PRC10

3 6

Olmo van den Akker @denolmo.bsky.social · Sep 4

eLife (talk by Nicola Adamson) uses a publish-review-curate method and uses common terms to assess manuscripts.

For strength of evidence: exceptional, compelling, convincing, solid, incomplete, & inadequate

For significance of findings: landmark, fundamental, important, valuable, & useful

#PRC10

2 1 5

Olmo van den Akker @denolmo.bsky.social · Sep 4

New peer review dataset incoming!

Involves authors, topic area, editorial decision, author characteristics (institutional prestige, region, gender), BoRE evaluations, review characteristics (length, sentiment, z-score, reviewer gender).

(Talk by Aaron Clauset)

#PRC10

4

Olmo van den Akker @denolmo.bsky.social · Sep 4

Christos Kotanidis checked differences in abstracts between submissions and published papers & assessed whether these differences indicated higher or lower research quality.

Abstracts typically improved, especially in big five medical journals. Evidence for the effectiveness of peer review?

#PRC10

1

Olmo van den Akker @denolmo.bsky.social · Sep 4

Andrea Corvillon on distributed vs. panel peer review at the ALMA Observatory:

Most experienced PIs no longer have the best ranks in a distributed review system, but why that is remains unclear.

#PRC10

1

Olmo van den Akker @denolmo.bsky.social · Sep 4

Interesting to see that the conference review process (and publishing norms) are do different in the field of computer science compared to other fields.

How do these differences come about? Fundamental differences between fields or chance and inertia?

#PRC10

1 1

Olmo van den Akker @denolmo.bsky.social · Sep 4

Alexander Goldberg did it by a 7-point Likert scale for overall review quality but also by assessing 4 sub-categories: reviewers' understanding of the paper, whether important elements were covered, whether reviewers substantiated their comments, and the constructiveness of reviewer comments.

Olmo van den Akker @denolmo.bsky.social · Sep 4

Di Girolamo explains why the use of the phrase "to our knowledge" lacks reproducibility and accountability.

Good trigger to make an edit in a grant proposal I'm writing.

#PRC10

3

Olmo van den Akker @denolmo.bsky.social · Sep 4

Note by Yulin Yu: Data repurposing may serve as an essential driving mechanism driving scientific innovation BUT may not always garner immediate recognition.

Olmo van den Akker @denolmo.bsky.social · Sep 4

Data repurposing: taking existing data and reusing it for a different purpose.

(Presentation by Yulin Yu)

Studies repurposing data are at higher risk of bias, so make sure to preregister them (check here for a template): research.tilburguniversity.edu/en/publicati...

#PRC10

1 1 9

Olmo van den Akker @denolmo.bsky.social · Sep 4

Different findings in terms of time and industry funding than in an earlier meta-analysis by Robert Thibault and others: www.medrxiv.org/content/10.1...

Can this discrepancy be explained by the use of AI?

#PRC10

www.medrxiv.org

1

Olmo van den Akker @denolmo.bsky.social · Sep 4

Ian Bulovic used OpenAI's GPT to assess selective outcome reporting.

Findings:
- Much outcome switching but decrease over time
- Industry-sponsored trials most at risk
- Assessing outcome switching may seem trivial but is even hard for human coders

#PRC10

1

Olmo van den Akker @denolmo.bsky.social · Sep 4

Ioannidis: Do we enough evidence for your proposed actions to improve peer review?

Macleod: The evidence is thin, partly because many journals are hesitant to accommodate meta-research, like RCTs.

#PRC10

1 4

Olmo van den Akker @denolmo.bsky.social · Sep 4

A meta-perspective by Malcolm Macleod on the presentations at #PRC10.

Are we going for low hanging fruit too much in research on peer review / publication?

1 3 5

Olmo van den Akker @denolmo.bsky.social · Sep 4

A question to kickstart day 2 of #PRC10:

How would you measure the quality of peer reviews in a scientific study?

Single question? Scale? How many raters? AI?

1 2

Olmo van den Akker @denolmo.bsky.social · Sep 3

Leslie McIntosh:

Markers of (dis)trust in science: Pay attention to email addresses (use of hotmail.com and underscores) and institutional affiliations (new and unknown organizations without verifiable addresses)

#PRC10

1 1 6

Olmo van den Akker @denolmo.bsky.social · Sep 3

See also this piece by Tony Ross-Hellauer and Serge Horbach: doi.org/10.1371/jour...

Open peer review urgently requires evidence: A call to action

Open Peer Review is gaining prominence in attention and use, but to responsibly open up peer review there is an urgent need for additional evidence. This Perspective proposes a preliminary research ag...

doi.org

1