Surprisingly never in the case of publication bias tests :D

This is also likely to be the last update of this version of the package. Next year, I will introduce breaking changes to the interface with the 4.0 major release, which will make the interface much more similar to metafor.

As such, it provides an easy-to-apply state-of-the-art Bayesian meta-analytic methodology for most meta-analytic settings!

See an overview of the current functionality with a brief description of all vignettes fbartos.github.io/RoBMA/articl...

Yep, its ridiculous. Those studies should not be published...

Extracting the study-level data from existing meta-analyses is quite feasible, so, there is almost no excuse not to do so.

Also, you cannot really evaluate between-study heterogeneity, see e.g. our latest study-level meta-meta-analysis that shows the limitations of the previous meta-analysis-level meta-meta-analysis doi.org/10.31234/osf...

My main worry is that they might have synthesized the meta-analytic estimates rather than the study-level estimates? The manuscript wasn't super clear on that and the OSF had only meta-analysis level data?
If so, that makes the publication bias adjustment ineffective...

Also, this should not be a reason to stop exercising.
1) There are other benefits of exercise
2) Some populations/exercises show benefit
3) There might be wider effects on cognition; however, the literature is too heterogeneous and contaminated with publication bias to be certain

I think that the field needs to clean up the published literature a bit. Additional small studies are not going to move the needle at this point; maybe a couple of large-scale, pre-registered studies might provide more insight?

We also re-analyzed all of the original meta-analyses individually. Many of them are consistent with publication bias: the evidence for and the degree of the pooled effects decrease once publication bias is adjusted for.

We run subgroup analyses for each outcome/population/intervention. We found that most results are too heterogeneous to tell (see wide prediction intervals), but some interventions seem to be promising and some have substantive evidence against them. See figures for each outcome.

First, we found notable publication bias, especially in studies on general cognition and executive function. Importantly, there was extreme between-study heterogeneity (tau ~ 0.3-0.6!). This means that the results were consistent with both large benefit but also large harm.

We were not the only ones to notice, also see @matthewbjane.bsky.social commenting on this when the study came out:
x.com/MatthewBJane...

So, we manually extracted the study-level data from the included meta-analyses and re-evaluated the evidence.

Previous meta-meta-analysis (doi.org/10.1136/bjsp...) indicated consistent benefits of exercise for cognitive benefits across all domains and populations. However, it synthesized meta-analytic estimates and, as such, it could not adjust for publication nor evaluate heterogeneity.