Lightnews — Scholar-powered news

James E. Pustejovsky

@jepusto.bsky.social

1.5K followers 780 following 280 posts

Statistician interested in meta-analysis, data science, R, special education. Associate Professor at UW Madison. Also @[email protected] https://jepusto.com

jepusto.com

Posts Media Videos Starter Packs

Pinned

James E. Pustejovsky @jepusto.bsky.social · Nov 13

Okay made a starter pack for folks into evidence synthesis, systematic review, meta-analysis. I've surely missed people, so please ping with suggestions or self-nominations.
go.bsky.app/JLw77VZ

21 35 60

James E. Pustejovsky @jepusto.bsky.social · 6d

Thank you!

James E. Pustejovsky @jepusto.bsky.social · 6d

Here's your utterly inconsequential statistics trivia question for the day. (I ask because I don't know but would like to find appropriate keywords.) Complete the analogies:

L1 loss : Laplace distribution
as
L2 loss : Gaussian distribution
as
L3 loss : ???
as
L4 loss : ???

Reposted by James E. Pustejovsky

Carlton J. Fong @carltonfong.bsky.social · 7d

As co-chair of AERA's @srma-sig.bsky.social, I am pleased to announce our Fall 2025 webinar series focused on meta-analysis and systematic reviews!

On Friday (Oct 3), our first webinar will be given by James Pustejovsky @jepusto.bsky.social! 🎉

Register here: us06web.zoom.us/meeting/regi...

7 14

Reposted by James E. Pustejovsky

Brian Nosek @briannosek.bsky.social · 7d

Two open positions on COS's research team!

Project Coordinator: Undergraduate degree in research, or equivalent experience ats.rippling.com/cos-careers/...

Program Manager: 10 yrs of project management experience or 2+ program management ats.rippling.com/cos-careers/...

Please share w/colleagues

2 21 28

James E. Pustejovsky @jepusto.bsky.social · 14d

Without maintaining the constant shift assumption, I understand MW’s U as a test of stochastic dominance, which is something that some statisticians get into on the weekends with other consenting statisticians.

a woman in a catsuit says thank you

ALT: a woman in a catsuit says thank you

media.tenor.com

1 3

James E. Pustejovsky @jepusto.bsky.social · 14d

As far as I understand, inference for the median requires assuming a constant shift in the distribution (e.g., a unit-constant treatment effect), in which case you can also interpret MW’s U as a test of mean differences, or differences in 3rd quartiles, or 42nd percentiles.

James E. Pustejovsky @jepusto.bsky.social · 14d

Mann-Whitney U is not a test of medians.

patrick star from spongebob squarepants is standing in front of a building and says fite me

ALT: patrick star from spongebob squarepants is standing in front of a building and says fite me

media.tenor.com

1 2

Reposted by James E. Pustejovsky

Fred Oswald @foswald.bsky.social · 24d

Psychological Methods invites early-career psychologists to apply to be a 2026 Editorial Fellow. This year kicked off our EF program -- it was enriching for the EFs and rewarding for everyone involved. Let's do it again!

For details:
www.apa.org/pubs/journal...

@apajournals.bsky.social

www.apa.org

14 17

Reposted by James E. Pustejovsky

Gravel Influencer @gravelinfluencer.bsky.social · 28d

Everyone should really listen to this episode. One of the things I appreciate most about The War on Cars is that so much of what they discuss is applicable not just to urbanism/livable streets issues, but to broader progressive causes. A good reminder of how important it is to dismantle car culture.

The War on Cars @thewaroncars.bsky.social · 29d

NEW: @bradlander.bsky.social joins us to talk political courage, from defending bike lanes early in his career to his efforts to rein in reckless drivers to his cross endorsement of @zohrankmamdani.bsky.social. We are thrilled to welcome Brad to The War on Cars!

podcasts.apple.com/us/podcast/p...

3 18 120

James E. Pustejovsky @jepusto.bsky.social · Aug 21

And @jrzhang.bsky.social and I have been thinking about PPCs for evaluating meta-analytic models, especially around choices of effect metric.

James E. Pustejovsky @jepusto.bsky.social · Aug 21

For sure this is one direction I have in mind. @paulinagrekov.bsky.social and I have been writing about PPCs as a bridge for helping applied researchers (w/o much statistical training, from Special Education) interpret and evaluate Bayesian GLMMs. jepusto.com/publications...

A gentle introduction to Bayesian posterior predictive checking for single-case researchers – James E. Pustejovsky

Education Statistics and Meta-Analysis

jepusto.com

1 1

James E. Pustejovsky @jepusto.bsky.social · Aug 21

For sure. Psychometricians will talk about predictive validity (or predictive validation evidence) but that does quite capture the vibe of what I mean.

James E. Pustejovsky @jepusto.bsky.social · Aug 21

I’m not familiar. Will look it up!

James E. Pustejovsky @jepusto.bsky.social · Aug 21

(As oppose to using a model merely as a conventional summary of a dataset, or worse, a traditional incantation one recites over one’s data before offering it up for publication.)

1 1

James E. Pustejovsky @jepusto.bsky.social · Aug 21

Stats Q: Is there a word or succinct phrase for the discipline or humility that comes from having to use a statistical model to actually make predictions about future data?

4 3

Reposted by James E. Pustejovsky

Chanda Prescod-Weinstein 🌌 @chanda.blacksky.app · Aug 16

This should get WIDE circulation:
MIT stating that it “has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper.”

gizmodo.com/mit-backs-aw...

MIT Backs Away From Paper Claiming Scientists Make More Discoveries with AI

The retracted paper had impressed a Nobel Prize winner in economics.

gizmodo.com

46 1.7K 3.6K

James E. Pustejovsky @jepusto.bsky.social · Aug 12

I've been playing with it too, with pretty good results, but I'm still struggling with latex table layout and placement. Have you had success using it with flextable or any other tools for programmatically generating tex tables?

James E. Pustejovsky @jepusto.bsky.social · Aug 9

It’s not a surprise. I’d say methodology folks were well aware that p-curve had problems since McShane, Bockenholt, & Hansen 2016 doi.org/10.1177/1745.... The contribution of the new paper is to build up theory for *why* and what specific features of the method create problems.

Adjusting for Publication Bias in Meta-Analysis - Blakeley B. McShane, Ulf Böckenholt, Karsten T. Hansen, 2016

We review and evaluate selection methods, a prominent class of techniques first proposed by Hedges (1984) that assess and adjust for publication bias in meta-an...

doi.org

1 4

James E. Pustejovsky @jepusto.bsky.social · Aug 9

T&F is a zombie method. I still see it used regularly in meta-analysis, sometimes without any other publication bias analysis methods. (Though perhaps less so in mainstream psych, and not really ever for forensic meta-science.)

James E. Pustejovsky @jepusto.bsky.social · Aug 8

@richarddmorey.bsky.social @clintin.bsky.social would you consider doing trim-and-fill next pleeezzzz?

1 3

James E. Pustejovsky @jepusto.bsky.social · Aug 8

Also this...

Given what is needed to improve the P-curve tests, we do not recommend their use in their current form. Their statistical properties are problematic and it is not clear what substantive conclusions they afford. Given the stated purpose of the P-curve—evaluating the trustworthiness of scientific literatures—the stakes are too high to use tests with such poor, or poorly-understood, properties.

1 4

James E. Pustejovsky @jepusto.bsky.social · Aug 8

This is an insightful and timely analysis. This point in the conclusions resonated

Simulation is a powerful tool and can help build intuition, but it is not a substitute for formal analysis. Simulation may provide hints of problems with a procedure, but only if the simulator’s formal knowledge helps guide the choice of simulations. A simulator might quit after running a few simulations that tell them what they think is true while problems remain uncovered. Given the implications of poor forensic procedures for science, all such procedures demand deeper formal scrutiny.

2 1 11

Reposted by James E. Pustejovsky

Richard D. Morey @richarddmorey.bsky.social · Aug 8

Paper drop, for anyone interested in #metascience, #statistics, or #metaanalysis! @clintin.bsky.social and I show in a new paper in JASA that the P-curve, a popular forensic meta-analysis method, has deeply undesirable statistical properties. www.tandfonline.com/doi/full/10.... 1/?

Cover page for the manuscript: Morey, R. D., & Davis-Stober, C. P. (2025). On the poor statistical properties of the P-curve meta-analytic procedure. Journal of the American Statistical Association, 1–19. https://doi.org/10.1080/01621459.2025.2544397

Abstract for the paper: The P-curve (Simonsohn, Nelson, & Simmons, 2014; Simonsohn, Simmons, & Nelson, 2015) is a widely-used suite of meta-analytic tests advertised for detecting problems in sets of studies. They are based on nonparametric combinations of p values (e.g., Marden, 1985) across significant (p < .05) studies and are variously claimed to detect “evidential value”, “lack of evidential value”, and “left skew” in p values. We show that these tests do not have the properties ascribed to them. Moreover, they fail basic desiderata for tests, including admissibility and monotonicity. In light of these serious problems, we recommend against the use of the P-curve tests.

17 120 290

James E. Pustejovsky @jepusto.bsky.social · Aug 1

Oh snap, should have started my advice with "Arguably, starting a sentence..."

James E. Pustejovsky @jepusto.bsky.social · Aug 1

Apropos of nothing, here's James' writing advice for Friday:

Starting a sentence with "Arguably,...." is basically just signaling to the reader that you're about to start bullshitting them. Try replacing with "So my garbage take is..." and see if the rest of the paragraph still works.

1 1