Lightnews — Scholar-powered news

Brent W. Roberts @bwroberts.bsky.social · 16m

Kudos for pushing this through to publication. Null findings are always interesting if the questions you ask are interesting, like this one. And, in terms of a null being constructive, think of all the future researchers who won't go down this path and will instead test a different idea.

1

Reposted by Brent W. Roberts

Matt Southward @mattsouthward.bsky.social · 1d

When we measure personality multiple times in a study, does it matter if we ask people about their personality *in general* or *since the last time point*?

Turns out: yes!

We found differences in internal consistency, Ms, & SDs but not in the underlying constructs 🧵

doi.org/10.31234/osf...

https://doi.org/10.31234/osf.io/tb94v_v1🧵

1 9 42

Brent W. Roberts @bwroberts.bsky.social · 3d

Don't tell me. I want to believe...

1 1

Brent W. Roberts @bwroberts.bsky.social · 3d

The tour was great. The company and conversation better.

2

Brent W. Roberts @bwroberts.bsky.social · 3d

I went to Rotterdam for a reproducibility therapy session with @lakens.bsky.social. I am happy to report that the patient is doing much better—maybe due to a placebo effect. Who knows.

2 1 27

Brent W. Roberts @bwroberts.bsky.social · 6d

No irony in the fact that the best way to convince people to use OS is to have 1) a compelling spokesperson, 2) a stimulating anecdote, 3) a convincing data point, and 4) an overgeneralization. We are, after all, human. (That, btw, is the Gladwell formula).

2

Brent W. Roberts @bwroberts.bsky.social · 6d

Once the story is published, god forbid you would tell them the method was flawed since it already worked for them once. Then you find the PI arguing for the method regardless of what is right. 3/

2

Brent W. Roberts @bwroberts.bsky.social · 6d

Smart people can get all or most of the fancy models to work on the data whether it makes sense to or not. Once you've done that and little asterisks float to the surface, the PI takes a microsecond to glom onto them and write a story. 2/

1 1

Brent W. Roberts @bwroberts.bsky.social · 6d

And, having been in your position before, I know that if you come back to them with the fact that their inspiration to use CLPM or some other model is unfounded, you soon find yourself not included on the next grant/paper. 1/

1 1

Brent W. Roberts @bwroberts.bsky.social · 6d

Don't make me blog again...it is not doomerism in this case, it is simply the typical way we employ stats which is in a thoughtless fashion. There are some places where CLPM is entirely appropriate as well as RI-CLPM. It's just that we almost never do studies that fit with the models.

2 2

Brent W. Roberts @bwroberts.bsky.social · 6d

Sean you took the bait! For the most part, no model is viable, largely because the problem is not with the statistical models it is with the data, our methods, our research questions and how they don't align with any of the statistical models to address the questions we really want to answer.

2 4

Brent W. Roberts @bwroberts.bsky.social · 6d

Good scientific practices get in the way of a clean narrative. A clean narrative is everything.

1 9

Brent W. Roberts @bwroberts.bsky.social · 6d

We are bad storytellers who use anecdata to hide our stilted prose.

1 4

Brent W. Roberts @bwroberts.bsky.social · 6d

RI-CLPM is just as intellectually bankrupt as CLPM. Don’t be fooled by the fact that some think it is more acceptable.

1

Brent W. Roberts @bwroberts.bsky.social · 6d

Yup

1 4

Brent W. Roberts @bwroberts.bsky.social · 6d

This is THE reason people hold on to, rationalize, argue for, won't give up on CLPM. It gets you those happy little asterisks that get you free entry into your preferred journal.

Rich Lucas @richlucas.bsky.social · 6d

We already know that lagged effects in CLPMs are likely to be upwardly biased, but just how easy is it to find significant effects? Way too easy. I tested CLPMS in 100 randomly selected pairs of correlated variables and found significant effects in 98 of them. New preprint: osf.io/preprints/ps...

OSF

osf.io

2 1 23

Brent W. Roberts @bwroberts.bsky.social · 7d

es ist sehr schön....

2

Brent W. Roberts @bwroberts.bsky.social · 7d

Some people might wonder why I'm always hanging out in Germany only to realize that you folks collect better data than we do thus obviating the need to collect our own data. Okay, no-one really wonders why I like to hang out in Germany...

1 1

Reposted by Brent W. Roberts

Julia M. Rohrer @dingdingpeng.the100.ci · 8d

A lot of psych is already conducted with online convenience samples & ppl are probably excited about silicon samples bc it would allow them to crank out more studies for even less 💸

How about we reconsider the idea that sciencey science involves collecting own data.
www.science.org/content/arti...

AI-generated ‘participants’ can lead social science experiments astray, study finds

Data produced by “silicon samples” depends on researchers’ exact choice of models, prompts, and settings

www.science.org

13 71 230

Reposted by Brent W. Roberts

GESIS - Leibniz-Institut für Sozialwissenschaften @gesis.org · 7d

#PIAAC #PIAAC2025 #livestream #day2

Join us at the International PIAAC Research Conference 2025 - Tune in today and watch the livestream of conference day 2 starting at 9 a.m.

👉Check out the program:
📌www.gesis.org/en/dat...

👉Watch the livestream:
📌www.gesis.org/piaac-...

1 1

Reposted by Brent W. Roberts

Bastian Jaeger @bxjaeger.bsky.social · 15d

This review suggests that the average effectiveness of choice architecture interventions on behaviour is smaller than often reported and that there is substantial heterogeneity in their effects
@szaszibarnabas.bsky.social

www.nature.com/articles/s44...

5 4

Reposted by Brent W. Roberts

Egon Dejonckheere @dejonckheeregon.bsky.social · 23d

Cool preprint from @ajwright.bsky.social et al. demonstrating that personality change occurs at different time scales!

www.researchgate.net/profile/Aman...

www.researchgate.net

1 2 5

Reposted by Brent W. Roberts

Lukas Röseler @aufdroeseler.bsky.social · Sep 3

At the FORRT Replication Hub, our mission is to support researchers who want to replicate previous findings. We have now published a big new component with which we want to fulfill this mission: An open access handbook for reproduction and replication studies: forrt.org/replication_...

2 36 60

Reposted by Brent W. Roberts

Julia M. Rohrer @dingdingpeng.the100.ci · 27d

New paper out with @boryslaw.bsky.social 🥳 In which we sketch out how to rethink measurement invariance causally for applied researchers. And provide a causal definition of measurement invariance!

www.sciencedirect.com/science/arti...

Rethinking measurement invariance causally

Highlights:
It is preferable to work with a causal definition of measurement invariance
A violation of measurement invariance is a potentially substantively interesting observation
Standard tests for measurement invariance rely on strong assumptions
Group differences can be thought of as descriptive results

Conceptual graph illustration the central points of the manuscript. A group variable is potentiall connected to a construct of interest which affects items. Measurement invariance is violated if the group variable directly affects the items, for example by modifying the loadings from the construct to the items, or by directly affecting an item

To make this less abstract, consider a scenario where students take an exam, R, meant to capture some ability, T, and then are admitted to a program, V, depending on their exam results: R → V. This is sufficient to result in a violation of the statistical definition of measurement invariance. Exam results and admission are not independent given ability because exam results have a direct effect on admission. Even if we know somebody’s ability (e.g., we know it’s very high), learning about their admission status (e.g., they were not admitted) can tell us something about their exam result (e.g., it may have been worse than expected). According to the causal definition, this in itself does not constitute measurement bias, which seems a sensible conclusion here. After all, the scenario does not involve any reason to believe that the measurement process varied systematically by admission status. Admission happens after the exams took place, it cannot retroactively influence the measurement process (and, for example, lead to unfair treatment depending on admission status).

3 36 110

Reposted by Brent W. Roberts

Michael Inzlicht @minzlicht.bsky.social · 28d

Major new paper by finds implicit measures like the IAT are no better than asking people directly about their biases. After decades of avoiding self-reports, turns out our sophisticated replacement tools work no better than what we abandoned. New post!

The Great Implicit Bias Bamboozle

Where were you when you first learned about implicit bias?

open.substack.com

2 25 70