Lightnews — Scholar-powered news

Rich Lucas @richlucas.bsky.social · 5d

And I, too, would think I was beating a dead horse if it weren't for the fact that top journals keep publishing simple lag-1 CLPMs with no controls (meta-analytic CLPMs are still CLPMs): www.apa.org/pubs/journal...

www.apa.org

4

Rich Lucas @richlucas.bsky.social · 5d

Reposting this thread about another recent preprint that discusses some of the reasons why: bsky.app/profile/rich...

Rich Lucas @richlucas.bsky.social · 18d

Interested in models used to estimate lagged effects in panel data? We (@rebiweidmann.bsky.social, Hyewon Yang) have a new paper looking at patterns of stability and their implications for bias and model choice: osf.io/preprints/ps... [1/x]

OSF

osf.io

1 3

Rich Lucas @richlucas.bsky.social · 5d

Rates of significant effects are also arguably too high with the RI-CLPM. Individual effects are about half as likely to be significant with the RI-CLPM as with the CLPM, but you still get at least one significant lagged effect in 61% of models.

1 2

Rich Lucas @richlucas.bsky.social · 5d

We already know that lagged effects in CLPMs are likely to be upwardly biased, but just how easy is it to find significant effects? Way too easy. I tested CLPMS in 100 randomly selected pairs of correlated variables and found significant effects in 98 of them. New preprint: osf.io/preprints/ps...

OSF

osf.io

4 16 33

Rich Lucas @richlucas.bsky.social · 18d

Anyway, we welcome any thoughts or suggestions you have about the paper! [15/15]

2

Rich Lucas @richlucas.bsky.social · 18d

Also: state variance is not just measurement error, so using latent variables with the CLPM or RI-CLPM won't always fix this (though it helps). Even with multiple-item Big Five scores modeled using latent traits, the same pattern emerged. [14/x]

Patterns of stability over increasingly long lags for the Big Five personality traits. The slope of these stability coefficients is very shallow.

1 1

Rich Lucas @richlucas.bsky.social · 18d

Our analyses also suggest that once you account for state variance, stabilities of variables included in these panel studies are extremely high: Median 1-year stability = .90. This suggests that there is often very little change occurring that could be accounted for by a lagged effect [13/x].

1 1

Rich Lucas @richlucas.bsky.social · 18d

Our analyses suggest that the CLPM and RI-CLPM would be the most appropriate model for just 4% of variables we examined; the other 96% were divided equally between ARTS and STARTS [12/x]

1 1

Rich Lucas @richlucas.bsky.social · 18d

What type of model can account for these patterns? Models that include a state component (like the STARTS or even the ARTS, which drops the stable trait). You can reproduce actual patterns of stability even without assuming the existence of a stable trait (this plot is for the ARTS) [11/x]

Implied and actual stability coefficients for a 10-year ARTS model. The implied and actual stability coefficients are very similar.

1 1

Rich Lucas @richlucas.bsky.social · 18d

It turns out it's very common. Here are the plots of stability over increasingly long lags for about 400 variables from a large panel study. For anything less than almost perfect short-term stability, stability coefficients should reach an asymptote long before 22 years; very few do [10/x]

Actual stability coefficients across increasingly long lags for over 400 variables. Most stabilities decline slowly without ever reaching an asymptote.

1 1

Rich Lucas @richlucas.bsky.social · 18d

One pattern it can't handle is when short-term stability is moderate, medium-term stability is just slightly lower than long-term stability, and there is no clear asymptote with increasingly long lags (as is true with life satisfaction and health). How commons is this pattern? [9/x]

1 1

Rich Lucas @richlucas.bsky.social · 18d

Why doesn't the RI-CLPM just estimate a lower asymptote (corresponding to less stable trait variance)? It turns out that the RI-CLPM is quite limited in the types of patterns of stability with which it is compatible (won't go into details here, but paper does). [8/x]

1 1

Rich Lucas @richlucas.bsky.social · 18d

This gets even worse with more waves of data. In our experience, this is a pretty common pattern: If you have more than 4 or 5 waves, the RI-CLPM overestimates long-term stability (and often underestimates medium-term stability). This misfit can also lead to bias in estimates [7/x]

Implied and actual stability coefficients for a 10-wave RI-CLPM. Evidence of misspecification emerges at longer lags.

1 1

Rich Lucas @richlucas.bsky.social · 18d

The RI-CLPM does better, but even here, there is a hint of misspecification (see how stability coefficients continue to decline even after the RI-CLPM predicts an asymptote) [6/x]

Implied and actual stability coefficients for the RI-CLPM. These values are close to one another with a hint of misspecification at long lags.

1 1

Rich Lucas @richlucas.bsky.social · 18d

Here's a plot of actual stability coefficients and those implied by a fitted CLPM with life satisfaction and health. It's really bad...CLPM dramatically underestimates stability. This misspecification introduces bias, which can be severe [5/x]

Implied and actual stability coefficients over five years for life satisfaction and health. The implied stability coefficients are much lower than the actual stability coefficients.

1 1

Rich Lucas @richlucas.bsky.social · 18d

There are already lots of good papers comparing these models from a causal inference perspective, but we focus more on typical patterns of stability in real data and how well they match with the patterns of stability implied by these models. Why is this useful? Consider the CLPM. [4/x]

1 2

Rich Lucas @richlucas.bsky.social · 18d

TL/DR: More reasons why the standard lag-1 CLPM is really bad, but also some important concerns about models that assume the existence of a stable trait (like the RI-CLPM). Our analyses suggest that we should be paying more attention to state variance when modeling these effects [3/x]

1 2

Rich Lucas @richlucas.bsky.social · 18d

We try to do three new things. 1. Focus explicitly on model misfit and its implications for bias. 2. Conduct simulations showing how specific forms of misfit affect bias. 3. Analyze the longitudinal structure of over 400 variables to see what type of misfit is likely for specific models. [2/x]

1 1

Rich Lucas @richlucas.bsky.social · 18d

Interested in models used to estimate lagged effects in panel data? We (@rebiweidmann.bsky.social, Hyewon Yang) have a new paper looking at patterns of stability and their implications for bias and model choice: osf.io/preprints/ps... [1/x]

OSF

osf.io

1 12 21

Rich Lucas @richlucas.bsky.social · Aug 12

If you're looking for a good quarto extension to do APA-style documents, this one does it! Seems to have everything papaja had for Rmarkdown and more. Really useful and easy to implement (including great export to docx): wjschne.github.io/apaquarto/

Introduction to apaquarto – APA Style Documents with apaquarto

wjschne.github.io

1 6 21

Rich Lucas @richlucas.bsky.social · Feb 12

Will get right on that (checks to see whether @syeducation.bsky.social has something I can use as a template that I can pass on to @jnfrltackett.bsky.social )

1 4

Reposted by Rich Lucas

mark brandt @mjbsp.bsky.social · Nov 3

The problems w/ two-wave cross-lagged models are becoming widely known. Some proposed a "triangulation method" (2nd screenshot) to identify spurious effects in such designs

We show that this test doesn't work in plausible situations. It shouldn't be used.

journals.sagepub.com/doi/10.1177/...

Abstract: The cross-lagged panel model (CLPM) is an analytic technique used to examine the reciprocal causal effects of two or more variables assessed on two or more occasions. Although widely used, the CLPM has been criticized for relying on implausible assumptions, the violation of which can often lead to biased estimates of causal effects. Recently, a triangulation method has been proposed to identify spurious effects in simple CLPM analyses
(e.g., Sorjonen & Melin, 2024b). We use simulations and a discussion of the formulas underlying regression coefficients to show that this method does not provide a valid indicator of spuriousness. This method identifies true causal effects as spurious in realistic situations and should not be used to diagnose whether a causal effect estimated from the CLPM is spurious or not. There are clear reasons to doubt causal estimates from the CLPM, but the results of the triangulation method do not add information about whether such estimates are spurious.

4 14 34

Rich Lucas @richlucas.bsky.social · Apr 23

Me, trying to rerun analyses from a years-old paper for a request from a meta-analyist

Liz Neeley @lizneeley.bsky.social · Apr 23

Just incredible news on reviving Voyager I.

Think of it: a computer chip 15 billion(!) miles away is broken. The solution is to repackage & move key software, w/ code sent via radio signal that takes 22+ hrs to reach that little 46yr old machine.

AND IT IS WORKING.

blogs.nasa.gov/voyager/2024...

NASA’s Voyager 1 Resumes Sending Engineering Updates to Earth – Voyager

blogs.nasa.gov

1 6

Rich Lucas @richlucas.bsky.social · Feb 23

Collabra: Psychology is looking for new associate editors for the Social Psychology section. If you're interested, please apply! rug.eu.qualtrics.com/jfe/form/SV_...

1 40 24

Reposted by Rich Lucas

Michael D. Krämer @mdkraemer.bsky.social · Feb 12

My first Skeet! Happy to announce that our paper on life events and life satisfaction is now published at the European Journal of Personality (w/ @dingdingpeng.the100.ci, @richlucas.bsky.social, and @drichter77.bsky.social)! journals.sagepub.com/doi/10.1177/... (preprint: osf.io/preprints/ps... ).

1 2 14