Lightnews — Scholar-powered news

Benjamin Kunc @benjaminkunc.bsky.social · 5d

Cool poster!

If the effect is robust, could you calculate the financial costs of increasing the statistical power in ESM studies by 1%? (And then perhaps create a little interactive ✨ Shiny app ✨ so researchers could calculate it themselves?)

Benjamin Kunc @benjaminkunc.bsky.social · 7d

Is there any documented procedure how to implement this for multilevel cfa?

Benjamin Kunc @benjaminkunc.bsky.social · 9d

If anyone has any ideas how the items could be improved, please let us know.

I'm not planning on improving them, but someone else might 🙃

Benjamin Kunc @benjaminkunc.bsky.social · 9d

Sorry...except for the momentary quality of online solitude. That scale doesn't work at all :)

1

Benjamin Kunc @benjaminkunc.bsky.social · 9d

Our interpretation is that the scales aren't completely useless, but they need revision, if anyone wants to use them.

See the preprint yourself: osf.io/preprints/ps...

OSF

osf.io

1

Benjamin Kunc @benjaminkunc.bsky.social · 9d

When we assessed the scales' measurement invariance across several groups, we found that none of the scales function equally among the tested subpopulations.

1

Benjamin Kunc @benjaminkunc.bsky.social · 9d

We used multilevel confirmatory factor analyses first on the whole sample (N = 1,913 adolescents), which yielded positively looking results. However...

1

Benjamin Kunc @benjaminkunc.bsky.social · 9d

We assessed the structural validity of four ESM scales measuring: the quality of current social company, the quality of current online company, the quality of in-person solitude, and (!) the quality of online solitude.

1

Benjamin Kunc @benjaminkunc.bsky.social · 9d

Are you an ESM item measuring the quality of social experiences looking for validation? 👀

Because if you are, I've got some bad news for you. Our new preprint is out: osf.io/preprints/ps...

@gudruneisele.bsky.social @ginettelafit.bsky.social @lisapeeters.bsky.social @oliviajkirtley.bsky.social

OSF

osf.io

2 5 6

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

For anyone interested in reading it, here is the link:
benjaminkunc.substack.com/p/farewell-d...

I will be glad for any thoughts you might have. Enjoy!

Farewell, dear psych

I wish you all the best. But I really need to go.

benjaminkunc.substack.com

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

Later, I found myself coming back to some of the thoughts I've had about the current state of psychology and its metascience. Since I wanted the post to be a conclusion of my psychological journey, I felt I needed to write it all down.

1

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

When I attempted to write down the reasoning behind it, I realized it is too long for a regular LinkedIn post (or even a bluesky thread!). As a result, I ended up with a full Substack blog consisting of two main parts. The first part is about (you guessed it) why I dropped the PhD.

1

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

The decision to drop my PhD might be surprising to some of you who weren't lucky enough to run away before I started rambling about methodology and psychological metascience.

1

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

First of all, I want to thank for the invaluable supervision and advice I got from Olivia J Kirtley, Gudrun Eisele, and Ginette Lafit throughout my PhD, and the whole Centre for Contextual Psychiatry for the opportunity to work with such amazing colleagues.

1

Benjamin Kunc @benjaminkunc.bsky.social · Aug 26

Farewell, dear psych

This year, I have made the difficult decision to end my PhD project, move from Belgium back to Czechia, and take an indeterminate break from psychology.

1 3

Benjamin Kunc @benjaminkunc.bsky.social · Jun 2

"Altogether, these findings point to the strength of most contemporary psychological research and suggest academic incentives have begun to promote such research. However, there remain key questions about the extent to which robustness is truly valued compared with other research aspects."

Benjamin Kunc @benjaminkunc.bsky.social · Jun 2

It's nice to find this post a few moments after discussing paper mills, academic incentives, and peer review.

Brian Nosek @briannosek.bsky.social · Jun 2

The proportion of p-values between .01 and .05 declined in psychology over the last decade, a strengthening of credibility.

Papers with fewer fragile p-values are cited more. Higher ranked journals assoc. with > improvement, but not higher ranked universities.

journals.sagepub.com/doi/10.1177/...

1

Benjamin Kunc @benjaminkunc.bsky.social · Apr 11

Wow. The correlation of replication success with IF seems to be positive, while it's negative for citations. That's the opposite of what I expected.

1 2

Reposted by Benjamin Kunc

ReproducibiliTea Leuven @leuventea.bsky.social · Apr 9

Join our workshop "Reproducibility Made Easy: Open Resources for All Disciplines"! 🧪🔬

📅 Date: 6th May 📍 KU Leuven Open Science Day

Dive into cross-disciplinary #openscience resources and boost your research #reproducibility!

Sign up: tinyurl.com/tfs5n7kb
Learn more: tinyurl.com/28pcr324

Reproducibility made easy: Open resources for all disciplines

An interactive workshop by KU Leuven ReproducibiliTea, designed to guide researchers in exploring and discussing cross-disciplinary open science resources to enhance research reproducibility.

tinyurl.com

4 4

Benjamin Kunc @benjaminkunc.bsky.social · Apr 3

IMHO, many researchers (implicitly) assume that positive results imply a successful measurement process, leading to the intuition that a thorough validation is unnecessary in such cases.

This would be reasonable, if we could trust our findings. Which doesn's seem to be the case. 4/4

Benjamin Kunc @benjaminkunc.bsky.social · Apr 3

There's also a slightly edgy take on finding positive results and the criterion validity of the scales used. 3/4

1

Benjamin Kunc @benjaminkunc.bsky.social · Apr 3

One of the points is that if one is about to commit factor analysis, it's best to first check the validity evidence based on content and response processes. Otherwise, one could end up with compelling, yet meaningless, statistical results. 2/4

1

Benjamin Kunc @benjaminkunc.bsky.social · Apr 3

👀Blogpost on measurement👀

I would have rather concluded that we don't really need factor analysis and can just rely on vibes (or previous literature). But here we are: "Factor analysis: Overrated, Misused, But Still Useful." 1/4

The ESM Item Repository @esmrepository.bsky.social · Apr 3

📣 New blog post! 📣 @ccp-kuleuven.bsky.social's @benjaminkunc.bsky.social gets into the nitty-gritty of factor analysis and its use and misuse: esmitemrepositoryinfo.com/blog-posts #Measurement #ExperienceSamplingMethod

Blog posts | ESM Item Repository

esmitemrepositoryinfo.com

1 1 2

Reposted by Benjamin Kunc

Milla Pihlajamäki @millapihlajamaki.bsky.social · Feb 27

The first preprint from my PhD is out: osf.io/preprints/ps...! 🥳

We explored the temporal dynamics of four careless responding indicators (response time, within-beep standard deviation, an inconsistency index, occasion-person correlation) in ESM data across different samples.

Thread below🧵

OSF

osf.io

6 15 47

Reposted by Benjamin Kunc

Daniel Lakens @lakens.bsky.social · Feb 19

People have already blamed science reform for what is happening.

For 15 years I have said: If we do not get our shit together (less publication bias, higher quality, more coordination) someone else is going to implement change top down, and we are not going to like how they do it.

And here we are.

5 6 35