Matt Makel
@mattmakel.bsky.social
educational researcher focusing on academic talent (identification and development) and open science research practices (implementation and assessment). Now in Canada @UCalgary
Pinned
Matt Makel
@mattmakel.bsky.social
· Dec 17
New Publication alert: Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists [https://journals.sagepub.com/doi/10.1177/00169862251392934] #OpenAccess
Reposted by Matt Makel
New: "Effect of Artificial Intelligence on Learning: A Meta-Meta-Analysis" by Wagenmakers and colleagues revealing evidence for "severe publication bias and extreme between-study heterogeneity" in existing meta-analyses of the effects of AI on learning: osf.io/preprints/ps...
OSF
osf.io
February 10, 2026 at 2:52 PM
New: "Effect of Artificial Intelligence on Learning: A Meta-Meta-Analysis" by Wagenmakers and colleagues revealing evidence for "severe publication bias and extreme between-study heterogeneity" in existing meta-analyses of the effects of AI on learning: osf.io/preprints/ps...
Reposted by Matt Makel
New blog post, inspired by the excellent recent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research.
I reflect on how I will incorporate realist ontologies in my own qualitative research.
daniellakens.blogspot.com/2026/02/on-r...
I reflect on how I will incorporate realist ontologies in my own qualitative research.
daniellakens.blogspot.com/2026/02/on-r...
On the reliability and reproducibility of qualitative research
With my collaborators, I am increasingly performing qualitative research. I find qualitative research projects a useful way to improve my un...
daniellakens.blogspot.com
February 8, 2026 at 7:46 AM
New blog post, inspired by the excellent recent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research.
I reflect on how I will incorporate realist ontologies in my own qualitative research.
daniellakens.blogspot.com/2026/02/on-r...
I reflect on how I will incorporate realist ontologies in my own qualitative research.
daniellakens.blogspot.com/2026/02/on-r...
Reposted by Matt Makel
New #RSOS paper: ‘Don’t hate the players, hate the game’: qualitative insights from education researchers on questionable and open research practices. Read more: doi.org/10.1098/rsos... @mattmakel.bsky.social @sarahcaroleo.bsky.social @jesse-fleming.bsky.social @bryancook.bsky.social
February 7, 2026 at 8:01 PM
New #RSOS paper: ‘Don’t hate the players, hate the game’: qualitative insights from education researchers on questionable and open research practices. Read more: doi.org/10.1098/rsos... @mattmakel.bsky.social @sarahcaroleo.bsky.social @jesse-fleming.bsky.social @bryancook.bsky.social
The Wikipedia page for Mozart Effect had a whole section on its popularization but don't overlook the section just after it on political impact.
February 2, 2026 at 1:25 PM
The Wikipedia page for Mozart Effect had a whole section on its popularization but don't overlook the section just after it on political impact.
54 weeks later, this preprint is now published: royalsocietypublishing.org/rsos/article...
January 29, 2026 at 2:46 PM
54 weeks later, this preprint is now published: royalsocietypublishing.org/rsos/article...
Reposted by Matt Makel
Need some tips for searching PsyArXiv? Wrote a blog post blog.psyarxiv.com/2026/01/27/h...
How to search for PsyArXiv preprints on the OSF platform – PsyArXiv Blog
PsyArXiv is hosted on the Open Science Framework (OSF) platform, and over the years their interface and search syntax has changed. Guidance on how to search is a snapshot in time. Presently, the searc...
blog.psyarxiv.com
January 29, 2026 at 2:13 PM
Need some tips for searching PsyArXiv? Wrote a blog post blog.psyarxiv.com/2026/01/27/h...
You may be eligible for $700 each! But that takes a phone call (I have had less productive phone calls)
December 31, 2025 at 9:24 PM
You may be eligible for $700 each! But that takes a phone call (I have had less productive phone calls)
Do you know the reason for the original delay? In Canada passengers can be eligible for compensation if the delay is not weather related: Compensation for flight delays and cancellations | Air Passenger Protection share.google/TOYuIi0pTs5N...
Compensation for flight delays and cancellations | Air Passenger Protection
Compensation for flight delays and cancellations Airlines must provide compensation for the inconvenience of flight cancellations and flight delays of 3 hours or more and if the disruption is within t...
share.google
December 31, 2025 at 6:53 PM
Do you know the reason for the original delay? In Canada passengers can be eligible for compensation if the delay is not weather related: Compensation for flight delays and cancellations | Air Passenger Protection share.google/TOYuIi0pTs5N...
Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists journals.sagepub.com/doi/10.1177/...
Sage Journals: Discover world-class research
Subscription and open access journals from Sage, the world's leading independent academic publisher.
journals.sagepub.com
December 17, 2025 at 8:40 PM
Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists journals.sagepub.com/doi/10.1177/...
Something I learned: some German colleagues did not know the title was a reference to a song!
Popular rendition: www.youtube.com/watch?v=uXK4...
My fav version: www.youtube.com/watch?v=76WF...
Popular rendition: www.youtube.com/watch?v=uXK4...
My fav version: www.youtube.com/watch?v=76WF...
Santa Claus is Comin' to Town | Fred Astaire | TUNE
YouTube video by TUNE - Musical Moments
www.youtube.com
December 17, 2025 at 8:39 PM
Something I learned: some German colleagues did not know the title was a reference to a song!
Popular rendition: www.youtube.com/watch?v=uXK4...
My fav version: www.youtube.com/watch?v=76WF...
Popular rendition: www.youtube.com/watch?v=uXK4...
My fav version: www.youtube.com/watch?v=76WF...
What’s needed?
Before we start using characteristics lists to make decisions or inform others, any list must:
A. Gather Empirical Evidence Including Prevalence Rates from gifted and typical students
B. Connect Lists With Specific Definitions, Domains, and Identification Practices.
Before we start using characteristics lists to make decisions or inform others, any list must:
A. Gather Empirical Evidence Including Prevalence Rates from gifted and typical students
B. Connect Lists With Specific Definitions, Domains, and Identification Practices.
December 17, 2025 at 8:38 PM
What’s needed?
Before we start using characteristics lists to make decisions or inform others, any list must:
A. Gather Empirical Evidence Including Prevalence Rates from gifted and typical students
B. Connect Lists With Specific Definitions, Domains, and Identification Practices.
Before we start using characteristics lists to make decisions or inform others, any list must:
A. Gather Empirical Evidence Including Prevalence Rates from gifted and typical students
B. Connect Lists With Specific Definitions, Domains, and Identification Practices.
For example, “having two eyes” is surely quite common in gifted students. But because the prevalence rate is likely indistinguishable from non-gifted students, it is not a particularly useful descriptor of gifted students.
December 17, 2025 at 8:38 PM
For example, “having two eyes” is surely quite common in gifted students. But because the prevalence rate is likely indistinguishable from non-gifted students, it is not a particularly useful descriptor of gifted students.
Limitation 6: Lack of Awareness of Prevalence Rates.
Lists typically do not cite: primary research, prevalence rates, or compares gifted and non-gifted students. To call a feature a “characteristic”, we must know the prevalence of that feature in both gifted and non-gifted students.
Lists typically do not cite: primary research, prevalence rates, or compares gifted and non-gifted students. To call a feature a “characteristic”, we must know the prevalence of that feature in both gifted and non-gifted students.
December 17, 2025 at 8:37 PM
Limitation 6: Lack of Awareness of Prevalence Rates.
Lists typically do not cite: primary research, prevalence rates, or compares gifted and non-gifted students. To call a feature a “characteristic”, we must know the prevalence of that feature in both gifted and non-gifted students.
Lists typically do not cite: primary research, prevalence rates, or compares gifted and non-gifted students. To call a feature a “characteristic”, we must know the prevalence of that feature in both gifted and non-gifted students.
Limitation 5: Self-Fulfilling Prophecy and Bias.
Any list created based on observation will reflect all biases that were part of the initial identification process.
Any list created based on observation will reflect all biases that were part of the initial identification process.
December 17, 2025 at 8:36 PM
Limitation 5: Self-Fulfilling Prophecy and Bias.
Any list created based on observation will reflect all biases that were part of the initial identification process.
Any list created based on observation will reflect all biases that were part of the initial identification process.
Limitation 4: Lack of conceptual clarity.
Many items are vague and difficult to differentiate from other items. jingle/jangle fallacies: Are “high level of language development” and “high level of verbal ability” different characteristics or different examples of the same latent characteristic?
Many items are vague and difficult to differentiate from other items. jingle/jangle fallacies: Are “high level of language development” and “high level of verbal ability” different characteristics or different examples of the same latent characteristic?
December 17, 2025 at 8:36 PM
Limitation 4: Lack of conceptual clarity.
Many items are vague and difficult to differentiate from other items. jingle/jangle fallacies: Are “high level of language development” and “high level of verbal ability” different characteristics or different examples of the same latent characteristic?
Many items are vague and difficult to differentiate from other items. jingle/jangle fallacies: Are “high level of language development” and “high level of verbal ability” different characteristics or different examples of the same latent characteristic?
Limitation 3: Lack of Alignment With Identification and Selection Practices. Different ID criteria ID different different students. eg: the top 1% of students is not the same as the top 10%. Bc dif practices identify dif students we can’t assume lists developed using 1 set of criteria generalizes
December 17, 2025 at 8:35 PM
Limitation 3: Lack of Alignment With Identification and Selection Practices. Different ID criteria ID different different students. eg: the top 1% of students is not the same as the top 10%. Bc dif practices identify dif students we can’t assume lists developed using 1 set of criteria generalizes
Limitation 2: Lack of alignment with Definitions and Domains.
Different definitions of giftedness identify different students. Just like basketball coaches ID different students than theater directors. Any list using one definition/domain does not automatically generalize to others
Different definitions of giftedness identify different students. Just like basketball coaches ID different students than theater directors. Any list using one definition/domain does not automatically generalize to others
December 17, 2025 at 8:33 PM
Limitation 2: Lack of alignment with Definitions and Domains.
Different definitions of giftedness identify different students. Just like basketball coaches ID different students than theater directors. Any list using one definition/domain does not automatically generalize to others
Different definitions of giftedness identify different students. Just like basketball coaches ID different students than theater directors. Any list using one definition/domain does not automatically generalize to others
Limitation 1: lack empirical evidence.
Many lists give no development background or provide anecdotes. Others that do include citations don’t cite primary research. And many that do cite primary research cite things like case studies or research that did not compare gifted with typical students.
Many lists give no development background or provide anecdotes. Others that do include citations don’t cite primary research. And many that do cite primary research cite things like case studies or research that did not compare gifted with typical students.
December 17, 2025 at 8:30 PM
Limitation 1: lack empirical evidence.
Many lists give no development background or provide anecdotes. Others that do include citations don’t cite primary research. And many that do cite primary research cite things like case studies or research that did not compare gifted with typical students.
Many lists give no development background or provide anecdotes. Others that do include citations don’t cite primary research. And many that do cite primary research cite things like case studies or research that did not compare gifted with typical students.
My argument: To be useful, characteristics lists must be based on more than good intentions. They need empirical support. Without it, lists will not help schools and can exacerbate inequity and distrust in research. Calling something a characteristic is a privilege that must be empirically earned.
December 17, 2025 at 8:28 PM
My argument: To be useful, characteristics lists must be based on more than good intentions. They need empirical support. Without it, lists will not help schools and can exacerbate inequity and distrust in research. Calling something a characteristic is a privilege that must be empirically earned.
Characteristics lists are everywhere. Government agency websites, school district sites, advocacy groups, popular press books, textbooks, and across the internet. But what about the evidence supporting these lists?
a cartoon of a man holding a scroll with the letter j written on it
ALT: a cartoon of a man holding a scroll with the letter j written on it
media.tenor.com
December 17, 2025 at 8:28 PM
Characteristics lists are everywhere. Government agency websites, school district sites, advocacy groups, popular press books, textbooks, and across the internet. But what about the evidence supporting these lists?
New Publication alert: Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists [https://journals.sagepub.com/doi/10.1177/00169862251392934] #OpenAccess
December 17, 2025 at 8:27 PM
New Publication alert: Making a List Requires Checking it Twice: A Call for Empirical Evidence in Characteristics Lists [https://journals.sagepub.com/doi/10.1177/00169862251392934] #OpenAccess
Reposted by Matt Makel
Job opportunity — Junior Professorship in Psychological Metascience @zpid.bsky.social leibniz-psychology.onlyfy.jobs/job/10kku5n7 h/t @bethclarke.bsky.social
November 26, 2025 at 3:10 AM
Job opportunity — Junior Professorship in Psychological Metascience @zpid.bsky.social leibniz-psychology.onlyfy.jobs/job/10kku5n7 h/t @bethclarke.bsky.social
Reposted by Matt Makel
🏆 Individual: @simine.com, psychologist at @unimelb.bsky.social & editor-in-chief of Psychological Science, is recognized for pioneering methodological rigor, reproducibility & collaborative research, driving initiatives such as @improvingpsych.org & the journal Collabra @ucpress.bsky.social. (2/5)
November 24, 2025 at 10:00 AM
🏆 Individual: @simine.com, psychologist at @unimelb.bsky.social & editor-in-chief of Psychological Science, is recognized for pioneering methodological rigor, reproducibility & collaborative research, driving initiatives such as @improvingpsych.org & the journal Collabra @ucpress.bsky.social. (2/5)