Steve Haroz
@steveharoz.com
Visual perception and cognition scientist
(he/him)
My site: http://steveharoz.com
R guide: https://r-guide.steveharoz.com
StatCheck Simple: http://statcheck.steveharoz.com
(he/him)
My site: http://steveharoz.com
R guide: https://r-guide.steveharoz.com
StatCheck Simple: http://statcheck.steveharoz.com
Pinned
Steve Haroz
@steveharoz.com
ยท Mar 14
Invalid Conclusions Built on Statistical Errors
When you see the statistical result of an experiment like p = 0.003 or a 95% confidence interval of , you might assume a certain clarity and definitiveness in those results. A null effect is very unli...
steveharoz.com
BLOG POST: Invalid Conclusions Built on Statistical errors
If you see a small p-value or a large separation in confidence intervals, you may assume that an effect is reliable. But overly simple reporting may be hiding serious conclusion-altering errors.
Here are some example scenarios:
1/๐งต #stats
If you see a small p-value or a large separation in confidence intervals, you may assume that an effect is reliable. But overly simple reporting may be hiding serious conclusion-altering errors.
Here are some example scenarios:
1/๐งต #stats
I am happy to share that our preprint โ๐ช๐ผ๐ฟ๐ธ๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐๐ถ๐ฟ๐ฐ๐๐น๐ฎ๐ฟ ๐๐ฎ๐๐ฎ: ๐ ๐ง๐๐๐ผ๐ฟ๐ถ๐ฎ๐น ๐ณ๐ผ๐ฟ ๐๐ผ๐ด๐ป๐ถ๐๐ถ๐๐ฒ ๐ฎ๐ป๐ฑ ๐๐ฒ๐ต๐ฎ๐๐ถ๐ผ๐ฟ๐ฎ๐น ๐ฅ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ตโ is now out.
Huge thanks to @bayslab.org, Julie de Falco, Zahara, @cjungerius.bsky.social, @ivntmc.bsky.social, Adam, and Xiaolu for the lovely collaboration.
doi.org/10.31234/osf...
Huge thanks to @bayslab.org, Julie de Falco, Zahara, @cjungerius.bsky.social, @ivntmc.bsky.social, Adam, and Xiaolu for the lovely collaboration.
doi.org/10.31234/osf...
OSF
doi.org
January 13, 2026 at 5:18 AM
Reposted by Steve Haroz
๐ Sleuthing challenge ๐
How many suspicious patterns can you find in this table?
(You don't need to know what the variables are.)
Hints in next post! ๐
How many suspicious patterns can you find in this table?
(You don't need to know what the variables are.)
Hints in next post! ๐
January 11, 2026 at 6:14 AM
๐ Sleuthing challenge ๐
How many suspicious patterns can you find in this table?
(You don't need to know what the variables are.)
Hints in next post! ๐
How many suspicious patterns can you find in this table?
(You don't need to know what the variables are.)
Hints in next post! ๐
I suspect that a lot of these errors are due to people using educational materials or example code from fields where within-subject and repeated measures are uncommon.
January 11, 2026 at 4:06 PM
I suspect that a lot of these errors are due to people using educational materials or example code from fields where within-subject and repeated measures are uncommon.
#statstab #460 Invalid Conclusions Built on Statistical Errors
Thoughts: It's easy to make analysis mistakes, providing code should be the norm.
#analysis #pseudoreplication #error #design #code #guide #educational
steveharoz.com/blog/2024/wr...
Thoughts: It's easy to make analysis mistakes, providing code should be the norm.
#analysis #pseudoreplication #error #design #code #guide #educational
steveharoz.com/blog/2024/wr...
Invalid Conclusions Built on Statistical Errors
When you see p = 0.003 or a 95% confidence interval of , you might assume a certain clarity and definitiveness. A null effect is very unlikely to yield those results, right? But be careful! Such overl...
steveharoz.com
January 11, 2026 at 6:24 AM
Tiptoeing back onto social media after a year away.
January 6, 2026 at 10:08 PM
Tiptoeing back onto social media after a year away.
Reposted by Steve Haroz
RFK Jr is a consequence of decades of anti-science media, promoted by both the right and left. Subtle forms of anti-sciencism like organic food or raw milk or "removing toxins" snowball over time. Calling them out in their early phases may be critical.
The media outlets rightly admonishing RFK Jr. need to reckon with their own roll in his rise to prominence.
In 2005 Rolling Stone and Salon would publish his op-ed โDeadly Immunityโ linking vaccines to autism in children, based on tiny - later retracted - British study.
In 2005 Rolling Stone and Salon would publish his op-ed โDeadly Immunityโ linking vaccines to autism in children, based on tiny - later retracted - British study.
December 9, 2024 at 5:40 AM
RFK Jr is a consequence of decades of anti-science media, promoted by both the right and left. Subtle forms of anti-sciencism like organic food or raw milk or "removing toxins" snowball over time. Calling them out in their early phases may be critical.
Reposted by Steve Haroz
After spending more than an hour on the phone with Microsoft Support, I have learned:
1. It is impossible to disable Copilot in OneNote, Excel, PowerPoint, or Windows itself.
2. It will not become possible to do so for another month AT THE EARLIEST.
(1/?)
1. It is impossible to disable Copilot in OneNote, Excel, PowerPoint, or Windows itself.
2. It will not become possible to do so for another month AT THE EARLIEST.
(1/?)
Anyone know how to disable Copilot in OneNote? There's no tickybox for it in the "Options" page, I already turned off "online experiences" or whatever, and when I tried uninstalling the Copilot app, the only change seems to be that I no longer have an "uninstall" option on the Copilot app.
January 24, 2025 at 9:32 PM
After spending more than an hour on the phone with Microsoft Support, I have learned:
1. It is impossible to disable Copilot in OneNote, Excel, PowerPoint, or Windows itself.
2. It will not become possible to do so for another month AT THE EARLIEST.
(1/?)
1. It is impossible to disable Copilot in OneNote, Excel, PowerPoint, or Windows itself.
2. It will not become possible to do so for another month AT THE EARLIEST.
(1/?)
Reposted by Steve Haroz
Whenever I see these articles on 'publish or perish' culture being to blame, I always think of Tal Yarkoni's blog post: talyarkoni.org/blog/2018/10.... Yes, the pressures are there and they can force our hand, but that means we should work to improve the structures where we can (not abide by them).
No, itโs not The Incentivesโitโs you
Thereโs a narrative I find kind of troubling, but that unfortunately seems to be growing more common in science. The core idea is that the mere existence of perverse incentives is a valid andโฆ
talyarkoni.org
January 23, 2025 at 9:23 AM
Whenever I see these articles on 'publish or perish' culture being to blame, I always think of Tal Yarkoni's blog post: talyarkoni.org/blog/2018/10.... Yes, the pressures are there and they can force our hand, but that means we should work to improve the structures where we can (not abide by them).
Reposted by Steve Haroz
Are you not a fan of journals' "enhanced online PDF viewers"? Me neither. I put together a little Firefox add-on that helps you skip the "enhanced" reader and download the PDF directly.
January 23, 2025 at 10:08 AM
Are you not a fan of journals' "enhanced online PDF viewers"? Me neither. I put together a little Firefox add-on that helps you skip the "enhanced" reader and download the PDF directly.
You can't add numbers without checking for normality first!!!!
The first step of multiplication is checking for heteroscedasticity!!!
๐
The first step of multiplication is checking for heteroscedasticity!!!
๐
January 6, 2025 at 11:52 PM
You can't add numbers without checking for normality first!!!!
The first step of multiplication is checking for heteroscedasticity!!!
๐
The first step of multiplication is checking for heteroscedasticity!!!
๐
Reposted by Steve Haroz
Check out this incredible video from 1941 that details the physics of visual perception using the mechanics of the eyeball from Karl Kurt Bosse
pennstateoffice365-my.sharepoint.com/:v:/g/person...
pennstateoffice365-my.sharepoint.com/:v:/g/person...
Sign in to your account
pennstateoffice365-my.sharepoint.com
January 6, 2025 at 9:20 PM
Check out this incredible video from 1941 that details the physics of visual perception using the mechanics of the eyeball from Karl Kurt Bosse
pennstateoffice365-my.sharepoint.com/:v:/g/person...
pennstateoffice365-my.sharepoint.com/:v:/g/person...
Reposted by Steve Haroz
Hey #StatsSky, what are you favorite papers to cite when you need to justify something that is obvious (I once had a reviewer ask we justify the use of logistic regression on a binary outcome)
or when you need to push-back on silly reviewer requests (e.g., asking for p-values in table 1)?
or when you need to push-back on silly reviewer requests (e.g., asking for p-values in table 1)?
January 6, 2025 at 8:15 AM
Hey #StatsSky, what are you favorite papers to cite when you need to justify something that is obvious (I once had a reviewer ask we justify the use of logistic regression on a binary outcome)
or when you need to push-back on silly reviewer requests (e.g., asking for p-values in table 1)?
or when you need to push-back on silly reviewer requests (e.g., asking for p-values in table 1)?
Reposted by Steve Haroz
Flu cases have quadrupled over the past month in hospitals in England, with NHS leaders warning that โskyrocketingโ cases could make this winter โone of the worst we have ever seenโ. Just published @financialtimes.com: on.ft.com/4j4g0h5
January 3, 2025 at 12:41 PM
Flu cases have quadrupled over the past month in hospitals in England, with NHS leaders warning that โskyrocketingโ cases could make this winter โone of the worst we have ever seenโ. Just published @financialtimes.com: on.ft.com/4j4g0h5
While I understand the concern about motives, it shouldn't necessarily impeach any outcomes, as long as there isn't any indication of funder interference. Otherwise, the same concerns would hold for all research funded by the Trump administration (NIH, NSF, etc.).
January 5, 2025 at 12:28 PM
While I understand the concern about motives, it shouldn't necessarily impeach any outcomes, as long as there isn't any indication of funder interference. Otherwise, the same concerns would hold for all research funded by the Trump administration (NIH, NSF, etc.).
Reposted by Steve Haroz
Have tried to make the dish many times, with varying success. This is an important pepe:
"Finally, we present a scientifically optimized recipe based on our findings, enabling a consistently flawless execution of this classic dish."
"Finally, we present a scientifically optimized recipe based on our findings, enabling a consistently flawless execution of this classic dish."
1/n Some time ago my colleague, excellent cook, and friend Ivan told me: "Cacio e pepe is the recipe that I screw up more often. Let's make a project studying systematically the physics of that sauce".
Prepare to get cheesy, I'm glad to share the Cacio e paper preprint:
arxiv.org/abs/2501.00536
Prepare to get cheesy, I'm glad to share the Cacio e paper preprint:
arxiv.org/abs/2501.00536
January 5, 2025 at 10:19 AM
Have tried to make the dish many times, with varying success. This is an important pepe:
"Finally, we present a scientifically optimized recipe based on our findings, enabling a consistently flawless execution of this classic dish."
"Finally, we present a scientifically optimized recipe based on our findings, enabling a consistently flawless execution of this classic dish."
Reposted by Steve Haroz
Everyone is running away with this one, but I took the time to trace the source, and these are 80% confidence intervals, the true meta-science story (and there is one!), was written up as one about selective reporting and calibrating standard errors..
January 4, 2025 at 9:47 PM
Everyone is running away with this one, but I took the time to trace the source, and these are 80% confidence intervals, the true meta-science story (and there is one!), was written up as one about selective reporting and calibrating standard errors..
The NYT is my quintessential example of a service that I will never sign up for again. They make everything too difficult to quit.
January 4, 2025 at 5:27 PM
The NYT is my quintessential example of a service that I will never sign up for again. They make everything too difficult to quit.
I've settled on an approach to giving up on TV shows:
If a show has a serialized plot and goes more than 1 year between seasons, it's dead. It doesn't matter how much I like it, I cannot keep track of characters or follow a plot with a gap of more than one year.
If a show has a serialized plot and goes more than 1 year between seasons, it's dead. It doesn't matter how much I like it, I cannot keep track of characters or follow a plot with a gap of more than one year.
January 4, 2025 at 8:32 AM
I've settled on an approach to giving up on TV shows:
If a show has a serialized plot and goes more than 1 year between seasons, it's dead. It doesn't matter how much I like it, I cannot keep track of characters or follow a plot with a gap of more than one year.
If a show has a serialized plot and goes more than 1 year between seasons, it's dead. It doesn't matter how much I like it, I cannot keep track of characters or follow a plot with a gap of more than one year.
The way many US jobs come with medical coverage, this one also needs legal coverage.
January 3, 2025 at 11:21 PM
The way many US jobs come with medical coverage, this one also needs legal coverage.
Reposted by Steve Haroz
Researcher: "We let the data speak for itself."
Earlier that day:
Earlier that day:
January 2, 2025 at 3:31 PM
Researcher: "We let the data speak for itself."
Earlier that day:
Earlier that day:
Reposted by Steve Haroz
How can sharing our uncertainty with others alter our confidence when we're alone?
Delighted to share the lab's new paper, with Einar Andreassen and @cdfrith.bsky.social. Particularly pleased as it's Einar's first!
Link: osf.io/preprints/ps...
๐งต๐
#PsychSciSky
#neuroskyence
Delighted to share the lab's new paper, with Einar Andreassen and @cdfrith.bsky.social. Particularly pleased as it's Einar's first!
Link: osf.io/preprints/ps...
๐งต๐
#PsychSciSky
#neuroskyence
January 2, 2025 at 11:54 AM
How can sharing our uncertainty with others alter our confidence when we're alone?
Delighted to share the lab's new paper, with Einar Andreassen and @cdfrith.bsky.social. Particularly pleased as it's Einar's first!
Link: osf.io/preprints/ps...
๐งต๐
#PsychSciSky
#neuroskyence
Delighted to share the lab's new paper, with Einar Andreassen and @cdfrith.bsky.social. Particularly pleased as it's Einar's first!
Link: osf.io/preprints/ps...
๐งต๐
#PsychSciSky
#neuroskyence
That's a great start, but that law should apply to sales taxes too.
December 31, 2024 at 11:10 PM
That's a great start, but that law should apply to sales taxes too.