Sean Harrison, PhD
@sean-h.bsky.social
Pinned
Sean Harrison, PhD
@sean-h.bsky.social
· Jun 14
I was saving this for an article, but I want something to refer to when I talk about how I think the best way to fix peer-review is to abolish journals.
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
When I read through parts of the Cass review when it came out, I thought the same as Chris here.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
"The Cass review did not find puberty blockers unsafe; it found the evidence base to be weak." Letter in today's Times
December 16, 2025 at 2:47 PM
When I read through parts of the Cass review when it came out, I thought the same as Chris here.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
I went back in time as far as I could go, and the first post was DELETED.
bsky.app/profile/mlob...
Damn.
bsky.app/profile/mlob...
Damn.
December 15, 2025 at 8:40 PM
I went back in time as far as I could go, and the first post was DELETED.
bsky.app/profile/mlob...
Damn.
bsky.app/profile/mlob...
Damn.
I despise that "novelty" is a driver of anything to do with publication in epidemiology.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
Analyses of open health datasets can offer valuable contributions to health & medicine, but recent years have seen a proliferation of papers lacking robust or novel findings. In this Editorial, we provide guidance for conducting & reporting high-quality analyses using open datasets🧪
plos.io/4poBmZo
plos.io/4poBmZo
Setting the standard for high-quality studies using open health datasets
Large open health datasets present unique opportunities for studies that when well-designed, conducted, and reported, can offer valuable contributions to health and medicine. However, recent years hav...
plos.io
December 15, 2025 at 3:08 PM
I despise that "novelty" is a driver of anything to do with publication in epidemiology.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
Reposted by Sean Harrison, PhD
King Charles argued on Friday that, thanks to early diagnosis, his treatment will be reduced in the New Year.
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
King Charles 'deeply touched' by reaction to cancer TV message, says Buckingham Palace
In a TV broadcast on Friday night, the King said an early diagnosis was key to the "good news" that his treatment is being scaled back.
www.bbc.co.uk
December 14, 2025 at 8:02 AM
King Charles argued on Friday that, thanks to early diagnosis, his treatment will be reduced in the New Year.
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
Reposted by Sean Harrison, PhD
So...my undergrad thesis student is doing a quality analysis of studies found in meta-analyses. She identified a few and we contacted the authors to request their effect sizes and other variables for the studies in their papers.
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
The Impact of Social Media on Adolescent Mental Health: A Meta-Analysis | Scientia Psychiatrica
Introduction: The proliferation of social media has raised significant concerns about its potential effects on the mental health of adolescents. This meta-analysis aims to provide a comprehensive asse...
scientiapsychiatrica.com
December 7, 2025 at 11:11 PM
So...my undergrad thesis student is doing a quality analysis of studies found in meta-analyses. She identified a few and we contacted the authors to request their effect sizes and other variables for the studies in their papers.
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
December 1, 2025 at 9:23 AM
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
November 28, 2025 at 1:55 PM
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
one of my most contrarian thoughts is to tax IHT at 100% - & use the money to lower tax on actually working for a living
November 24, 2025 at 3:18 PM
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Most scientists don't understand how effect sizes work and are therefore far too quick to dismiss "small" effects.
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
November 24, 2025 at 9:12 AM
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Reposted by Sean Harrison, PhD
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
October 9, 2025 at 4:01 PM
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Y'all. I just got ChatGPT to do everything in R for this manuscript. I mean EVERYTHING. And it's all legit and reproducible. I'm shook.
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
Statistics in the era of AI
How do we mentor, teach, and do stats when AI can do so much of the work?
scienceforeveryone.science
October 9, 2025 at 3:10 PM
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
I read close to 50 papers in order to give an evidence-based, accurate response to Monday's press conference.
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
All About the Tylenol-Autism Brouhaha
At an unhinged press conference last Monday, Donald Trump and his medical acolytes revealed an actionable cause of autism at long last: Tylenol. In what felt like a social media post come to life, Tru...
www.mcgill.ca
September 29, 2025 at 9:57 AM
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Can large language models stand in for human participants?
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
The threat of analytic flexibility in using large language models to simulate human data: A call to attention
Social scientists are now using large language models to create "silicon samples" - synthetic datasets intended to stand in for human respondents, aimed at revolutionising human subjects research. How...
arxiv.org
September 18, 2025 at 9:29 AM
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Reposted by Sean Harrison, PhD
John Finnemore's Souvenir Programme back for 2025
John Finnemore's Souvenir Programme back for 2025
The best of the week's comedy on TV, radio and streaming
dlvr.it
August 24, 2025 at 10:03 AM
John Finnemore's Souvenir Programme back for 2025
p=1, bitches
New rule: Not engaging with people who can not understand that when you randomize, the pre-experiment population means in each group are the same and that any differences are due to sampling variability.
August 16, 2025 at 6:46 PM
p=1, bitches
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
August 15, 2025 at 12:38 PM
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
this is excellent. some junior colleagues keep thinking high IF = good paper, and im trying to explain repeatedly that this is not a good way to appraise a publication
This is an excellent illustration of how useless the "Impact Factor" metric is.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
August 14, 2025 at 11:03 AM
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
Reposted by Sean Harrison, PhD
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...
August 12, 2025 at 9:54 AM
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...
To win a bet, I once flipped a random coin I had in my wallet 20 times, and had it land on heads every time.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
I don't understand much of the stats here or probably all of the implications, but I love this! Great work @fbartos.bsky.social and all.
I also hope there are no triallists still clinging on to coin-flipping as a means of randomisation...
I also hope there are no triallists still clinging on to coin-flipping as a means of randomisation...
Fair coins tend to land on the same side they started: evidence from 350,757 flips.
That's the title of our paper summarizing ~650 hours of coin-tossing experimentation just published in the Journal of the American Statistical Association.
doi.org/10.1080/0162...
That's the title of our paper summarizing ~650 hours of coin-tossing experimentation just published in the Journal of the American Statistical Association.
doi.org/10.1080/0162...
August 11, 2025 at 8:40 PM
To win a bet, I once flipped a random coin I had in my wallet 20 times, and had it land on heads every time.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
The person who wrote this list seems to have no idea either what each of these jobs entails or the limits of LLMs.
tbf it could be an AI-generated list.
That'd be on-brand.
tbf it could be an AI-generated list.
That'd be on-brand.
Generative AI is the biggest bubble for some time. It can help with certain tasks, but my experience is that it cannot do even the most basic things unsupervised and regularly fabricates its work.
August 5, 2025 at 8:59 AM
The person who wrote this list seems to have no idea either what each of these jobs entails or the limits of LLMs.
tbf it could be an AI-generated list.
That'd be on-brand.
tbf it could be an AI-generated list.
That'd be on-brand.
Reposted by Sean Harrison, PhD
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
The Uncertain Origins of Aspirin
The history of humanity’s pharmacopeia is often muddied by folklore. What can the origins of aspirin teach us about separating fact from fiction?
press.asimov.com
July 14, 2025 at 4:23 PM
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
There was Guardian opinion piece about remove age limits from voting entirely: www.theguardian.com/books/2025/j...
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
July 17, 2025 at 11:15 AM
There was Guardian opinion piece about remove age limits from voting entirely: www.theguardian.com/books/2025/j...
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
Ha, possibly a true dichotomy:
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
Britons are more regular swearers than Australians or Americans, but what are their top swear words?
Britons
F*ck: 67% use regularly
Sh*t: 65%
Bloody: 54%
Australians
Sh*t: 61%
F*ck: 56%
Bullsh*t: 52%
Americans
Damn: 55%
Sh*t: 54%
F*ck: 47%
👇 more in chart below
yougov.co.uk/society/arti...
Britons
F*ck: 67% use regularly
Sh*t: 65%
Bloody: 54%
Australians
Sh*t: 61%
F*ck: 56%
Bullsh*t: 52%
Americans
Damn: 55%
Sh*t: 54%
F*ck: 47%
👇 more in chart below
yougov.co.uk/society/arti...
July 15, 2025 at 9:58 AM
Ha, possibly a true dichotomy:
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!