Jay Patel
@infotainment.bsky.social
🎷 vibe adulting
#HCI #PeerReview #SciPub
#toolsforthought #ResearchSynthesis
#OpenScience #MetaSci #FoSci
🔎 Research: ethnography of peer review
🧑🏫 Teaching: Stats, DataViz
🐢 UMD: College of Info
🌐 PhD Candidate: Info Studies / HCI + Data
🏝️ OASISlab
#HCI #PeerReview #SciPub
#toolsforthought #ResearchSynthesis
#OpenScience #MetaSci #FoSci
🔎 Research: ethnography of peer review
🧑🏫 Teaching: Stats, DataViz
🐢 UMD: College of Info
🌐 PhD Candidate: Info Studies / HCI + Data
🏝️ OASISlab
Pinned
Jay Patel
@infotainment.bsky.social
· Jun 11
🌐 It strikes me that crowdsourced evaluation/benchmarking of LLMs for scientific work would be valuable and possible in the style of ManyLabs projects.
If I coordinate crowds of volunteers to assess how well LLMs can evaluate papers, would you be interested?
#openscience #metascience #AI #LLM
If I coordinate crowds of volunteers to assess how well LLMs can evaluate papers, would you be interested?
#openscience #metascience #AI #LLM
Reposted by Jay Patel
🌏 A thought experiment for researchers:
If you could click one button to share your (non-sensitive) notes from your articles in Zotero, Mendeley, PaperPile, etc. would you open those data up to the world?
#metascience
If you could click one button to share your (non-sensitive) notes from your articles in Zotero, Mendeley, PaperPile, etc. would you open those data up to the world?
#metascience
July 5, 2024 at 8:49 PM
🌏 A thought experiment for researchers:
If you could click one button to share your (non-sensitive) notes from your articles in Zotero, Mendeley, PaperPile, etc. would you open those data up to the world?
#metascience
If you could click one button to share your (non-sensitive) notes from your articles in Zotero, Mendeley, PaperPile, etc. would you open those data up to the world?
#metascience
Me: Paper titles are hard, require sophistication, and must conform to sensible standards. A/B test them, go back to the drawing board, attend a seance, consult an astrologer, run simulations on supercomputers, and meditate until I achieve Nirvana.
Philosophers: "On [topic name here]"
Philosophers: "On [topic name here]"
November 25, 2025 at 2:02 AM
Me: Paper titles are hard, require sophistication, and must conform to sensible standards. A/B test them, go back to the drawing board, attend a seance, consult an astrologer, run simulations on supercomputers, and meditate until I achieve Nirvana.
Philosophers: "On [topic name here]"
Philosophers: "On [topic name here]"
Title and abstract: This is a "qualitative study."
Results: Everything is quantitative.
Am I missing something #medsky ? Wouldn't "content analysis" be more accurate?
Results: Everything is quantitative.
Am I missing something #medsky ? Wouldn't "content analysis" be more accurate?
November 24, 2025 at 10:33 PM
Title and abstract: This is a "qualitative study."
Results: Everything is quantitative.
Am I missing something #medsky ? Wouldn't "content analysis" be more accurate?
Results: Everything is quantitative.
Am I missing something #medsky ? Wouldn't "content analysis" be more accurate?
Reposted by Jay Patel
It would, of course, be great if the underlying results were made public but it looks like meta in 2019 ran withdrawal experiments to examine polarization and well being ahead of forging collaborations with academics. Seems like more than enough to design bias the collaborations towards nulls 🧵
November 24, 2025 at 12:46 PM
It would, of course, be great if the underlying results were made public but it looks like meta in 2019 ran withdrawal experiments to examine polarization and well being ahead of forging collaborations with academics. Seems like more than enough to design bias the collaborations towards nulls 🧵
This Thanksgiving, I'll be thankful for the time to test the newer foundation models. Three to choose from...
November 24, 2025 at 8:48 PM
This Thanksgiving, I'll be thankful for the time to test the newer foundation models. Three to choose from...
#SciComment, what do we think of the benefits of using Batman in metros to nudge riders?
Strong odds ratio, low p-value, but a priming claim in 2025 requires a bit more thought IMO.
www.nature.com/articles/s44...
Strong odds ratio, low p-value, but a priming claim in 2025 requires a bit more thought IMO.
www.nature.com/articles/s44...
November 22, 2025 at 2:44 AM
#SciComment, what do we think of the benefits of using Batman in metros to nudge riders?
Strong odds ratio, low p-value, but a priming claim in 2025 requires a bit more thought IMO.
www.nature.com/articles/s44...
Strong odds ratio, low p-value, but a priming claim in 2025 requires a bit more thought IMO.
www.nature.com/articles/s44...
I'm tempted to add a full page of quotes to my next preprint because it's too hard to choose among the great options and because I really want to hammer home some ideas that I fear readers won't get.
Is that too much?
Are there precedents in the literature?
Is that too much?
Are there precedents in the literature?
November 21, 2025 at 3:41 AM
I'm tempted to add a full page of quotes to my next preprint because it's too hard to choose among the great options and because I really want to hammer home some ideas that I fear readers won't get.
Is that too much?
Are there precedents in the literature?
Is that too much?
Are there precedents in the literature?
Reposted by Jay Patel
In a field where evidence suggests that a high proportion of papers have serious flaws, a database of "potentially important" papers which appear *not* to have serious flaws might be useful, as well as identifying those that do.
November 9, 2025 at 10:36 AM
In a field where evidence suggests that a high proportion of papers have serious flaws, a database of "potentially important" papers which appear *not* to have serious flaws might be useful, as well as identifying those that do.
Reposted by Jay Patel
I'll come back to this idea because that complements something that's been brewing in my mind for some time. Some needed resolution in our citation practices where we explicitly distinguish between "load-bearing" work on which our inferences rely and others (kitchen sink citations). And the need to
November 20, 2025 at 4:22 AM
I'll come back to this idea because that complements something that's been brewing in my mind for some time. Some needed resolution in our citation practices where we explicitly distinguish between "load-bearing" work on which our inferences rely and others (kitchen sink citations). And the need to
After a light exploratory session, my verdict on Google Scholar Labs (AI-assisted search) is mixed. 🤖
Good:
✅ effective at picking up relevant papers
✅ easy
✅ speedy
✅ free
Bad:
❌ limited to simple chat UI
❌ lack of tables
❌ lack of links for sharing
❌ opaque process
tinyurl.com/3vje892n
Good:
✅ effective at picking up relevant papers
✅ easy
✅ speedy
✅ free
Bad:
❌ limited to simple chat UI
❌ lack of tables
❌ lack of links for sharing
❌ opaque process
tinyurl.com/3vje892n
November 21, 2025 at 2:41 AM
After a light exploratory session, my verdict on Google Scholar Labs (AI-assisted search) is mixed. 🤖
Good:
✅ effective at picking up relevant papers
✅ easy
✅ speedy
✅ free
Bad:
❌ limited to simple chat UI
❌ lack of tables
❌ lack of links for sharing
❌ opaque process
tinyurl.com/3vje892n
Good:
✅ effective at picking up relevant papers
✅ easy
✅ speedy
✅ free
Bad:
❌ limited to simple chat UI
❌ lack of tables
❌ lack of links for sharing
❌ opaque process
tinyurl.com/3vje892n
Reposted by Jay Patel
🌍 Have you experienced language barriers at academic conferences, or not at all?
📝 Take our short survey and help make conferences more inclusive and multilingual:
forms.gle/u8ESktcJP8Y6...
#DEI #JEDI #EDI #Academia #AcademicChatter
📝 Take our short survey and help make conferences more inclusive and multilingual:
forms.gle/u8ESktcJP8Y6...
#DEI #JEDI #EDI #Academia #AcademicChatter
November 19, 2025 at 2:48 PM
🌍 Have you experienced language barriers at academic conferences, or not at all?
📝 Take our short survey and help make conferences more inclusive and multilingual:
forms.gle/u8ESktcJP8Y6...
#DEI #JEDI #EDI #Academia #AcademicChatter
📝 Take our short survey and help make conferences more inclusive and multilingual:
forms.gle/u8ESktcJP8Y6...
#DEI #JEDI #EDI #Academia #AcademicChatter
I kinda like this. It seems like the appropriate level of irony to help us push for #researchreform.
If this isn't the most 2025 headline that had nothing to do with McFascist I've seen all day, I don't know what is.
Research integrity conference hit with AI-generated abstracts
The first of three themes for next year’s World Conference on Research Integrity will be the risks and benefits of artificial intelligence for research integrity. In an ironic and possibly predicta…
retractionwatch.com
November 18, 2025 at 7:55 PM
I kinda like this. It seems like the appropriate level of irony to help us push for #researchreform.
📋 Poll: How many times have you used your university's free statistical consultant when your study could have done so?
#statistics #reproducibility #openscience
#statistics #reproducibility #openscience
November 17, 2025 at 11:04 PM
📋 Poll: How many times have you used your university's free statistical consultant when your study could have done so?
#statistics #reproducibility #openscience
#statistics #reproducibility #openscience
Reposted by Jay Patel
Defend central bank independence: reinstate Lisa Cook
Defend academic integrity: retract Cook (2014)
Defend academic integrity: retract Cook (2014)
Let's look at one problem in Cook's paper; it's easy to see that the results are noise.
Cook finds a negative effect of racial violence on patenting, using annual data. But when we use the state-year data to construct an annual dataset, the results disappear.
1/
Cook finds a negative effect of racial violence on patenting, using annual data. But when we use the state-year data to construct an annual dataset, the results disappear.
1/
Should Lisa Cook resign from the Federal Reserve? No.
Should her most famous paper be retracted? Yes.
Should her most famous paper be retracted? Yes.
August 26, 2025 at 12:51 AM
Defend central bank independence: reinstate Lisa Cook
Defend academic integrity: retract Cook (2014)
Defend academic integrity: retract Cook (2014)
Reposted by Jay Patel
I am hoping to recruit a graduate student for next year. That person would help conduct research on leadership, individual differences, and methodological skullduggery. Please forward to any potentially interested students.
November 17, 2025 at 6:13 PM
I am hoping to recruit a graduate student for next year. That person would help conduct research on leadership, individual differences, and methodological skullduggery. Please forward to any potentially interested students.
About a year after academics migrated from Twitter to Bluesky, still no pubpeerbot? Or similar bots working on adding context to research discussions?
I'm tempted to build something in this area. Would that be worth doing?
I'm tempted to build something in this area. Would that be worth doing?
November 14, 2025 at 10:31 PM
About a year after academics migrated from Twitter to Bluesky, still no pubpeerbot? Or similar bots working on adding context to research discussions?
I'm tempted to build something in this area. Would that be worth doing?
I'm tempted to build something in this area. Would that be worth doing?
Reposted by Jay Patel
Our paper on improving statistical reporting in psychology is now online 🎉
As a part of this paper, we also created the Transparent Statistical Reporting in Psychology checklist, which researchers can use to improve their statistical reporting practices
www.nature.com/articles/s44...
As a part of this paper, we also created the Transparent Statistical Reporting in Psychology checklist, which researchers can use to improve their statistical reporting practices
www.nature.com/articles/s44...
November 14, 2025 at 8:43 PM
Our paper on improving statistical reporting in psychology is now online 🎉
As a part of this paper, we also created the Transparent Statistical Reporting in Psychology checklist, which researchers can use to improve their statistical reporting practices
www.nature.com/articles/s44...
As a part of this paper, we also created the Transparent Statistical Reporting in Psychology checklist, which researchers can use to improve their statistical reporting practices
www.nature.com/articles/s44...
Reposted by Jay Patel
You know what they say about academics...
"causal" in the streets
"preliminary and hypothesis-generating" in the sheets
"causal" in the streets
"preliminary and hypothesis-generating" in the sheets
November 12, 2025 at 11:16 PM
You know what they say about academics...
"causal" in the streets
"preliminary and hypothesis-generating" in the sheets
"causal" in the streets
"preliminary and hypothesis-generating" in the sheets
New study idea:
Examine the mental health impacts of reading RetractionWatch posts every day.
Methods: qualitative, of course
retractionwatch.com/2025/11/12/b...
Examine the mental health impacts of reading RetractionWatch posts every day.
Methods: qualitative, of course
retractionwatch.com/2025/11/12/b...
BMJ places expression of concern on heavily criticized stem cell paper
The BMJ has issued an expression of concern for a paper claiming stem cell therapy can reduce the risk of heart failure. The move comes after sleuths and scientists critiqued the “complete mismatch…
retractionwatch.com
November 13, 2025 at 1:24 AM
New study idea:
Examine the mental health impacts of reading RetractionWatch posts every day.
Methods: qualitative, of course
retractionwatch.com/2025/11/12/b...
Examine the mental health impacts of reading RetractionWatch posts every day.
Methods: qualitative, of course
retractionwatch.com/2025/11/12/b...
Reposted by Jay Patel
This is one of the most remarkable academic debacles I've ever seen.
A large RCT got published in BMJ. There are currently 44 Pubpeer comments, mostly about the data, including...well. Read for yourself.
pubpeer.com/publications...
A large RCT got published in BMJ. There are currently 44 Pubpeer comments, mostly about the data, including...well. Read for yourself.
pubpeer.com/publications...
PubPeer - Prevention of acute myocardial infarction induced heart fail...
There are comments on PubPeer for publication: Prevention of acute myocardial infarction induced heart failure by intracoronary infusion of mesenchymal stem cells: phase 3 randomised clinical trial (P...
pubpeer.com
November 11, 2025 at 1:19 AM
This is one of the most remarkable academic debacles I've ever seen.
A large RCT got published in BMJ. There are currently 44 Pubpeer comments, mostly about the data, including...well. Read for yourself.
pubpeer.com/publications...
A large RCT got published in BMJ. There are currently 44 Pubpeer comments, mostly about the data, including...well. Read for yourself.
pubpeer.com/publications...
Hands down, the best PubPeer comment I've read in a while.
November 10, 2025 at 11:12 PM
Hands down, the best PubPeer comment I've read in a while.
I've never seen a qualitative research paper with excellent figures (relevant, eye-catching, etc.).
They only rely on text and photos/screenshots to communicate.
#dataviz + #qualitative research seem opposed, no?
#scicomm #AcademicWriting
They only rely on text and photos/screenshots to communicate.
#dataviz + #qualitative research seem opposed, no?
#scicomm #AcademicWriting
November 10, 2025 at 11:07 PM
I've never seen a qualitative research paper with excellent figures (relevant, eye-catching, etc.).
They only rely on text and photos/screenshots to communicate.
#dataviz + #qualitative research seem opposed, no?
#scicomm #AcademicWriting
They only rely on text and photos/screenshots to communicate.
#dataviz + #qualitative research seem opposed, no?
#scicomm #AcademicWriting
If synesthesia can help perceptual discrimination, I wonder if there are any synesthetic sleuths doing #ImageFraud checks.
#SciComment
matthewjamestaylor.com/synesthesia-...
#SciComment
matthewjamestaylor.com/synesthesia-...
Synesthesia Helps Me Find Four Leaf Clovers
I explain how my synesthesia condition helps me to find four-leaf clovers
matthewjamestaylor.com
November 10, 2025 at 10:42 PM
If synesthesia can help perceptual discrimination, I wonder if there are any synesthetic sleuths doing #ImageFraud checks.
#SciComment
matthewjamestaylor.com/synesthesia-...
#SciComment
matthewjamestaylor.com/synesthesia-...
Reposted by Jay Patel
Have increased capacity for this December INSPECT-SR online training workshop following a successful 1st event today. Book here: www.trybooking.com/uk/FKHV
Introduction to INSPECT-SR Training Workshop December
An introductory 2-hour online workshop will introduce participants to the INSPECT-SR tool for assessing trustworthiness of randomised controlled...
www.trybooking.com
November 6, 2025 at 5:55 PM
Have increased capacity for this December INSPECT-SR online training workshop following a successful 1st event today. Book here: www.trybooking.com/uk/FKHV
While reading a data science conference paper today, I noticed an interesting transparency statement.
Is this common to data mining conferences?
I like this sort of statement; it reminds me a bit of the 21-word solution. #metascience
Is this common to data mining conferences?
I like this sort of statement; it reminds me a bit of the 21-word solution. #metascience
November 7, 2025 at 2:47 AM
While reading a data science conference paper today, I noticed an interesting transparency statement.
Is this common to data mining conferences?
I like this sort of statement; it reminds me a bit of the 21-word solution. #metascience
Is this common to data mining conferences?
I like this sort of statement; it reminds me a bit of the 21-word solution. #metascience