Lightnews — Scholar-powered news

Jay Patel

@infotainment.bsky.social

2.4K followers 4.1K following 2.4K posts

🎷 vibe adulting #HCI #PeerReview #SciPub #toolsforthought #ResearchSynthesis #OpenScience #MetaSci #FoSci 🔎 Research: ethnography of open peer review 🧑‍🏫 Teaching: Stats, DataViz 🐢 UMD: College of Info 🌐 PhD Candidate: Info Studies / HCI + Data 🏝️ OASISlab

Posts Media Videos Starter Packs

Pinned

Jay Patel @infotainment.bsky.social · Jun 11

🌐 It strikes me that crowdsourced evaluation/benchmarking of LLMs for scientific work would be valuable and possible in the style of ManyLabs projects.

If I coordinate crowds of volunteers to assess how well LLMs can evaluate papers, would you be interested?

#openscience #metascience #AI #LLM

Reposted by Jay Patel

Ben Schmidt @bschmidt.bsky.social · 10d

Despite the gutting of the National Center for Educational Statistics, the dept of Ed *did* manage to release 2024 college major counts in the usual format, so I can run it through the same code I do every year. First off, the change since peak of the largest fields -- another year of drops.

A line chart captioned "The big humanities majors were mostly still falling in 2024", showing drops since 2008 for most humanities fields between 10% (Study of the Arts) to 68% (religion) with history, english, and foreign languages all clustered around 50-55%

Jay Patel @infotainment.bsky.social · 4d

Link to paper?

Jay Patel @infotainment.bsky.social · 4d

Not fully, AI for diagnosis and drug discovery has already shown benefits. Of course, it's still important to steer such AI towards desired ends.

Jay Patel @infotainment.bsky.social · 4d

Which ones? How was it worded?

Reposted by Jay Patel

derek guy @dieworkwear.bsky.social · 5d

Matthew 15:29-31: And great crowds came to him, bringing with them the lame, the blind, the crippled, the mute, and many others, and they put them at his feet, and he healed them after verifying their citizenship status.

Jay Patel @infotainment.bsky.social · 6d

I think the term is part of the issue. I prefer to think of it as an attempt to give authors some feedback or where they're going. Unfortunately, few platforms put us in direct conversation with authors (PubPeer only alerts authors via email). alphaXiv gets closer to a cordial commenter-author convo

Jay Patel @infotainment.bsky.social · 6d

I'd also add that if a platform has rules for authoring comments, then its technology should support alignment with said rules (many UI options could be tested here). @pubpeer.com This would be a good way to use the funding you have.

Jay Patel @infotainment.bsky.social · 6d

Nice, I didn't catch that in my reading. It appears 3 times in Related Work (Lit Review). Seems likely they automated the boring part, but it requires further investigation. I'll delve into it.

Jay Patel @infotainment.bsky.social · 6d

Some comments in my review were generic, but a read of the paper would support writing them. I wonder if these review platforms even skim the papers and preprints. The context matters for moderating reviews. It was an unclear paper for me.

Jay Patel @infotainment.bsky.social · 6d

They're more meaningful when you make them. People are not trained in making clear and enlightening visuals.

Jay Patel @infotainment.bsky.social · 6d

which form of stubborness is desired? i'm willing for a remote role.

Jay Patel @infotainment.bsky.social · 6d

Yeah, I just copy-paste on both and it only takes a minute or so. Not too tough.

I also appreciate newer systems with modern look and feel and the ability to help guide new features. Reviewers can and should shape the platform where they work.

Jay Patel @infotainment.bsky.social · 6d

Yes! Left is a blank PubPeer page where I submitted a comment (awaiting moderation, then never accepted). Right is Paperstars with the same comment.

pubpeer page with a blank space where my post should be

paperstars review of a paper on assessing novelty with LLMs, review is posted in full

Jay Patel @infotainment.bsky.social · 7d

One reason I like to post reviews of research papers on @paperstarsorg.bsky.social instead of @pubpeer.com: they actually pass moderation.

The same neutral, substantive comment gets very different results.

Jay Patel @infotainment.bsky.social · 11d

This is big! I'd also like to see a push to move to journal-independent peer review.

Jay Patel @infotainment.bsky.social · 11d

Pittsburgh?

Jay Patel @infotainment.bsky.social · 11d

Care to share the paper and eventual pppr? I think posting to Bluesky or LinkedIn then using scholars' reactions to move it to PubPeer has worked out well for me recently.

Then after allowing authors to respond, other pathways open up.

Jay Patel @infotainment.bsky.social · 11d

Hi, I'm the student.

Jay Patel @infotainment.bsky.social · 11d

And initiatives like Peer Community In and MetaROR reinforce journals' power by provided journals with peer reviewed preprints free of charge.

Jay Patel @infotainment.bsky.social · 12d

Loving "the plot thickens" for the button text.

Reposted by Jay Patel

Joachim Goedhart @joachimgoedhart.bsky.social · 12d

What’s the best platform for post publication peer review? @pubpeer.com ?

Jay Patel @infotainment.bsky.social · 12d

Currently, the UX and UI on @alphaxiv.org seem best. The experience is fluid and modern.

The dominant player is certainly PubPeer, but more consistent compensation on @researchhubf.bsky.social .

@paperstarsorg.bsky.social is a rising... star... mind the pun.

I'd love even more entrants.

Jay Patel @infotainment.bsky.social · 13d

#StatSky Curious to know where y'll go to creating interactive lessons for undergrads. For t-tests, z-tests, and other commonly used sig tests, what are some good #teaching resources?

I have a few ideas, but am curious where the wealth of wisdom resides.

Books, sites?

Reposted by Jay Patel

Susana Wadgymar🌿🔬 @smwadgymar.bsky.social · Oct 22

If you need a GPT-proof assignment... go for a concept map.

Below I am showing the terms that need to be included, the concept map that we made in class, and two different concept maps that ChatGPT suggested.

HOPODSIS TESTING! 🤡

#AcademicSky

A list of terms, including standard error, central limit theorem, two-tailed test, confidence interval, p-value, test statistic, degrees of freedom, sample size, margin of error, z-score, t distribution, standard deviation, alpha, point estimate, alternative hypothesis, null hypothesis, standard error, sampling distribution, conditional probability, type II error, type I error, t-test.

A concept map linking terms in a logical way.

A concept map including words that aren't real words. For example, hypothesis testing became 'hopodsis testing'.

A concept map including words that aren't real words or are blurred to now show real words.

Jay Patel @infotainment.bsky.social · 13d

Yes, this student is from one of those countries. I might just begin with a bit of a chat about incentives and why they exist.