Lightnews — Scholar-powered news

Reposted by Joe Bak-Coleman

Carl T. Bergstrom @carlbergstrom.com · 11h

Fustiliarian Friday: Trump/Kennedy/GOP use the shutdown as an excuse to destroy the CDC’s elite field epidemiology team and one of the most important epidemiology journals in the world.

Gift link.

Trump Administration Lays Off Dozens of C.D.C. Officials

www.nytimes.com

4 82 150

Joe Bak-Coleman @jbakcoleman.bsky.social · 5h

Science Taylorism strikes again.

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 5h

Yeah, especially given figure 6 uptake is almost certainly driven by “oh shit I need to buy a paper”

Joe Bak-Coleman @jbakcoleman.bsky.social · 6h

I certainly believe folks could rip out more papers but to the extent that science (and any collective professional) thrives on diversity of experience and perspective it’s hard to see how driving us towards the most likely translates to scientific productivity broadly.

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 6h

Yah every inch of the paper is a causal pitfall. Especially when you look at their model ran on unmatched data. Wild.

1 1

Joe Bak-Coleman @jbakcoleman.bsky.social · 6h

If it were helping with translation or whatnot we wouldn’t expect this because the language of the first draft is pinned down by the original text before assistance.

The quality indicator is equally explained by this as well, just a little less intuitively.

Joe Bak-Coleman @jbakcoleman.bsky.social · 6h

Figure 6, the effect sizes dwindle as you move away from the top single digit percentile of most ai heavy users. As we know some papers are *entirely* ai slop, these stricter criteria can be viewed as containing a higher abundance of slop.

In other words, these gains are happening for ai slop.

2 1 4

Joe Bak-Coleman @jbakcoleman.bsky.social · 7h

The gutting of CDC talent happening right now is just fucking scary.

7

Joe Bak-Coleman @jbakcoleman.bsky.social · 7h

I just got hit with a reviewer request for ai slop from a nature journal (and one of the exclusive ones..) so I’d wager a lot of this is just slop proliferating.

1 2

Joe Bak-Coleman @jbakcoleman.bsky.social · 7h

Bespoke: paying a paper mill to write your ChatGPT paper

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 8h

Quality here is measured by impact factor, which…

My guess is the signal being picked up on is ai slop proliferation in the shoddy self-citing corner of scam journals.

1 6

Joe Bak-Coleman @jbakcoleman.bsky.social · 8h

Broke: ai slop infiltrating journals and citing itself
Woke: lots of high impact papers

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 8h

I love the author didn’t even consider paper mills.

1 2

Joe Bak-Coleman @jbakcoleman.bsky.social · 9h

Betteridge’s law of headlines is strong with this one.

arxiv.org/abs/2510.02408

Can GenAI Improve Academic Performance? Evidence from the Social and Behavioral Sciences

This paper estimates the effect of Generative AI (GenAI) adoption on scientific productivity and quality in the social and behavioral sciences. Using matched author-level panel data and a difference-i...

arxiv.org

2 1 10

Joe Bak-Coleman @jbakcoleman.bsky.social · 19h

If you have theory there’s a lot less poking in the dark then it seems. You can intuit the functional forms from first principles and the distributions to be expected. A bit of algebra and statistical theory goes a long way towards a better model.

Joe Bak-Coleman @jbakcoleman.bsky.social · 19h

Something we don’t discuss enough are the trade offs between getting it wrong because the model/theory is shit and getting it wrong because of wishful modification to the model or data. The modal errors I see in papers are the former these days.

1 2

Joe Bak-Coleman @jbakcoleman.bsky.social · 19h

You go back and revise and the model fits like a glove to the bulk properties of the data across groups. Your decision isn’t conditioned on the difference in means, so you’re not coaxing your inferences but you’re working to avoid drawing the wrong inference from bad model fit.

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 19h

As a simple example, say you’re interested in means and originally assume normal distributions for both groups. Your simulations come back fine but then you look at data simulated from the posterior and notice it’s extremely skewed by comparison. You adjust your model to allow for skew.

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 19h

There are workflows and inferential approaches that let you walk and chew gum here. The insight is that you ground model development in theory and simulations first, then features of the data distinct from the inferences you want to draw.

betanalpha.github.io/assets/case_...

Towards A Principled Bayesian Workflow

betanalpha.github.io

1 2

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

bsky.app/profile/kevi...

Kevin Zollman @kevinzollman.com · 1d

It's not that I think RFK is going to do good science on this topic. I think it's important to criticize it for the right reasons.

I think perpetuating the children's version of the scientific method leads to scientific skepticism.

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

There’s a risk of epistemic filtering within science being adopted by the admin that prioritizes some modes of inference over others and we can play into it or not. If all we can do is use preregister RCTs that test nulls (as suggested by the OP) we’re cooked for climate, vaccines.

2 3

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

Tu Quoque aside, it’s worth considering that maybe other areas of science function differently than you’re accustomed to and they’re as threatened by internal statements about how science should be done as the admin, and my points above are in response to that.

1

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

Yah they’re being disingenuous but their arguments that science needs to fit an absurd mold to be valid will survive a Daubert challenge unless we get better with precision than we have been.

1 1

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

I guess all I can say is that they cited this kind of idea heavily in their gold standard science policy as pretext to exert state control… it’s their stated justification and the one they will cite in courts. Either we can clarify in hopes the courts hold or let that fly as admissible evidence.

1 1

Joe Bak-Coleman @jbakcoleman.bsky.social · 1d

Perhaps Kevin said it simplest.

bsky.app/profile/kevi...

Kevin Zollman @kevinzollman.com · 1d

It's not that I think RFK is going to do good science on this topic. I think it's important to criticize it for the right reasons.

I think perpetuating the children's version of the scientific method leads to scientific skepticism.

1