Lightnews — Scholar-powered news

Phillip Isola @phillipisola.bsky.social · Aug 6

Interesting reaction from ChatGPT to the HHS mRNA memo. It finds it so implausible that it thinks it's fake. From the perspective of a ~2024(?) trained model, 2025 policies are so absurd as to be unbelievable...

chatgpt.com/share/689364...

3 13 53

Phillip Isola @phillipisola.bsky.social · Jul 31

Unless it turns out it that capable intelligence is actually not so simple!

1

Phillip Isola @phillipisola.bsky.social · Jul 31

Yeah, it helps me to consider that much of the history of science has been about finding a simpler-than-expected explanation of something that previously seemed magical: life (evolution), motion of the planets (law of gravitation), etc. Now those are among our most celebrated discoveries.

1 2

Phillip Isola @phillipisola.bsky.social · Jul 31

Of course, personally, I think we need not shy away from this possibility. Maybe intelligence is simpler than we thought, and there's a beauty in that too.

3 11

Phillip Isola @phillipisola.bsky.social · Jul 31

I think part of it is that people might be overestimating the complexity of intelligence, and it's hard not to.

How weird it would be if an LLM (a Markov chain!) could explain "thinking".

It feels like it makes us less special, like Copernicus placing the sun at the center, rather than the Earth.

3 3 27

Phillip Isola @phillipisola.bsky.social · Jul 27

I enjoy your posts! I hope you keep at it.

6

Phillip Isola @phillipisola.bsky.social · Jul 16

Finite: right, you would need to train the student on inputs beyond the GT x's.

Wrong: the teacher could underfit and be more correct than the "GT" y's. This paper is about one version of this: arxiv.org/abs/2206.15477

Denoised MDPs: Learning World Models Better Than the World Itself

The ability to separate signal from noise, and reason with clean abstractions, is critical to intelligence. With this ability, humans can efficiently perform real world tasks without considering all p...

arxiv.org

1 6

Phillip Isola @phillipisola.bsky.social · Jul 16

One reason is that GT may be finite or, yes, wrong. A regression model fit to GT can potentially generalize beyond the GT and correct errors.

I like to think of this as: the data is a bad model of the world.

1 4

Phillip Isola @phillipisola.bsky.social · Jul 12

To me it’s more like there exist some sci fi that predicted pretty well each of the things we are seeing, although no sci fi got them all right. But that still just seems incredible given that what happened is an infinitesimal point in the space of all possibilities…

1

Phillip Isola @phillipisola.bsky.social · Jul 12

Yeah and relatedly it’s odd to me when people dismiss future predictions as “pure science fiction” as if that means they are wrong or unlikely. The overwhelming feeling I have when reading old sci fi is how accurately it often predicted what came to pass.

2 2

Phillip Isola @phillipisola.bsky.social · Jul 12

I agree, maybe we need to qualify novelty wrt the audience: like novelty(x | me), novelty(x | biologists), novelty(x | 6th graders), novelty(x | world’s top expert), etc. All are useful. Some are more like “research” others are more like “teaching”, all are quite related.

1 4

Phillip Isola @phillipisola.bsky.social · Jul 12

Ah weird, yeah you are probably right for CS but for the natural sciences I think “novelty” often means “novel finding” rather than “novel method”, as in we discovered something new about the world. I like that definition more! Agree that most papers need not have any new algorithm to be worthwhile.

1 4

Phillip Isola @phillipisola.bsky.social · Jul 12

I’m more in the “pro novelty” camp but I think maybe it’s because I see novelty differently. I think, for example, that showing that known method X solves open problem Y is hugely novel. For me novelty is basically: did I learn something new and important from this work.

2 16

Reposted by Phillip Isola

#CVPR2026 @cvprconference.bsky.social · Jun 19

#CVPR2025 provided coaching for all orals. Do you think the talks were improved compared to last year?

* Better than last year
* About the same
* Worse than last year

Share your thoughts in the thread!

3 10 14

Phillip Isola @phillipisola.bsky.social · Jun 15

Our computer vision textbook is now available for free online here:
visionbook.mit.edu

We are working on adding some interactive components like search and (beta) integration with LLMs.

Hope this is useful and feel free to submit Github issues to help us improve the text!

Foundations of Computer Vision

The print version was published by

visionbook.mit.edu

3 31 110

Phillip Isola @phillipisola.bsky.social · May 22

haha yes, "reminds me of Plato’s cave from Philosophy 101" is the closest to what we meant in our PRH paper.

We didn't mean to advocate wholesale, unelaborated platonism :)

2 1 10

Reposted by Phillip Isola

#CVPR2026 @cvprconference.bsky.social · May 10

Behind every great conference is a team of dedicated reviewers. Congratulations to this year’s #CVPR2025 Outstanding Reviewers!

cvpr.thecvf.com/Conferences/...

12 48

Phillip Isola @phillipisola.bsky.social · May 8

interesting, that's not great :) but I'm not sure it would fair to apply this critique to all scaling laws papers

Phillip Isola @phillipisola.bsky.social · May 8

Wait, the plots I've seen are a lot more than 4 points, am I missing something?

1 1

Reposted by Phillip Isola

Greta Tuckute @gretatuckute.bsky.social · Apr 10

PINEAPPLE, LIGHT, HAPPY, AVALANCHE, BURDEN

Some of these words are consistently remembered better than others. Why is that?
In our paper, just published in J. Exp. Psychol., we provide a simple Bayesian account and show that it explains >80% of variance in word memorability: tinyurl.com/yf3md5aj

APA PsycNet

tinyurl.com

1 14 40

Phillip Isola @phillipisola.bsky.social · Apr 1

Chapters 32-34 of our computer vision book cover these and might be useful: visionbook.mit.edu

Slides for these chapters are here: www.dropbox.com/personal/vis...

Login or Sign Up - Dropbox

Login to Dropbox. Bring your photos, docs, and videos anywhere and keep your files safe.

www.dropbox.com

1 9

Phillip Isola @phillipisola.bsky.social · Mar 26

Compositional generalization might just be one piece of the puzzle. Here's a nice paper on this (but for diffusion): arxiv.org/abs/2412.20292.

The point is there are many ways LLMs might generalize far beyond the training data. I think it's happening. But exactly how and to what degree is open.

An analytic theory of creativity in convolutional diffusion models

We obtain the first analytic, interpretable and predictive theory of creativity in convolutional diffusion models. Indeed, score-based diffusion models can generate highly creative images that lie far...

arxiv.org

5

Phillip Isola @phillipisola.bsky.social · Mar 26

For example:

1) LLMs are approximate N-gram models (with large N). N-gram models extrapolate _compositionally_. They stitch together high probability text snippets in endlessly new ways.
2) Many problems we apply LLMs to are truly compositional in the same way.

1 4

Phillip Isola @phillipisola.bsky.social · Mar 26

It all depends on 1) how the model extrapolates vs 2) how the world actually works.

With LLMs this is open science. But we do have some ideas about both 1 and 2.

1

Phillip Isola @phillipisola.bsky.social · Mar 26

Maybe this seems obvious and trivial, but I think it's a useful starting point, and the principle is the same with any ML model.

They all can make predictions outside the support of their training data. The predictions might be right or they might be wrong.

1 2