Lightnews — Scholar-powered news

Johan Edstedt

@parskatt.bsky.social

1.6K followers 400 following 1.8K posts

PhD student @ Linköping University I like 3D vision and training neural networks. Code: https://github.com/parskatt Weights: https://github.com/Parskatt/storage/releases/tag/roma

github.com

Posts Media Videos Starter Packs

Johan Edstedt @parskatt.bsky.social · 2d

1 28

Johan Edstedt @parskatt.bsky.social · 2d

Another way to view interpolation is that your new datapoints should lie in the convex hull of your training data (typically not true for LLMs).

Johan Edstedt @parskatt.bsky.social · 2d

Yes, this seems like a more accurate description.

The distinction between interpolation and extrapolation or compression is a bit diffuse.

For me interpolation is connected with linear combinations of samples, which I don't think is a very accurate description for LLMs.

Johan Edstedt @parskatt.bsky.social · 2d

Yes, most of the time (unless RL).

Johan Edstedt @parskatt.bsky.social · 2d

Worth going!

Giorgos Tolias @gtolias.bsky.social · 2d

The Visual Recognition Group at CTU in Prague organizes the 50th Pattern Recognition and Computer Vision Colloquium with
Torsten Sattler, Paul-Edouard Sarlin, Vicky Kalogeiton, Spyros Gidaris, Anna Kukleva, and Lukas Neumann.
On Thursday Oct 9, 11:00-17:00.

cmp.felk.cvut.cz/colloquium/

Johan Edstedt @parskatt.bsky.social · 2d

I believe extrapolation (or possibly compression) is in general a much more suitable term than interpolation.

Johan Edstedt @parskatt.bsky.social · 2d

The basic way to train a language model is next token (word) prediction.

This is a model for how language is generated (hence LM).

2 1

Johan Edstedt @parskatt.bsky.social · 3d

Hinton, the random plumber of AI

shawnburks.bsky.social @shawnburks.bsky.social · 4d

Why are you citing someone from a completely unrelated field as a source. You might as well source some random plumber.

Johan Edstedt @parskatt.bsky.social · 3d

LLMs are not autoencoders. Autoencoding implies output and input being the same.
New samples are not drawn by interpolation.

Johan Edstedt @parskatt.bsky.social · 3d

From reading the replies here I now understand this

⚡️🌙 @dystopiabreaker.xyz · 5d

Johan Edstedt @parskatt.bsky.social · 5d

Are people still saying ai is useless? I'm in an ai useful echochamber so i have no idea what's out there.

2 31

Johan Edstedt @parskatt.bsky.social · 5d

Besides Carta Marina, this place has some other incredible originals (Celsius temperature measurements, Galileos descriptions of sunspots, Newtons Principia Mathematica)

Johan Edstedt @parskatt.bsky.social · 5d

Went to Carolina Revidiva in Uppsala to see Carta Marina (my thesis cover) in person

2 1 25

Johan Edstedt @parskatt.bsky.social · 5d

I'm mostly talking to myself (although I do see this quite often in monodepth papers).

1 2

Johan Edstedt @parskatt.bsky.social · 5d

A good visualization typically gives you a new perspective, and might reveal limitations in your model that numbers don't show.

I would however be very careful with using this for judging whether one model is "better" than another.

Johan Edstedt @parskatt.bsky.social · 5d

I'd like to reiterate this point, but even for outputs with 3 or less channels (xyz, depth, warps, etc).

What you end up selecting for is aesthetic appeal, although that might be good for getting papers accepted...

Johan Edstedt @parskatt.bsky.social · Aug 14

Slightly obvious take:

Good features have more than 3 active PCs per image. You can't really judge whether they're good by visual inspection (unless you have really good hyperspectral vision).

2 3

Johan Edstedt @parskatt.bsky.social · 5d

Seems you can see the impact of covid on the curve, and that we still haven't come back to the earlier trajectory.