Lightnews — Scholar-powered news

Damien Teney @damienteney.bsky.social · Jul 26

Academic Strava?🤓 It feels like an underrepresented group in my Strava feed!

Damien Teney @damienteney.bsky.social · Jul 24

It'd be nice to provide complete analyses (that you have precomputed) of existing papers, so we can see what kind of output the tool provides, without having to submit any of my own work.

1 23

Damien Teney @damienteney.bsky.social · Jul 22

Dang it just never ends 😱

Damien Teney @damienteney.bsky.social · Jul 16

In this setting, does the student (sometimes?) get better than the teacher? One hypothesis could be that the teacher, even if "less correct" than the GT, provides supervision that's easier to learn for another NN (the student). The optimization follows a less tortuous path & finds a better solution.

1 3

Damien Teney @damienteney.bsky.social · Jul 7

👍 I had my very first paper published at DAGM. It was a while ago but I remember it as a very welcoming conference.

1

Damien Teney @damienteney.bsky.social · Jul 7

🎯There's already a plethora of methods to handle distribution shifts: most gains may now simply be in better using them! Automatic selection looks promising, yet there's lots more to do. Interested? Come chat with us at ICML!
📄 arxiv.org/abs/2410.02735
💻 github.com/LiangzeJiang...

OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?

Out-of-distribution (OOD) generalization is challenging because distribution shifts come in many forms. Numerous algorithms exist to address specific settings, but choosing the right training algorith...

arxiv.org

Damien Teney @damienteney.bsky.social · Jul 7

🔎Last but not least: OOD-Chameleon shines new light on existing algorithms! We can interpret the selection process as a tree and get interpretable guidelines for choosing algorithms.

1

Damien Teney @damienteney.bsky.social · Jul 7

✅We test OOD-Chameleon on unseen datasets & shifts: it accurately predict suitable algorithms on synthetic, vision, and language tasks. The downstream models (trained with selected algorithms) consistently have lower error than with standard selection heuristics.

1 1

Damien Teney @damienteney.bsky.social · Jul 7

🛠️To create our "dataset of datasets", we resample CelebA and CivilComments with constraints specifying diverse types/magnitudes of shifts. We also train small models with candidate algorithms to obtain their "ground truth performance" in each condition.

1

Damien Teney @damienteney.bsky.social · Jul 7

🦎We propose OOD-Chameleon as a proof-of-concept: an algorithm selector as a classifier (over candidate algorithms) trained on a "dataset of datasets" representing diverse shifts. The model learns which algorithms perform best in different conditions.

1

Damien Teney @damienteney.bsky.social · Jul 7

💡We're aiming for an "auto-ML for distribution shifts". We conjecture that datasets have properties predictive of the suitability of various algorithms to handle dist. shifts: size/complexity of the data, magnitudes/types of shifts, etc.

1

Damien Teney @damienteney.bsky.social · Jul 7

"Distribution shift" means many things: spurious correlations, covariate shift, label shift... with no one-size-fits-all! Many algorithms exist, 📊each for specific conditions. Could we automate the selection without trial-and-error❓

1

Damien Teney @damienteney.bsky.social · Jul 7

Coming up at ICML: 🤯Distribution shifts are still a huge challenge in ML. There's already a ton of algorithms to address specific conditions. So what if the challenge was just selecting the right algorithm for the right conditions?🤔🧵

1 1 4

Damien Teney @damienteney.bsky.social · Jun 30

Nice design!

Damien Teney @damienteney.bsky.social · Jun 30

On the contrary it's a discussion that needs bringing up inside the CV research community. They're not just a bunch of white dudes with evil intentions or industry pressure to build a surveillance state. As one example see the thoughts from one such researcher lucasb.eyer.be/snips/cv-eth...

Ethical considerations around Vision and Robotics

A rough outline on how I think about doing research in Computer Vision given the many possible unethical uses.

lucasb.eyer.be

1 1

Damien Teney @damienteney.bsky.social · Jun 30

Brilliant! Some pushback I heard against mandatory reviewing was from people misunderstanding that a submission entails such a partnership with the rest of the community.

1

Damien Teney @damienteney.bsky.social · Jun 30

Looks quiet! Best time of day 👌🏼

1

Damien Teney @damienteney.bsky.social · Jun 30

Good points. It's a difficult task to automate the classification of papers/patents. Would the tracking of hands/gestures for sign language interfaces count as surveillance here?

1 1