I'm very happy to present our work "Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?" this afternoon at #ICLR2025! Come have a chat at stand #439 :)

What can be done?

👉 Stricter validity criteria?
👉 Maybe interpretability is inherently underdetermined? and we can only get control and predictability but not "understanding"

This is a fascinating topic, and we keep investigating. If you're interested, come and chat at ICLR!

We find a lot of identifiability issues:
- Multiple explanatory algorithms exists
- Even for one algorithm, there are many localizations in the network

Identifiability problems remain across scenarios: changing levels of over-parametrization, progress in training, multi-tasks, model size.

In our work, we stress-test the identifiability of research programs of MI with small MLPs and simple boolean logic tasks.
Why? It allows us to enumerate all possible explanations and see how many pass various MI testing criteria.

This brings us to identifiability. In statistics a property is identifiable if a unique value is compatible with the data. Identifiability matters because it is a prerequisite for doing statistical and causal inference.

Interpretability is also an exercise in causal inference!

Mechanistic Interpretability aims to produce statements like: "Model M solves task T by doing X."
To do so, many causal manipulations are performed to validate an explanation. But what if (many) other, incompatible explanations also pass the causal tests?

Our paper "Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?" will be presented at #ICLR2025!
It's also the first paper of my first PhD student, congrats @maximemeloux.bsky.social ! 🎉

blog: melouxm.github.io/MI-identifia...

An explanatory thread 🧵:

An assembly of 18 European companies, labs, and universities have banded together to launch 🇪🇺 EuroBERT!

It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc.

Details in 🧵

bc i haven't done so yet, i decided to burn any remaining bridge to the land of statistics. it wasn't statisticians nor statistics but it was me. i am simply not good enough to do statistics myself.

so, @peyrardmax.bsky.social and i decided to turn statistical estimation into supervised learning.

Check out our new paper on social determinants of on-campus food choice, now out in @pnasnexus.org!

academic.oup.com/pnasnexus/ar...

Hey, thanks for making it, can you also add me

I tried to find everyone who works in the area but I certainly missed some folks so please lmk...
go.bsky.app/BYkRryU

Thanks for creating the pack, I am also working on this topic :)