Explaining similarity in vision-language encoders with weighted Banzhaf interactions
Check out the paper on arXiv: arxiv.org/abs/2508.05430
Code to be released soon
👆4/4

Moreover, we derive three evaluation metrics to facilitate future work in this direction. 𝐅𝐈𝐱𝐋𝐈𝐏 achieves state-of-the-art faithfulness performance across the popular insertion/deletion and pointing game benchmarks.
👇3/4

We show that explaining vision–language interactions is essential to faithfully interpret models like OpenAI CLIP & Google SigLIP-2. 𝐅𝐈𝐱𝐋𝐈𝐏 is grounded in cooperative game theory, where we analyze its intriguing properties compared to prior art like Shapley values.
👇2/4

𝗖𝗧𝗘 is a simple yet powerful plug-in for any explainable AI (xAI) method that now relies on i.i.d. sampling. Check out the examples on GitHub!

Code: github.com/hbaniecki/co...

5/5 @xai-research.bsky.social

𝗖𝗧𝗘 improves the accuracy and stability of explanation estimation with negligible computational overhead, often achieving an on-par error using 2–3× fewer samples, i.e. requiring 2–3× fewer model inferences (⌛ = 💰).

👇4/5

𝗖𝗧𝗘 results in more accurate explanations of smaller variance as benchmarked with 4 popular methods (SHAP, SAGE, PDP, Expected Gradients) across 50 datasets and 2 model classes.

👇3/5

𝗖𝗼𝗺𝗽𝗿𝗲𝘀𝘀 𝘁𝗵𝗲𝗻 𝗲𝘅𝗽𝗹𝗮𝗶𝗻! We discover a connection between explanation estimation and distribution compression that significantly improves the approximation of feature attributions, importance, and effects.

Paper: arxiv.org/abs/2406.18334

👇2/5