LLMs have shown impressive performance in some reasoning tasks, but what internal mechanisms do they use to solve these tasks? In a new preprint, we find evidence that abstract reasoning in LLMs depends on an emergent form of symbol processing arxiv.org/abs/2502.20332 (1/N)

Every experience is unique 🌟 light shifts, angles change, yet we recognize objects effortlessly. How do our minds do this? And (how) do they differ from machines? In our new preprint with @ericschulz.bsky.social, we review human generalization and compare it to machine generalization: osf.io/k6ect

About a month late posting this, but here's a new project with @ericschulz.bsky.social, @akjagadish.bsky.social, @marvinmathony.bsky.social and Tobias Ludwig

We are using LLMs to propose cognitive models in learning and decision making data. Presenting this work at RLDM!

arxiv.org/abs/2502.00879

In previous work we found that VLMs fall short of human visual cognition. To make them better, we fine-tuned them on visual cognition tasks. We find that while this improves performance on the fine-tuning task, it does not lead to models that generalize to other related tasks:

We are currently building the largest, cross-domain data set of human behavior as part of an open collaborative project. Contributions of any form are welcome, but especially experiments with meta-data from developmental, cross-cultural, or clinical studies.

More details: github.com/marcelbinz/P...

In the first paper, @candemircan.bsky.social and @tankred-saanum.bsky.social use sparse autoencoders to show that LLMs can implement temporal difference learning in context. This work is together with Akshay Jagadish and @marcelbinz.bsky.social.
arxiv.org/abs/2410.01280

Many thanks to @tankred-saanum.bsky.social @marcelbinz.bsky.social @mgarvert.bsky.social @ericschulz.bsky.social and Doeller lab

Check the paper here: arxiv.org/abs/2306.09377

Reach out if you're at NeurIPS and wanna talk about representational alignment, mechanistic interpretability, or CogSci!

Lastly, we found that previously established alignment methods do not consistently yield better results compared to non-aligned baselines.

Several other factors were important for alignment, such as model size, how separated class representations were, and intrinsic dimensionality.

We found that this cannot be fully attributed to pretraining data size in additional analyses.

CLIP-style models predicted human choices the best across the tasks, suggesting multimodal pretraining is important for representational alignment.

We tested humans on reward and category learning tasks using naturalistic images, where the underlying functions were generated using the THINGS embedding.

Alignment is more than comparing similarity judgments! How well do pretrained neural networks align with humans in few-shot learning settings? Come check our poster #3904 at #NeurIPS on Wednesday to find out

tried messaging you, but the app says you cannot be messaged

🚨Join our team! We’re hiring a PhD student in Cognitive & Clinical Neuroscience 🧠 🎓 at @uni_wue & @UKW_Wuerzburg! Explore mechanisms of decision-making in healthy people & Parkinson’s using new deep brain stimulation methods. German & English required. Apply by 20 Dec! 🌟🎄
Details: shorturl.at/IcNa0