Lightnews — Scholar-powered news

Arna Ghosh @arnaghosh.bsky.social · 19d

Very cool study, with interesting insights about theta sequences and learning!

Abhilasha Joshi, PhD @rhythmicspikes.bsky.social · 21d

1/
🚨 New preprint! 🚨

Excited and proud (& a little nervous 😅) to share our latest work on the importance of #theta-timescale spiking during #locomotion in #learning. If you care about how organisms learn, buckle up. 🧵👇

📄 www.biorxiv.org/content/10.1...
💻 code + data 🔗 below 🤩

#neuroskyence

1 1

Reposted by Arna Ghosh

Blake Richards @tyrellturing.bsky.social · Jul 9

Together with @repromancer.bsky.social, I have been musing for a while that the exponentiated gradient algorithm we've advocated for comp neuro would work well with low-precision ANNs.

This group got it working!

arxiv.org/abs/2506.17768

May be a great way to reduce AI energy use!!!

#MLSky 🧪

Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networks

Studies in neuroscience have shown that biological synapses follow a log-normal distribution whose transitioning can be explained by noisy multiplicative dynamics. Biological networks can function sta...

arxiv.org

3 13 39

Arna Ghosh @arnaghosh.bsky.social · Jun 24

Congratulations, Dan!! 😁

1

Arna Ghosh @arnaghosh.bsky.social · Jun 15

This looks like a very cool result! 😀
Can't wait to read in detail.

Samuel Liebana @samuel-liebana.bsky.social · Jun 15

Does the brain learn by gradient descent?

It's a pleasure to share our paper at @cp-cell.bsky.social, showing how mice learning over long timescales display key hallmarks of gradient descent (GD).

The culmination of my PhD supervised by @laklab.bsky.social, @saxelab.bsky.social and Rafal Bogacz!

Dopamine encodes deep network teaching signals for individual learning trajectories

Longitudinal tracking of long-term learning behavior and striatal dopamine reveals that dopamine teaching signals shape individually diverse yet systematic learning trajectories, captured mathematical...

www.cell.com

3

Arna Ghosh @arnaghosh.bsky.social · Jun 9

Fantastic work on Multi-agent RL from
@dvnxmvlhdf5.bsky.social & @tyrellturing.bsky.social! 🤩

Dane Carnegie Malenfant @dvnxmvlhdf5.bsky.social · Jun 5

Preprint Alert 🚀

Multi-agent reinforcement learning (MARL) often assumes that agents know when other agents cooperate with them. But for humans, this isn’t always the case. For example, plains indigenous groups used to leave resources for others to use at effigies called Manitokan.
1/8

Manitokan are images set up where one can bring a gift or receive a gift. 1930s Rocky Boy Reservation, Montana, Montana State University photograph. Colourized with AI

2 8

Reposted by Arna Ghosh

Yiğit Demirağ @yigit.ai · May 8

Our team is hiring in Zurich!

www.google.com/about/career...

Research Scientist, Paradigms of Intelligence — Google Careers

www.google.com

4 16

Arna Ghosh @arnaghosh.bsky.social · Apr 2

Re diff implicit biases of architecture: the metrics implemented here (roughly) characterize the eigenspectrum (eigenval distribution) of the representation space. They don't really incorporate the eigenvector information --> hence, "what" features don't matter, only "how" matters.

Arna Ghosh @arnaghosh.bsky.social · Apr 2

Comparing models of different architectures often is tricky because of the different implicit biases of each architecture.
RSA can be helpful to in some cases.
But if you are looking for a metric (number) that tells you which model is better, I have some ideas but they are not implemented here. 😜

1

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Indeed!
The metrics work best when comparing networks of comparable architectures, though. So, if you are looking to select the best model checkpoint from a pretraining routine or diff hyperparam configurations, and the loss function is not as insightful, these metrics are incredibly helpful. :)

1 1

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Also, big shoutout to @quentin-garrido.bsky.social+gang and
@aggieinca.bsky.social+gang for developing Rankme and Lidar, respectively.
Reptrix incorporates these representation quality metrics. 🚀
Let's make it easier to select good SSL/foundation models. 💪

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Are you training self-supervised/foundation models, and worried if they are learning good representations? We got you covered! 💪
🦖Introducing Reptrix, a #Python library to evaluate representation quality metrics for neural nets: github.com/BARL-SSL/rep...
🧵👇[1/6]
#DeepLearning

2

Arna Ghosh @arnaghosh.bsky.social · Apr 1

PS: It was fun to put Reptrix together with @daniebenes.bsky.social, our open-source expert and new addition to the α-Squad comprising of Arnab Mondal, @kumarkagrawal.bsky.social @tyrellturing.bsky.social and I.
[6/6]

3

Arna Ghosh @arnaghosh.bsky.social · Apr 1

If you do end up giving Reptrix a try, we would love to hear from you! We also encourage you to contribute to our repo and add example notebooks trying out these metrics for your networks, beyond natural language and vision domains.
github.com/BARL-SSL/rep...
[5/6]

reptrix/CONTRIBUTING.md at main · BARL-SSL/reptrix

Library that provides metrics to assess representation quality - BARL-SSL/reptrix

github.com

1 1

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Metrics for evaluating representation quality:
- α-ReQ: Measures discriminativeness. Lower alpha = better!
- RankMe: Assesses representation capacity. Higher rank = higher capacity!
- LiDAR: Evaluates separability among object manifolds. Higher rank = better separability!
[4/6]

1 2

Arna Ghosh @arnaghosh.bsky.social · Apr 1

✨ Key Features of Reptrix:
- 📈 Suite of metrics to assess representation quality: α-ReQ, RankMe, LiDAR, and more!
- 🤝 Seamless PyTorch integration for minimal setup, maximum insights
- 💻 Open Source: Contribute and enhance!
[3/6]

1 1

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Inspired by conversations after our α-ReQ paper (NeurIPS 2022) and subsequent work, we created Reptrix as an open-source library for assessing representation quality across models of vision, language… and more.
Check out our @mila-quebec.bsky.social blogpost: mila.quebec/en/article/a...
[2/6]

α-ReQ: Assessing Representation Quality in SSL | Mila

The success of self-supervised learning algorithms has drastically changed the landscape of training deep neural networks. With well-engineered architectures and training objectives, SSL models learn ...

mila.quebec

1 2

Arna Ghosh @arnaghosh.bsky.social · Apr 1

Are you training self-supervised/foundation models, and worried if they are learning good representations? We got you covered! 💪
🦖Introducing Reptrix, a #Python library to evaluate representation quality metrics for neural nets: github.com/BARL-SSL/rep...
🧵👇[1/6]
#DeepLearning

3 9 27

Arna Ghosh @arnaghosh.bsky.social · Mar 8

Super cool paper!
It formalizes a lot of ideas I have been mulling over the past year, and connects tons of historical ideas neatly.
Definitely worth a read if you are working/interested in mechanistic interp and neural representations.

David Klindt @david-klindt.bsky.social · Mar 4

🔵 New paper! We explore sparse coding, superposition, and the Linear Representation Hypothesis (LRH) through identifiability theory, compressed sensing, and interpretability. If you’re curious about lifting neural reps out of superposition, this might interest you! 🤓
arxiv.org/abs/2503.01824

From superposition to sparse codes: interpretable representations in neural networks

Understanding how information is represented in neural networks is a fundamental challenge in both neuroscience and artificial intelligence. Despite their nonlinear architectures, recent evidence sugg...

arxiv.org

2

Arna Ghosh @arnaghosh.bsky.social · Feb 18

I do not have words that can capture how grateful I am to you for your support throughout this journey. 🥹

I joined the lab as a fan of our work (apical dendrites FTW 😉), 5.5 years later I am a fan of you! ⭐

1 2

Arna Ghosh @arnaghosh.bsky.social · Feb 15

Thank you so much Rui! :)
I was blessed with great examiners, who made my job easier. 😅

1

Arna Ghosh @arnaghosh.bsky.social · Feb 15

Thanks Koustuv da for your guidance and wise words throughout! ❤️

1

Arna Ghosh @arnaghosh.bsky.social · Feb 15

Thanks Yigit!! ❤️
Looking forward to doing exciting things together. :)

1

Arna Ghosh @arnaghosh.bsky.social · Feb 15

Thanks a lot, Adrien!
You have been an integral part of my grad school journey, thank you so much for your support and encouragement throughout. 🥹

1

Arna Ghosh @arnaghosh.bsky.social · Feb 10

I have been very fortunate to have the support of close friends & family through my academic journey, as well as the support of my stellar defence committee: Aaron Courville, Adriana Romero-Soriano, @somnirons.bsky.social @apeyrache.bsky.social, David Adelani. 🚀

2 3

Arna Ghosh @arnaghosh.bsky.social · Feb 10

Just over a week since I defended my 🤖+🧠PhD thesis, and the feeling is just sinking in. Extremely grateful to
@tyrellturing.bsky.social for supporting me through this amazing journey! 🙏
Big thanks to all members of the LiNC lab, and colleagues at mcgill University and @mila-quebec.bsky.social. ❤️😁

3 3 30

Reposted by Arna Ghosh

Paul Frankland @franklandlab.bsky.social · Jan 23

Just giving this a boost for those who may not have seen it yet... we have a PI position (molecular and cellular basis of cognition) at The Hospital for Sick Children (Toronto). The position comes with an appointment at Assist/Assoc Prof level at U of T. Share widely!
can-acn.org/scientist-se...

Scientist/Senior Scientist – Research Institute, Hospital for Sick Children, University of Toronto – Canadian Association for Neuroscience

can-acn.org

1 110 150