Lightnews — Scholar-powered news

Robert Rosenbaum @robertrosenbaum.bsky.social · 23h

We're hiring 5 T/TT faculty in Neuroscience, including Computational Neuroscience, at U Notre Dame

We'll start reviewing applications very soon, so if you're thinking about applying, please apply now/soon!

apply.interfolio.com/173031

Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

apply.interfolio.com

2

Robert Rosenbaum @robertrosenbaum.bsky.social · 7d

I'm posting this again for anyone who might have missed it last time:

Notre Dame is hiring 5 tenure or tenure-track professors in Neuroscience, including Computational Neuroscience, across 4 departments.

Feel free to reach out with any questions.

And please share!

apply.interfolio.com/173031

Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

apply.interfolio.com

4

Robert Rosenbaum @robertrosenbaum.bsky.social · Sep 3

The University of Notre Dame is hiring 5 tenure or tenure-track professors in Neuroscience, including Computational Neuroscience, across 4 departments.

Come join me at ND! Feel free to reach out with any questions.

And please share!

apply.interfolio.com/173031

Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

apply.interfolio.com

33 39

Robert Rosenbaum @robertrosenbaum.bsky.social · Jun 3

Fixed! (I think) Try again and let me know if you still have trouble. You might need to refresh the page.

Robert Rosenbaum @robertrosenbaum.bsky.social · Jun 3

Thank you for the feedback, I'll work on both of those!

Robert Rosenbaum @robertrosenbaum.bsky.social · Jun 2

Thanks for the suggestion, that makes sense. I am just trying to figure out the best implementation. It's difficult (for me) to combine email verification and profile creation on the same page. Maybe a link to a screenshot of an example profile on the registration page?

Robert Rosenbaum @robertrosenbaum.bsky.social · Jun 2

Most universities have generous "Conflict of Commitment" policies that allow faculty to devote a portion of their time to consulting work, but these policies are under-utilized.

Consulting work can provide valuable industry experience, and also extra cash.

1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 20

Couldn't the same argument be made for conference presentations (which 90% of the time only describe published work)?

3

Robert Rosenbaum @robertrosenbaum.bsky.social · May 20

When _you_ publish a new paper, lots of people notice, lots of people read it. No explainer thread needed. Deservedly so, because you have a reputation for writing great papers.

When Dr. Average Scientist publishes a paper, nobody notices, nobody reads it without some leg work to get it out there

25

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

Thanks! Let us know if you have comments or questions

1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

In other words:
Plasticity rules like Oja's let us go beyond studying how synaptic plasticity in the brain can _match_ the performance of backprop.

Now, we can study how synaptic plasticity can _beat_ backprop in challenging, but realistic learning scenarios.

1 4

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

Finally, we meta-learned pure plasticity rules with no weight transport, extending our previous work. When Oja's rule was included, the meta-learned rule _outperformed_ pure backprop.

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

We find that Oja's rule works, in part, by preserving information about inputs in hidden layers. This is related to its known properties in forming orthogonal representations. Check the paper for more details.

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

Vanilla RNNs trained with pure BPTT fail on simple memory tasks. Adding Oja's rule to BPTT drastically improves performance.

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

We often forget how important careful weight initialization is for training neural nets because our software initializes them for us. Adding Oja's rule to backprop also eliminates the need for careful weight initialization.

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

We propose that plasticity rules like Oja's rule might be part of the answer. Adding Oja's rule to backprop improves learning in deep networks in an online setting (batch size 1).

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

For example, a 10-layer ffwd network trained on MNIST using online learning (batch size 1) performs poorly when trained with pure backprop. How does the brain learn effectively without all of these engineering hacks?

1 1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

In our new preprint, we dug deeper into this observation. Our motivation is that modern machine learning depends on lots of engineering hacks beyond pure backprop: gradients averaged over batches, batchnorm, momentum, etc. These hacks don't have clear, direct biological analogues.

1 1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

In previous work on this question, we meta-learned linear combos of plasticity rules. In doing so, we noticed something intersting:

One plasticity rule improved learning, but its weight updates weren't aligned with backprop's. It was doing something different. That rule is Oja's plasticity rule.

1 2

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

A lot of work in "NeuroAI," including our own, seeks to understand how synaptic plasticity rules can match the performance of backprop in training neural nets.

1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 19

New preprint with my postdoc, Navid Shervani-Tabar, and former postdoc, Marzieh Alireza Mirhoseini.

Oja’s plasticity rule overcomes challenges of training neural networks under biological constraints.

arxiv.org/abs/2408.08408

Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints

Deep neural networks have achieved impressive performance through carefully engineered training strategies. Nonetheless, such methods lack parallels in biological neural circuits, relying heavily on n...

arxiv.org

2 6 27

Reposted by Robert Rosenbaum

Nosrat Mohammadi @nosrat.bsky.social · May 16

I made this figure panel size guide to avoid thinking about dimensions every time. Apparently this post is going to be a 🧵! So feel free to bookmark it and save some time of yours.

If this seems empty it's bcz I don't plan to use it anytime soon!

7 19 84

Robert Rosenbaum @robertrosenbaum.bsky.social · May 15

Interesting comment, but you need to define what you mean by "neuroanatomy." Does such a thing actually exist? As a thing in itself or as a phenomenon? What would Kant have to say? ;)

1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 13

Sorry, I didn't mean to phrase that antagonistically.

I just think that unless we're talking just about anatomy and we're restricting to a direct synaptic pathway (which maybe you are) then it's difficult to make this type of question precise without concluding that everything can query everything

1 1

Robert Rosenbaum @robertrosenbaum.bsky.social · May 13

Unless we're talking about a direct synapse, I don't know how we can expect to answer this question meaningfully when a neuromuscular junction in my pinky toe can "readout" and "query" photoreceptors in my retina.

1