Lightnews — Scholar-powered news

Matteo Tiezzi @mtiezzi.bsky.social · May 15

🚨 Survey Alert 🚨

Our paper "Back to recurrent processing at the crossroad of transformers and state-space models" is out in Nature Machine Intelligence!

🔗 www.nature.com/articles/s42...
Read it here: rdcu.be/el85K

#Mamba #SSMs #Transformers #LLMs

Back to recurrent processing at the crossroad of transformers and state-space models - Nature Machine Intelligence

While transformers and large language models excel at efficiently processing long sequences, new approaches have been proposed that incorporate recurrence to overcome the quadratic cost of self-attent...

www.nature.com

2

Matteo Tiezzi @mtiezzi.bsky.social · Feb 19

(Top) A standard MLP catastrophically forgets samples learned online when data distribution shifts

(Bottom) A Memory Head is capable to avoid forgetting thanks to its learnable key-value routing mechanism. Keys become representative of the data distribution.

proceedings.mlr.press/v274/tiezzi2...

Matteo Tiezzi @mtiezzi.bsky.social · Feb 19

TL;DR: in Memory Heads, neurons route their computation across a learnable key-value mechanism
⚡️ dynamic behaviour depending on their own input.
⚡ Only the units(weights) that are relevant to process the observed sample are blended ➡ parameter isolation

proceedings.mlr.press/v274/tiezzi2...

1

Matteo Tiezzi @mtiezzi.bsky.social · Feb 19

@collasconf.bsky.social The #CoLLAs 2024 proceedings of our paper "Memory Head for Pre-Trained Backbones in Continual Learning" are out:

proceedings.mlr.press/v274/tiezzi2...

Video: lifelong-ml.cc/Conferences/...

🖥 Memory Head Code repository: github.com/sailab-code/...

A thread🧵

Memory Head for Pre-Trained Backbones in Continual Learning

This paper is focused on the role of classification heads for pre-trained backbones in the context of continual learning. A novel neuron model is proposed as basic constituent of what we refer to a...

proceedings.mlr.press

1 1

Reposted by Matteo Tiezzi

Ali Behrouz @alibehrouz.bsky.social · Dec 3

🌟 Best of Both Worlds!

❓Have you ever wondered why hybrid models (RNN + Transformers) are powerful? We answer this through the lens of circuit complexity and graphs!

Excited to share our work on understanding Graph Sequence Models (GSM), which allows the use of any sequence model for graphs.

1 1 7

Matteo Tiezzi @mtiezzi.bsky.social · Dec 26

🐱 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy
🔗 sites.google.com/unisi.it/lot...

Organizers:
Stefano Melacci, V. Lomonaco, Andrea Cossu, Alessandro Betti, @mtiezzi.bsky.social

#ContinualLearning #CL #LifelongLearning #CollectionlessAI

LOT Spring School

Topics

sites.google.com

Matteo Tiezzi @mtiezzi.bsky.social · Dec 26

😸 Join the LOT Spring School with an incredible speaker lineup:
Battista Biggio, Nicolò Cesa-Bianchi, Elisa Ricci, Antonio Orvieto, Marco Gori, Tinne, Tinne Tuytelaars

🔗 sites.google.com/unisi.it/lot...

🔗 Pre-registrations are now open, secure your spot today: sites.google.com/unisi.it/lot...

LOT Spring School - Registration

PRE-REGISTRATION IS REQUIRED! Fill the form at the bottom of the page to pre-register now! Payment procedures will be activated in january 2025.

sites.google.com

1 2

Matteo Tiezzi @mtiezzi.bsky.social · Dec 26

🐱 Learning Over Time (LOT) Spring School
📅 24-27 March 2025 | 📍 Siena, Italy

💡 Are you considering models that continuously adapt over time instead of learning "offline" from pre-designed-huge collections of data? 🚀

🔗 sites.google.com/unisi.it/lot...

#ContinualLearning #CL #LifelongLearning

LOT Spring School

Topics

sites.google.com

1 2 3

Matteo Tiezzi @mtiezzi.bsky.social · Dec 18

Would love to join, working in CL and I was an organizer of CoLLAs 2024!

Reposted by Matteo Tiezzi

Fabrizio Frasca @ffabffrasca.bsky.social · Dec 3

🌟 GLOW is coming back in December with amazing speakers: Emily Jin and @joshsouthern.bsky.social !

🗓️ Dec 18th @ 17 CET on Zoom, don't miss that!

🌐 Find more here: sites.google.com/view/graph-l...

1 5 14

Reposted by Matteo Tiezzi

Max Thiessen @maxthiessen.bsky.social · Dec 17

Join this year's last edition of Graph Learning on Wednesdays (GLOW) tomorrow at 5pm CET!

Fabrizio Frasca @ffabffrasca.bsky.social · Dec 3

🌟 GLOW is coming back in December with amazing speakers: Emily Jin and @joshsouthern.bsky.social !

🗓️ Dec 18th @ 17 CET on Zoom, don't miss that!

🌐 Find more here: sites.google.com/view/graph-l...

3 4

Reposted by Matteo Tiezzi

Kimon Fountoulakis @kfountou.bsky.social · Nov 16

Here is an initial starter pack list on Machine Learning on Graphs: go.bsky.app/HN2MTzp

Machine Learning on Graphs

Join the conversation

go.bsky.app

18 21 34

Matteo Tiezzi @mtiezzi.bsky.social · Dec 17

May I join? Thanks

1

Matteo Tiezzi @mtiezzi.bsky.social · Dec 17

Entering the field, could you add me? Thanks!

Matteo Tiezzi @mtiezzi.bsky.social · Dec 17

✋

1

Matteo Tiezzi @mtiezzi.bsky.social · Dec 8

Starter pack for people in Lifelong and Continual Learning ✍️

Jorge Mendez-Mendez @jmendezm.bsky.social · Nov 23

I created a starter pack of folks working on lifelong or continual learning. It's quite sparse, so if you know anybody else who's around (or want to be added yourself) please reply below.

go.bsky.app/QXGY1iT

Matteo Tiezzi @mtiezzi.bsky.social · Dec 6

@logconference.bsky.social #LOG meetup with @federico-errica.bsky.social open talk on "What is going on with oversmoothing, oversquashing, and underreaching?"

1 2

Matteo Tiezzi @mtiezzi.bsky.social · Dec 6

Wonderful location for the Siena #LoG meetup's poster session🤩

3

Matteo Tiezzi @mtiezzi.bsky.social · Dec 4

#LOG2024 Franco Scarselli kicking off the Learning on Graphs (LOG) Italian Meetup!!!

4

Matteo Tiezzi @mtiezzi.bsky.social · Nov 30

Working in Lifelong/Continual Learning, and I was among the organisers of last CoLLAs conference! Please add me☺️

1