Lightnews — Scholar-powered news

Petar Veličković @petar-v.bsky.social · Jun 20

You may read this chapter in the usual place:

geometricdeeplearning.com/book/ 📖

Any and all feedback / thoughts / questions on the content... words of encouragement for finishing this book (pretty please! 😇) warmly welcomed!

@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social

1 3 19

Petar Veličković @petar-v.bsky.social · Jun 20

The first draft 'G' chapter of the geometric deep learning book is live! 🚀

Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!)

I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎

1 5 21

Petar Veličković @petar-v.bsky.social · May 26

...but our model regressed on Quickselect rather than improving it, so I'm indeed curious to explore what you've done differently :)

1 1

Petar Veličković @petar-v.bsky.social · May 26

Cool stuff! Will give it a read.

We also built a 'fully tropical' GNN following similar principles a few years back (from
the perspective of asynchrony invariance), might be a good reference:

arxiv.org/abs/2306.15632

1 1

Petar Veličković @petar-v.bsky.social · Apr 29

I would say it definitely does! The book does rely on theoretical material but is designed to be a self-contained, deep learning-oriented introduction to those topics.

So if you read the chapters in sequence it _should_ be just fine :)

(would appreciate any feedback if you do try it!)

1

Petar Veličković @petar-v.bsky.social · Apr 29

Read on at
geometricdeeplearning.com/book 📖

Comments, errata reports, and feedback warmly welcomed!

GDL Book

Grids, Groups, Graphs, Geodesics, and Gauges

geometricdeeplearning.com

5

Petar Veličković @petar-v.bsky.social · Apr 29

4yrs ago, the Geometric Deep Learning proto-book hit the arXiv 📚

Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀

Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢

@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social

2 5 57

Petar Veličković @petar-v.bsky.social · Mar 31

youtu.be/J1YCdVogd14?...

LLMs as Graph Neural Networks | Petar Veličković @ GLOW

YouTube video by GLOW

youtu.be

4

Petar Veličković @petar-v.bsky.social · Mar 31

The recording of my 'LLMs as GNNs' talk is now public (link in thread) 💬🕸️

Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟

(And I look forward to refining this content even further -- especially as new works come trickling in 👀)

1 1 12

Petar Veličković @petar-v.bsky.social · Mar 26

This is happening today! 🌟

Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️

2 12

Reposted by Petar Veličković

Max Thiessen @maxthiessen.bsky.social · Mar 20

GLOW is returning on 𝗠𝗮𝗿𝗰𝗵 𝟮𝟲𝘁𝗵, 𝟱𝗽𝗺 𝗖𝗘𝗧 with a special guest: @petar-v.bsky.social 🌟

He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session.

Specifically, we will learn how Graph ML tools can help understand LLM generalisation

1 7 12

Petar Veličković @petar-v.bsky.social · Feb 14

Seminar details + join instructions here:

www.cst.cam.ac.uk/seminars/lis...

Hope to see many of you there!

Probabilistic weather forecasting with machine learning | Department of Computer Science and Technology

Abstract: Leading up to 2024, rapid advances in machine learning (ML)-based weather prediction (MLWP) had produced ML-based models which exhibit less forecast error than single NWP simulations.

www.cst.cam.ac.uk

2

Petar Veličković @petar-v.bsky.social · Feb 14

I'm excited to share that we'll have Ilan Price giving a talk at the University of Cambridge on GenCast -- a state-of-the-art model for probabilistic weather forecasting 🌦️⛈️⛅️

If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all!

All details (+ Zoom link) below!

1 2 11

Petar Veličković @petar-v.bsky.social · Feb 11

It's been great working on this with Alex, @joao.omg.lol and Marc on this over the past couple of months!

We don't believe we solved the problems studied here -- mainly brought them to attention -- and we welcome any and all comments!

Read our work on the arXiv:
arxiv.org/abs/2502.06751

What makes a good feedforward computational graph?

As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects ...

arxiv.org

6

Petar Veličković @petar-v.bsky.social · Feb 11

And so we set out to understand _feedforward_ graphs (i.e. graphs w/o back edges) ⏩

Turns out these graphs are rather understudied for how often they are used in practice! 😮

We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀

1 3 29

Reposted by Petar Veličković

Chaitanya K. Joshi @chaitjo.bsky.social · Feb 10

Trying to copy the greats Pietro and @petar-v.bsky.social giving a guest lecture for the GDL class today!

1 15

Reposted by Petar Veličković

Simone Scardapane @sscardapane.bsky.social · Feb 10

*Round and Round We Go! What makes Rotary Positional Encodings useful?*

by @petar-v.bsky.social et al.

They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics.

arxiv.org/abs/2410.06205

1 6

Petar Veličković @petar-v.bsky.social · Feb 5

This summer, a group of elite AI scientists and engineers will come over to Sarajevo 🇧🇦 for #EEML2025 ❤️

We've been working hard behind the scenes to prepare a stellar program for our attendees!

Trust me, you don't want to miss this 🚀

Applications open now! (details in post)

EEML @eemlcommunity.bsky.social · Feb 5

Applications are now open for EEML 2025 in Sarajevo, Bosnia and Herzegovina, 21-26 July! 🎉

Learn from top AI researchers and connect with peers in Sarajevo 🇧🇦, a historical crossroads of East and West. Needs-based scholarships are available.

Deadline: 31 March 2025.

14

Petar Veličković @petar-v.bsky.social · Dec 19

Application forms should open early 2025 :)

1

Petar Veličković @petar-v.bsky.social · Dec 16

Exciting news: the NeurIPS Sci4DL Workshop has recognised our softmax paper as a best paper runner-up for its 'Debunking Challenge'! 🚀🧑‍🔬

I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌

10

Petar Veličković @petar-v.bsky.social · Dec 15

It's official! 🎉 I am excited to share that EEML will be coming to Sarajevo 🇧🇦 next summer!

For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!).

Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀

EEML @eemlcommunity.bsky.social · Dec 15

EEML'25, our yearly machine learning summer school event, will be organised next summer in the beautiful city of Sarajevo - the place where East meets West 🇧🇦🇧🇦🇧🇦.

More details coming soon, please see the link in the thread!

1 1 13

Petar Veličković @petar-v.bsky.social · Dec 15

Today it is time to present our softmax paper to the world! 🌡️

If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦

🧑‍🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am

🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am

Petar Veličković @petar-v.bsky.social · Dec 4

Just a poster I'm very proud of. Mainly because it's the first time I led a theory-focused paper 🥳

Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦

I unfortunately won't be there myself, but Christos and Federico will be around 🚀

2 21

Petar Veličković @petar-v.bsky.social · Dec 8

newsletter.pnote.eu/p/hashcode-d...

AI from DeepMind beats top human competitive programmers on Hash Code problems

🏆 A battle of heuristics. Evolutionary approach to improving a program. A "Deep Blue" moment for competitive programming?

newsletter.pnote.eu

3

Petar Veličković @petar-v.bsky.social · Dec 8

A very nice blog from Przemek Pietrzkiewicz, offering thoughts on our recent result in AI for competitive programming 🏆

Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀

Worth a read if you want to understand implications of our work! Link below ⬇️

1 3 22

Reposted by Petar Veličković

Marc Lanctot @sharky6000.bsky.social · Dec 6

If you will be at #NeurIPS2024 @neuripsconf.bsky.social and would like to come see our models in action, come say hi 👋 and check out our demo at the GDM booth!

Wednesday, Dec. 11th @ 9:30-10:00.

Lots of other great things to see as well! Check it out: 👇
deepmind.google/discover/blo...

1 6 41