Lightnews — Scholar-powered news

Jay Alammar @jayalammar.bsky.social · Aug 19

The Illustrated GPT-OSS

New post! A visual tour of the architecture, message formatting, and reasoning of the latest GPT.

newsletter.languagemodels.co/p/the-illust...

1 6 20

Jay Alammar @jayalammar.bsky.social · May 23

The legendary John Carmack at #upperbound:
- Current AI focus is RL (with Richard Sutton) solving Atari games
- Thinking in line with the Alberta Plan.
- It was a misstep to start working too low-level (e.g., at the cuda level). I kept stepping up the stack chain until now in pytorch

7

Reposted by Jay Alammar

Adam Hill @astroadamh.bsky.social · May 20

I'm really excited for this year's PyData London conference - there are some awesome talks on the schedule and I'm excited to hear the keynote speakers @jayalammar.bsky.social, Tony Wears, & Leanne Fitzpatrick

#pydata #datascience

PyData London @pydatalondon.bsky.social · May 20

Unleash your inner data aficionado at PyData London 2025, 6-8 June at Convene Sancroft, St. Paul’s!

We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.

Just 17 days left. Book your tickets now!
pydata.org/london2025

Advertisement for PyData London 2025 conference.

Headline: Meet your keynote speakers

- Jay Alammar
- Tony Mears
- Leanne Fitzpatrick

Book your tickets
https://pydata.org/london2025

1 3

Reposted by Jay Alammar

PyData London @pydatalondon.bsky.social · May 20

Unleash your inner data aficionado at PyData London 2025, 6-8 June at Convene Sancroft, St. Paul’s!

We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.

Just 17 days left. Book your tickets now!
pydata.org/london2025

1 6 7

Reposted by Jay Alammar

Max Bartolo @maxbartolo.bsky.social · Mar 27

I'm excited to share the tech report for our @cohere.com @cohereforai.bsky.social Command A and Command R7B models. We highlight our novel approach to model training including self-refinement algorithms and model merging techniques at scale. Read more below! ⬇️

1 4 11

Reposted by Jay Alammar

Tom Aarsen @tomaarsen.com · Feb 21

We've just released MMTEB, our multilingual upgrade to the MTEB Embedding Benchmark!

It's a huge collaboration between 56 universities, labs, and organizations, resulting in a massive benchmark of 1000+ languages, 500+ tasks, and a dozen+ domains.

Details in 🧵

2 4 23

Reposted by Jay Alammar

Maarten Grootendorst @maartengr.bsky.social · Feb 11

Did you know we continue to develop new content for the "Hands-On Large Language Models" book?

There's now even a free course available with
@deeplearningai.bsky.social!

1 3 11

Reposted by Jay Alammar

Mason Youngblood @masonyoungblood.bsky.social · Feb 5

Do whales optimize their vocalizations for efficiency, just like human language? 🐋🎶 My latest study in
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...

Language-like efficiency in whale communication

Whale vocalizations follow efficiency rules seen in human language, revealing striking similarities in communication systems.

www.science.org

5 66 200

Reposted by Jay Alammar

Ellen Garland @ellengarland.bsky.social · Feb 6

We uncovered the same statistical structure that is a hallmark of human language in whale song, published today in Science. @inbalarnon.bsky.social @simonkirby.bsky.social @jennyallen13.bsky.social @clairenea.bsky.social @emma-carroll.bsky.social
www.science.org/doi/10.1126/...

17 100 270

Reposted by Jay Alammar

Naomi Saphra @nsaphra.bsky.social · Jan 27

One of my grand interpretability goals is to improve human scientific understanding by analyzing scientific discovery models, but this is the most convincing case yet that we CAN learn from model interpretation: Chess grandmasters learned new play concepts from AlphaZero's internal representations.

Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains. This presents us with an opportunity to further human knowledge and improv...

arxiv.org

2 22 110

Jay Alammar @jayalammar.bsky.social · Jan 27

The Illustrated DeepSeek-R1

Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.

newsletter.languagemodels.co/p/the-illust...

1 22 72

Jay Alammar @jayalammar.bsky.social · Jan 15

Alphaxiv is an awesome way to discuss ML papers -- often with the authors themselves. Here's an intro and demo by @rajpalleti.bsky.social we shot at #Neurips2024

www.youtube.com/watch?v=-Kwl...

AlphaXiv - a great place to discuss ML papers

YouTube video by Jay Alammar

www.youtube.com

3 11

Reposted by Jay Alammar

Tom Aarsen @tomaarsen.com · Jan 14

The newest extremely strong embedding model based on ModernBERT-base is out: `cde-small-v2`. Both faster and stronger than its predecessor, this one tops the MTEB leaderboard for its tiny size!

Details in 🧵

1 7 30

Jay Alammar @jayalammar.bsky.social · Jan 14

It contains a notebook per chapter containing all the code examples in the book.

Check it out here: github.com/HandsOnLLM/H...

GitHub - HandsOnLLM/Hands-On-Large-Language-Models: Official code repo for the O'Reilly Book - "Hands-On Large Language Models"

Official code repo for the O'Reilly Book - "Hands-On Large Language Models" - HandsOnLLM/Hands-On-Large-Language-Models

github.com

1

Jay Alammar @jayalammar.bsky.social · Jan 14

Floored that the repo for Hands-On Large Language Models is now at 3.6k Github stars!

And excited that professors are starting to use the book to teach LLM courses. Reach out to us if we can be of assistance!

And if you've liked the book, leave us a review on Amazon or Goodreads!

1 10

Jay Alammar @jayalammar.bsky.social · Jan 13

SWE-Bench has been one of the most important tasks measuring the progress of agents tackling software engineering in 2024. I caught up with two of its creators, @ofirpress.bsky.social and Carlos E. Jimenez to share their ideas on the state of LLM-backed agents.

www.youtube.com/watch?v=bivZ...

SWE-Bench authors reflect on the state of LLM agents at Neurips 2024

YouTube video by Jay Alammar

www.youtube.com

2 3

Reposted by Jay Alammar

Nathan Lambert @natolambert.bsky.social · Dec 20

OpenAI's o3: The grand finale of AI in 2024
A step change as influential as the release of GPT-4. Reasoning language models are the current and next big thing.

I explain:
* The ARC prize
* o3 model size / cost
* Dispelling training myths
* Extreme benchmark progress

o3: The grand finale of AI in 2024

A step change as influential as the release of GPT-4. Reasoning language models are the current big thing.

buff.ly

8 12 81

Jay Alammar @jayalammar.bsky.social · Dec 12

Good morning #NeurIPS2024! Stop by the @cohere.com booth at 3PM today (Thursday) for a signed copy of Hands-On Large Language Models - it will introduce you to LLMs, their applications, as well as Cohere's Embed, Rerank, and Command-R models.

Come early as quantities are limited!

7

Jay Alammar @jayalammar.bsky.social · Dec 11

I'll be in the Cohere #NeurIPS2024 booth most of this afternoon. Come say hi, ask questions, and yes, we're hiring!

Tomorrow I'll be signing copies of my book at 3PM! Limited copies available!

6

Jay Alammar @jayalammar.bsky.social · Dec 10

Hi NeurIPS!

Explore ~4,500 NeurIPS papers in this interactive visualization:

jalammar.github.io/assets/neuri...
(Click on a point to see the paper on the website)

Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.

1 13 62

Jay Alammar @jayalammar.bsky.social · Nov 29

Sure to be thought provoking. The previous interview had fascinating thoughts on scifi (Dune Vs. Foundation), on AI competition for AI safety, and on successful scifi as a self-preventing prophecy.

David Brin @davidbrin.bsky.social · Nov 28

Tim Ventura interviewed me about big perspectives on AI. Can't put the AI genie back in the bottle, so how to make it safe? Come explore ethical, legal & safety implications of artificial intelligence.
youtu.be/34_Tub6vXDk

David Brin - Artificial Intelligence Safety

YouTube video by Tim Ventura

youtu.be

5

Jay Alammar @jayalammar.bsky.social · Nov 28

Excited to see you all at NeurIPS this year! Let's hang!

1 2

Jay Alammar @jayalammar.bsky.social · Nov 26

Great insights

Ofir Press @ofirpress.bsky.social · Nov 25

I wrote some thoughts on how to build good LM benchmarks: ofir.io/How-to-Build...

How to Build Good Language Modeling Benchmarks

Building benchmarks is important because they shine a spotlight on the weaknesses of existing language models and so can guide the community on how to improve them.

ofir.io

1 1

Jay Alammar @jayalammar.bsky.social · Nov 26

Join us for a panel on scientific communication Dec 4!

Marzieh Fadaee @mziizm.bsky.social · Nov 26

I'm always thinking about the aspects of research that aren't discussed enough online, and one key area is effective scientific communication.

Very excited about this panel discussion featuring two amazing researchers: @jayalammar.bsky.social and @shaynelongpre.bsky.social

Jay Alammar @jayalammar.bsky.social · Nov 23

Looks good! Do you have to leave town for better visibility or is the light pollution not that bad?

2 1