Saurabh Prasad
@saurabhprasad-2.bsky.social
💙 Software Architect & Sr. Fullstack Engineer / Dev - Java, JS, Python, ML/AI
Other interests: CompSci, Tech, Startups, Academia, Research, Electronics, Mathematics, Physics, Space, Science, AR/VR, Robotics, Guitar, Piano, Sci-fi, Languages
Other interests: CompSci, Tech, Startups, Academia, Research, Electronics, Mathematics, Physics, Space, Science, AR/VR, Robotics, Guitar, Piano, Sci-fi, Languages
Reposted by Saurabh Prasad
Here's why "alignment research" when it comes to LLMs is a big mess, as I see it.
Claude is not a real guy. Claude is a character in the stories that an LLM has been programmed to write. Just to give it a distinct name, let's call the LLM "the Shoggoth".
Claude is not a real guy. Claude is a character in the stories that an LLM has been programmed to write. Just to give it a distinct name, let's call the LLM "the Shoggoth".
December 19, 2024 at 11:15 PM
Here's why "alignment research" when it comes to LLMs is a big mess, as I see it.
Claude is not a real guy. Claude is a character in the stories that an LLM has been programmed to write. Just to give it a distinct name, let's call the LLM "the Shoggoth".
Claude is not a real guy. Claude is a character in the stories that an LLM has been programmed to write. Just to give it a distinct name, let's call the LLM "the Shoggoth".
Happy new year, Bluesky peeps! 🎉🥳
Wish you all a very happy, healthy, and prosperous 2025! 💐
January 1, 2025 at 8:06 AM
Happy new year, Bluesky peeps! 🎉🥳
Wish you all a very happy, healthy, and prosperous 2025! 💐
Reposted by Saurabh Prasad
@camachocollados.bsky.social and I taught ML for #nlp last semester.
Here is a list of resources that we shared with the students (sorted by theme) in this blogpost: github.com/nedjmaou/CMT...
[Not all students were familiar with coding so it also includes resources for beginners.]
Here is a list of resources that we shared with the students (sorted by theme) in this blogpost: github.com/nedjmaou/CMT...
[Not all students were familiar with coding so it also includes resources for beginners.]
GitHub - nedjmaou/CMT122-2425-Resources: Resources for CMT122 students (2024-2025).
Resources for CMT122 students (2024-2025). Contribute to nedjmaou/CMT122-2425-Resources development by creating an account on GitHub.
github.com
December 24, 2024 at 11:24 AM
@camachocollados.bsky.social and I taught ML for #nlp last semester.
Here is a list of resources that we shared with the students (sorted by theme) in this blogpost: github.com/nedjmaou/CMT...
[Not all students were familiar with coding so it also includes resources for beginners.]
Here is a list of resources that we shared with the students (sorted by theme) in this blogpost: github.com/nedjmaou/CMT...
[Not all students were familiar with coding so it also includes resources for beginners.]
Reposted by Saurabh Prasad
#NeurIPS2024 wrapped up last week. I put together a curated reading list for #DeepRL and #reinforcementlearning work. (represents my interests).
Talks and workshops:
third-crowd-c77.notion.site/NeurIPS2024-...
Curated reading list
fracturedplane.notion.site/NeurIPS2024-...
#Holidayreading
Talks and workshops:
third-crowd-c77.notion.site/NeurIPS2024-...
Curated reading list
fracturedplane.notion.site/NeurIPS2024-...
#Holidayreading
NeurIPS2024 Related RL papers | Notion
Deep RL papers
fracturedplane.notion.site
December 23, 2024 at 7:38 PM
#NeurIPS2024 wrapped up last week. I put together a curated reading list for #DeepRL and #reinforcementlearning work. (represents my interests).
Talks and workshops:
third-crowd-c77.notion.site/NeurIPS2024-...
Curated reading list
fracturedplane.notion.site/NeurIPS2024-...
#Holidayreading
Talks and workshops:
third-crowd-c77.notion.site/NeurIPS2024-...
Curated reading list
fracturedplane.notion.site/NeurIPS2024-...
#Holidayreading
Interviewer: Can you explain this gap in your resume?
Physicist: I can only tell you about my momentum at that time, not the position
Physicist: I can only tell you about my momentum at that time, not the position
Interviewer: Can you explain this gap in your resume?
Data scientist: yeah it's just missing data
Data scientist: yeah it's just missing data
Interviewer: Can you explain this gap in your resume?
Archaeologist: It's probably something ritualistic.
Archaeologist: It's probably something ritualistic.
December 22, 2024 at 11:37 PM
Interviewer: Can you explain this gap in your resume?
Physicist: I can only tell you about my momentum at that time, not the position
Physicist: I can only tell you about my momentum at that time, not the position
Reposted by Saurabh Prasad
One of the most bizarre + complex talks at NeurIPS [1] was given by my fellow Yorkshireman, the inimitable Prof Karl Friston [2], explaining active inference to a room full of #AI people who are not really neuroscientists. This was interesting to me because... (1/n)
December 22, 2024 at 6:56 AM
One of the most bizarre + complex talks at NeurIPS [1] was given by my fellow Yorkshireman, the inimitable Prof Karl Friston [2], explaining active inference to a room full of #AI people who are not really neuroscientists. This was interesting to me because... (1/n)
Reposted by Saurabh Prasad
Transformer LMs get pretty far by acting like ngram models, so why do they learn syntax? A new paper by sunnytqin.bsky.social, me, and @dmelis.bsky.social illuminates grammar learning in a whirlwind tour of generalization, grokking, training dynamics, memorization, and random variation. #mlsky #nlp
Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization
Language models (LMs), like other neural networks, often favor shortcut heuristics based on surface-level patterns. Although LMs behave like n-gram models early in training, they must eventually learn...
arxiv.org
December 20, 2024 at 5:56 PM
Transformer LMs get pretty far by acting like ngram models, so why do they learn syntax? A new paper by sunnytqin.bsky.social, me, and @dmelis.bsky.social illuminates grammar learning in a whirlwind tour of generalization, grokking, training dynamics, memorization, and random variation. #mlsky #nlp
Reposted by Saurabh Prasad
🧪 Emory leverages #DigitaTwin tech to guide GLP-1 prescriptions for weight and diabetes care. An AI model simulates clinical decisions, providing expert-level support for patient suitability. 🩺💻 #MLSky
Supporting GLP-1 prescribing with digital twin technology | TechTarget
Learn how digital twin technology could support clinical decision-making for providers prescribing GLP-1s.
www.techtarget.com
December 20, 2024 at 3:25 PM
🧪 Emory leverages #DigitaTwin tech to guide GLP-1 prescriptions for weight and diabetes care. An AI model simulates clinical decisions, providing expert-level support for patient suitability. 🩺💻 #MLSky
Reposted by Saurabh Prasad
The ever elusive quantum computer moved a few crucial steps toward practical applications this year.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
The Year in Computer Science | Quanta Magazine
Researchers got a better look at chatbots’ thoughts, amateurs learned just how complicated simple systems can be, and codes became expert self-fixers.
www.quantamagazine.org
December 19, 2024 at 4:04 PM
The ever elusive quantum computer moved a few crucial steps toward practical applications this year.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
Reposted by Saurabh Prasad
Read about this year’s biggest moments in biology.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
The Year in Biology | Quanta Magazine
Biologists used artificial intelligence to make discoveries about molecules and the brain, and overturned long-held assumptions about the immune system and RNA.
www.quantamagazine.org
December 18, 2024 at 3:34 PM
Read about this year’s biggest moments in biology.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
Reposted by Saurabh Prasad
This year, researchers detected a hint of a signal that, if real, could upend and deepen our understanding of the fundamental laws of the universe. Read our annual review of the biggest developments in physics:
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
The Year in Physics | Quanta Magazine
Physicists discovered strange supersolids, constructed new kinds of superconductors, and continued to make the case that the cosmos is far weirder than anyone suspected.
www.quantamagazine.org
December 17, 2024 at 4:00 PM
This year, researchers detected a hint of a signal that, if real, could upend and deepen our understanding of the fundamental laws of the universe. Read our annual review of the biggest developments in physics:
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
Reposted by Saurabh Prasad
Here are the biggest breakthroughs that happened in mathematics this year.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
The Year in Math | Quanta Magazine
Landmark results in geometry and number theory marked an exciting year for mathematics, at a time when advances in artificial intelligence are starting to transform the subject’s future.
www.quantamagazine.org
December 16, 2024 at 6:00 PM
Here are the biggest breakthroughs that happened in mathematics this year.
www.quantamagazine.org/the-year-in-...
www.quantamagazine.org/the-year-in-...
Reposted by Saurabh Prasad
I'll get straight to the point.
We trained 2 new models. Like BERT, but modern. ModernBERT.
Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.
It's much faster, more accurate, longer context, and more useful. 🧵
We trained 2 new models. Like BERT, but modern. ModernBERT.
Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.
It's much faster, more accurate, longer context, and more useful. 🧵
December 19, 2024 at 4:45 PM
I'll get straight to the point.
We trained 2 new models. Like BERT, but modern. ModernBERT.
Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.
It's much faster, more accurate, longer context, and more useful. 🧵
We trained 2 new models. Like BERT, but modern. ModernBERT.
Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.
It's much faster, more accurate, longer context, and more useful. 🧵
Reposted by Saurabh Prasad
Great blog post (by a 15-author team!) on their release of ModernBERT, the continuing relevance of encoder-only models, and how they relate to, say, GPT-4/llama. Accessible enough that I might use this as an undergrad reading.
Finally, a Replacement for BERT: Introducing ModernBERT
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
December 19, 2024 at 7:11 PM
Great blog post (by a 15-author team!) on their release of ModernBERT, the continuing relevance of encoder-only models, and how they relate to, say, GPT-4/llama. Accessible enough that I might use this as an undergrad reading.
Reposted by Saurabh Prasad
My replies are perpetually full of anti-vaxxers these days telling me about polio vaccines.
Not shockingly, most of what they are saying is wrong. Luckily, I trained with Vincent Racaniello & he taught me a few things about poliovirus.
So let’s discuss the king of the Picornaviridae👇🏻
Not shockingly, most of what they are saying is wrong. Luckily, I trained with Vincent Racaniello & he taught me a few things about poliovirus.
So let’s discuss the king of the Picornaviridae👇🏻
December 18, 2024 at 1:07 PM
My replies are perpetually full of anti-vaxxers these days telling me about polio vaccines.
Not shockingly, most of what they are saying is wrong. Luckily, I trained with Vincent Racaniello & he taught me a few things about poliovirus.
So let’s discuss the king of the Picornaviridae👇🏻
Not shockingly, most of what they are saying is wrong. Luckily, I trained with Vincent Racaniello & he taught me a few things about poliovirus.
So let’s discuss the king of the Picornaviridae👇🏻
Reposted by Saurabh Prasad
Artificial intelligence is going to improve productivity. That will also create more wealth. How do you keep AI-enhanced productivity from only benefiting the wealthy? Raj Reddy, one of the pioneers of AI research, has ideas. www.quantamagazine.org/the-ai-pione...
The AI Pioneer With Provocative Plans for Humanity | Quanta Magazine
While some fret about technology’s social impacts, Raj Reddy still believes in the power of artificial intelligence to improve lives.
www.quantamagazine.org
December 4, 2024 at 3:06 PM
Artificial intelligence is going to improve productivity. That will also create more wealth. How do you keep AI-enhanced productivity from only benefiting the wealthy? Raj Reddy, one of the pioneers of AI research, has ideas. www.quantamagazine.org/the-ai-pione...
Reposted by Saurabh Prasad
It took four days from submission to publication, and nearly five years from publication to retraction. After campaigning by many, many scientists, and an investigation by Elsevier, an infamous paper on hydroxychloroquine as a COVID-19 treatment has been retracted. 🧪
www.science.org/content/arti...
www.science.org/content/arti...
Infamous paper that popularized unproven COVID-19 treatment finally retracted
Study on hydroxychloroquine by Didier Raoult and colleagues gets pulled on ethical and scientific grounds
www.science.org
December 17, 2024 at 5:31 PM
It took four days from submission to publication, and nearly five years from publication to retraction. After campaigning by many, many scientists, and an investigation by Elsevier, an infamous paper on hydroxychloroquine as a COVID-19 treatment has been retracted. 🧪
www.science.org/content/arti...
www.science.org/content/arti...
Reposted by Saurabh Prasad
Ilya Sutskever full talk "Sequence to sequence learning with neural networks: what a decade" at NeurIPS 2024 in Vancouver, Canada.
www.youtube.com/watch?v=1yvB...
www.youtube.com/watch?v=1yvB...
Ilya Sutskever: "Sequence to sequence learning with neural networks: what a decade"
YouTube video by seremot
www.youtube.com
December 15, 2024 at 9:16 PM
Ilya Sutskever full talk "Sequence to sequence learning with neural networks: what a decade" at NeurIPS 2024 in Vancouver, Canada.
www.youtube.com/watch?v=1yvB...
www.youtube.com/watch?v=1yvB...
Reposted by Saurabh Prasad
Want all NeurIPS/ICML/ICLR papers in one single .bib file? Here you go!
🗞️ short blog post: fabian-sp.github.io/posts/2024/1...
📇 bib files: github.com/fabian-sp/ml-bib
🗞️ short blog post: fabian-sp.github.io/posts/2024/1...
📇 bib files: github.com/fabian-sp/ml-bib
A Bibliography Database for Machine Learning
Getting the correct bibtex entry for a conference paper (e.g. published at NeurIPS, ICML, ICLR) is annoyingly hard: if you search for the title, you will often find a link to arxiv or to the pdf file,...
fabian-sp.github.io
December 17, 2024 at 10:42 AM
Want all NeurIPS/ICML/ICLR papers in one single .bib file? Here you go!
🗞️ short blog post: fabian-sp.github.io/posts/2024/1...
📇 bib files: github.com/fabian-sp/ml-bib
🗞️ short blog post: fabian-sp.github.io/posts/2024/1...
📇 bib files: github.com/fabian-sp/ml-bib
Reposted by Saurabh Prasad
Fun fact related to this thread:
Do you know what brought the ReLU activation function back into vogue in AI?
It was this paper from Yoshua's group, which was motivated by the observation that ReLU is a better match to real neurons' IO-functions!
proceedings.mlr.press/v15/glorot11a
🧠📈 #NeuroAI
Do you know what brought the ReLU activation function back into vogue in AI?
It was this paper from Yoshua's group, which was motivated by the observation that ReLU is a better match to real neurons' IO-functions!
proceedings.mlr.press/v15/glorot11a
🧠📈 #NeuroAI
December 17, 2024 at 5:35 PM
Fun fact related to this thread:
Do you know what brought the ReLU activation function back into vogue in AI?
It was this paper from Yoshua's group, which was motivated by the observation that ReLU is a better match to real neurons' IO-functions!
proceedings.mlr.press/v15/glorot11a
🧠📈 #NeuroAI
Do you know what brought the ReLU activation function back into vogue in AI?
It was this paper from Yoshua's group, which was motivated by the observation that ReLU is a better match to real neurons' IO-functions!
proceedings.mlr.press/v15/glorot11a
🧠📈 #NeuroAI
Reposted by Saurabh Prasad
1/ Okay, one thing that has been revealed to me from the replies to this is that many people don't know (or refuse to recognize) the following fact:
The unts in ANN are actually not a terrible approximation of how real neurons work!
A tiny 🧵.
🧠📈 #NeuroAI #MLSky
The unts in ANN are actually not a terrible approximation of how real neurons work!
A tiny 🧵.
🧠📈 #NeuroAI #MLSky
Why does anyone have any issue with this?
I've seen people suggesting it's problematic, that neuroscientists won't like it, and so on.
But, I literally don't see why this is problematic...
I've seen people suggesting it's problematic, that neuroscientists won't like it, and so on.
But, I literally don't see why this is problematic...
This would be funny if it weren't sad...
Coming from the "giants" of AI.
Or maybe this was posted out of context? Please clarify.
I can't process this...
Coming from the "giants" of AI.
Or maybe this was posted out of context? Please clarify.
I can't process this...
December 16, 2024 at 8:03 PM
Reposted by Saurabh Prasad
🌟Noether's razor⭐️ Our NeurIPS 2024 paper connects ML symmetries to conserved quantities through a seminal result in mathematical physics: Noether's theorem. We can learn neural network symmetries from data by learning associated conservation laws. Learn more👇. 1/16🧵
December 6, 2024 at 1:42 PM
🌟Noether's razor⭐️ Our NeurIPS 2024 paper connects ML symmetries to conserved quantities through a seminal result in mathematical physics: Noether's theorem. We can learn neural network symmetries from data by learning associated conservation laws. Learn more👇. 1/16🧵
Reposted by Saurabh Prasad
🎯 How can we empower scientific discovery in millions of nature photos?
Introducing INQUIRE: A benchmark testing if AI vision-language models can help scientists find biodiversity patterns- from disease symptoms to rare behaviors- hidden in vast image collections.
Thread👇🧵
Introducing INQUIRE: A benchmark testing if AI vision-language models can help scientists find biodiversity patterns- from disease symptoms to rare behaviors- hidden in vast image collections.
Thread👇🧵
December 6, 2024 at 8:28 PM
🎯 How can we empower scientific discovery in millions of nature photos?
Introducing INQUIRE: A benchmark testing if AI vision-language models can help scientists find biodiversity patterns- from disease symptoms to rare behaviors- hidden in vast image collections.
Thread👇🧵
Introducing INQUIRE: A benchmark testing if AI vision-language models can help scientists find biodiversity patterns- from disease symptoms to rare behaviors- hidden in vast image collections.
Thread👇🧵
Reposted by Saurabh Prasad
Gukesh Dommaraju becomes youngest world chess champion after horrific Ding Liren blunder
Gukesh Dommaraju becomes youngest world chess champion after horrific Ding Liren blunder
* Indian teenager becomes 18th world chess champion
* Modi hails ‘historic and exemplary’ achievement
* Move-by-move report of Game 14 – as it happened
* Play through 22 famous world championship games
Indian teenager Gukesh Dommaraju capped a…
www.theguardian.com
December 12, 2024 at 10:24 PM
Gukesh Dommaraju becomes youngest world chess champion after horrific Ding Liren blunder