Dan Roy
@roydanroy.bsky.social
Research Director, Founding Faculty, Canada CIFAR AI Chair @VectorInst.
Full Prof @UofT - Statistics and Computer Sci. (x-appt) danroy.org
I study assumption-free prediction and decision making under uncertainty, with inference emerging from optimality.
Full Prof @UofT - Statistics and Computer Sci. (x-appt) danroy.org
I study assumption-free prediction and decision making under uncertainty, with inference emerging from optimality.
Tian and Karolina and team are at ICLR. Come say hi.
📣 The Journey Matters: Our #ICLR2025 paper shows how to pretrain sparse LLMs with half the size of dense LLMs while maintaining quality. We found that the average parameter count during sparse pre-training predicts quality, not final size. An MIT/Rice/Google/ISTA collab 🧵 1/N
April 21, 2025 at 1:00 PM
Tian and Karolina and team are at ICLR. Come say hi.
So who’s read the 2027 article? What do you think?
April 7, 2025 at 12:47 AM
So who’s read the 2027 article? What do you think?
Someone has suggested I check out bsky again. So I'm back looking around here. Notification list is kinda boring. So any good conversations going on? Perhaps about LLM/AI reasoning?
March 23, 2025 at 9:10 PM
Someone has suggested I check out bsky again. So I'm back looking around here. Notification list is kinda boring. So any good conversations going on? Perhaps about LLM/AI reasoning?
Anyone else have the worry that a lot of LLM research is .... just bad psychology?
February 3, 2025 at 3:26 AM
Anyone else have the worry that a lot of LLM research is .... just bad psychology?
Reposted by Dan Roy
Okay, so just a few* thoughts (*this got longer as I wrote 😅….long thread)-
Having a lot of thoughts & feelings today as someone who’s worked both on FBs misinfo interventions (back when they were making investments) and Birdwatch (pre-Musk Community Notes) 💔. Debating if a thread is worth the inevitable headache 😓
January 8, 2025 at 12:40 PM
Okay, so just a few* thoughts (*this got longer as I wrote 😅….long thread)-
I got to ski Revelstoke this winter break.
Couple observations: the price of receiving 600 cm of snow by Jan 8 is that it is constantly snowing. Saw almost no sun the whole time and the peak was often in whiteout conditions (though North Bowl was always clear…).
See image for more.
Couple observations: the price of receiving 600 cm of snow by Jan 8 is that it is constantly snowing. Saw almost no sun the whole time and the peak was often in whiteout conditions (though North Bowl was always clear…).
See image for more.
January 8, 2025 at 4:56 PM
I got to ski Revelstoke this winter break.
Couple observations: the price of receiving 600 cm of snow by Jan 8 is that it is constantly snowing. Saw almost no sun the whole time and the peak was often in whiteout conditions (though North Bowl was always clear…).
See image for more.
Couple observations: the price of receiving 600 cm of snow by Jan 8 is that it is constantly snowing. Saw almost no sun the whole time and the peak was often in whiteout conditions (though North Bowl was always clear…).
See image for more.
Multiple friends have likely lost their homes in Los Angeles. Can’t imagine how disorienting this would be. They had only minutes to flee and grab belongings.
January 8, 2025 at 4:55 PM
Multiple friends have likely lost their homes in Los Angeles. Can’t imagine how disorienting this would be. They had only minutes to flee and grab belongings.
OK. Practical question times. How are you adjusting your research given progress in reasoning style models? Also how are you adjusting the way you work?
December 22, 2024 at 7:39 AM
OK. Practical question times. How are you adjusting your research given progress in reasoning style models? Also how are you adjusting the way you work?
Pretraining is not done. It's just that theorists haven't told the hackers how to do it better.
ILYA: "PRETRAINING IS DONE. WE ARE NOW IN THE POST TRAINING ERA."
December 13, 2024 at 10:52 PM
Pretraining is not done. It's just that theorists haven't told the hackers how to do it better.
Now that I've had a taste of X without post length limitations, I've got to say that it is quite annoying have to fit tweets into 256 characters here on bsky. On X, when they get to long, they go below the fold, and so you're still incentivized to make it short. Can't we have that here?
December 12, 2024 at 10:35 PM
Now that I've had a taste of X without post length limitations, I've got to say that it is quite annoying have to fit tweets into 256 characters here on bsky. On X, when they get to long, they go below the fold, and so you're still incentivized to make it short. Can't we have that here?
I've now read this paper carefully if anyone wants to discuss it.
December 10, 2024 at 10:50 PM
I've now read this paper carefully if anyone wants to discuss it.
I've never understood people who block people on Twitter, outside of cases of harassment. Such weak sauce.
December 6, 2024 at 5:04 PM
I've never understood people who block people on Twitter, outside of cases of harassment. Such weak sauce.
Has anyone out there compared the GPU/memory specs for the new Macbook Pro M4 Pro/Max computers to the requirements to run inference in the more interesting open source LLM models, especially those aimed at reasoning?
[1/2]
[1/2]
December 6, 2024 at 5:03 PM
Has anyone out there compared the GPU/memory specs for the new Macbook Pro M4 Pro/Max computers to the requirements to run inference in the more interesting open source LLM models, especially those aimed at reasoning?
[1/2]
[1/2]
Has anyone read the new Shalev-Shwartz paper on PAC reasoning yet and want to discuss?
December 6, 2024 at 1:54 PM
Has anyone read the new Shalev-Shwartz paper on PAC reasoning yet and want to discuss?
Reposted by Dan Roy
*AI for Mathematics and Theoretical Computer Science*, a workshop hosted jointly by the Simons Institute for the Theory of Computing and the Simons Laufer Mathematical Sciences Institute, will be held in Berkeley at April 7-11, 2025.
simons.berkeley.edu/workshops/si...
simons.berkeley.edu/workshops/si...
Simons Institute for the Theory of Computing and SLMath Joint Workshop: AI for Mathematics and Theoretical Computer Science
This is an exciting time for mathematics, as new technologies for mathematical reasoning provide novel opportunities for mathematical research, communication, and discovery. Mathlib, a library of form...
simons.berkeley.edu
December 2, 2024 at 10:08 PM
*AI for Mathematics and Theoretical Computer Science*, a workshop hosted jointly by the Simons Institute for the Theory of Computing and the Simons Laufer Mathematical Sciences Institute, will be held in Berkeley at April 7-11, 2025.
simons.berkeley.edu/workshops/si...
simons.berkeley.edu/workshops/si...
Huge community response to this. Much appreciated.
My 9 yo has finished all the Harry Potters. Any suggestion on where to go from here? Maybe age appropriate sci fi?
December 2, 2024 at 12:09 PM
Huge community response to this. Much appreciated.
My 9 yo has finished all the Harry Potters. Any suggestion on where to go from here? Maybe age appropriate sci fi?
December 1, 2024 at 12:11 PM
My 9 yo has finished all the Harry Potters. Any suggestion on where to go from here? Maybe age appropriate sci fi?
Reposted by Dan Roy
Poem by Joseph Fasano
November 29, 2024 at 8:57 PM
Poem by Joseph Fasano
Congrats to Antonio Lijoi on being elected President of the International Society of Bayesian Analysis!
November 28, 2024 at 5:02 AM
Congrats to Antonio Lijoi on being elected President of the International Society of Bayesian Analysis!
NeurIPS Test of Time Awards:
Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le
Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le
November 27, 2024 at 5:32 PM
NeurIPS Test of Time Awards:
Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le
Generative Adversarial Nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le
As the semester is coming to an end, I've been able to steal some time to do research on my own. Such a buzz.
November 27, 2024 at 1:52 AM
As the semester is coming to an end, I've been able to steal some time to do research on my own. Such a buzz.
School’s out for summer.
School’s out for ever!
School’s out for ever!
November 26, 2024 at 7:08 PM
School’s out for summer.
School’s out for ever!
School’s out for ever!
“Next week we will derive [something awesome],” bragged Prof Roy.
“This is our last class, Professor!”, said the brave student.
“This is our last class, Professor!”, said the brave student.
November 26, 2024 at 5:12 PM
“Next week we will derive [something awesome],” bragged Prof Roy.
“This is our last class, Professor!”, said the brave student.
“This is our last class, Professor!”, said the brave student.
Waterloo is facing a 75 million CAD deficit and so they are implementing a hiring freeze. As one of the world’s top universities, it is ridiculous that Ontario is not funding it fully. It has generated so much SV wealth, I suggest a SV billionaire solve this deficit and then some.
November 24, 2024 at 8:33 PM
Waterloo is facing a 75 million CAD deficit and so they are implementing a hiring freeze. As one of the world’s top universities, it is ridiculous that Ontario is not funding it fully. It has generated so much SV wealth, I suggest a SV billionaire solve this deficit and then some.