Torsten Hoefler 🇨🇭
@thoefler.bsky.social
Professor ETHZ, head of SPCL, Chief Architect ML at CSCS researching large-scale #HPC and #AI systems and #Climate computing - youtube: http://bit.ly/3h1VgIU
Bill Dally on NVIDIA networking: "Jensen said: 'Absolutely not, we don't do networking, we're a GPU company'".
buff.ly/iUHv62C (4:30)
Then he talked the DOE into paying 100%, launching NVLINK
Example how a research lab changed the course of its organization. Another example is "all of AI" :-)
buff.ly/iUHv62C (4:30)
Then he talked the DOE into paying 100%, launching NVLINK
Example how a research lab changed the course of its organization. Another example is "all of AI" :-)
Insights From NVIDIA Research S73202 | GTC 2025 | NVIDIA On-Demand
The talk will give some highlights from NVIDIA Research for the past year. Detailed topics will be disclosed closer to the event.
buff.ly
November 26, 2025 at 6:00 AM
Bill Dally on NVIDIA networking: "Jensen said: 'Absolutely not, we don't do networking, we're a GPU company'".
buff.ly/iUHv62C (4:30)
Then he talked the DOE into paying 100%, launching NVLINK
Example how a research lab changed the course of its organization. Another example is "all of AI" :-)
buff.ly/iUHv62C (4:30)
Then he talked the DOE into paying 100%, launching NVLINK
Example how a research lab changed the course of its organization. Another example is "all of AI" :-)
Congrats to Saleh Ashkboos, SPCL's 13th PhD graduate (another prime number). A new expert in quantization and LLM optimization is born.
Thanks also to the great collaboration with Dan Alistarh from ISTA and James Hensman at Microsoft!
Thanks also to the great collaboration with Dan Alistarh from ISTA and James Hensman at Microsoft!
November 25, 2025 at 5:36 PM
Congrats to Saleh Ashkboos, SPCL's 13th PhD graduate (another prime number). A new expert in quantization and LLM optimization is born.
Thanks also to the great collaboration with Dan Alistarh from ISTA and James Hensman at Microsoft!
Thanks also to the great collaboration with Dan Alistarh from ISTA and James Hensman at Microsoft!
I am very honored to be part of to the two brilliant teams winning the 2025 Gordon Bell Prize for Climate Modeling and the 2025 Gordon Bell Prize (Honorable Mention). Both among the highest honors in #HPC. 🍾
buff.ly/gCkjDRN and buff.ly/4ewhdZ9
Congrats teams - Switzerland 🇨🇭 going strong - onward!
buff.ly/gCkjDRN and buff.ly/4ewhdZ9
Congrats teams - Switzerland 🇨🇭 going strong - onward!
November 24, 2025 at 6:00 AM
I am very honored to be part of to the two brilliant teams winning the 2025 Gordon Bell Prize for Climate Modeling and the 2025 Gordon Bell Prize (Honorable Mention). Both among the highest honors in #HPC. 🍾
buff.ly/gCkjDRN and buff.ly/4ewhdZ9
Congrats teams - Switzerland 🇨🇭 going strong - onward!
buff.ly/gCkjDRN and buff.ly/4ewhdZ9
Congrats teams - Switzerland 🇨🇭 going strong - onward!
Former member of SPCL, collaborator, and friend Daniele De Sensi speaks at both Broadcom's booth and at the main track of the Supercomputing conference.
Onward! We're all proud of you 🎉
Onward! We're all proud of you 🎉
November 21, 2025 at 12:18 PM
Former member of SPCL, collaborator, and friend Daniele De Sensi speaks at both Broadcom's booth and at the main track of the Supercomputing conference.
Onward! We're all proud of you 🎉
Onward! We're all proud of you 🎉
Reposted by Torsten Hoefler 🇨🇭
🌍 A 26-member team has been awarded the 2025 Gordon Bell Prize for Climate Modelling for their project, “Computing the Full Earth System at 1 km Resolution.”
Congratulations to this year’s outstanding team!
buff.ly/GKDYnH3
Congratulations to this year’s outstanding team!
buff.ly/GKDYnH3
November 20, 2025 at 8:30 PM
🌍 A 26-member team has been awarded the 2025 Gordon Bell Prize for Climate Modelling for their project, “Computing the Full Earth System at 1 km Resolution.”
Congratulations to this year’s outstanding team!
buff.ly/GKDYnH3
Congratulations to this year’s outstanding team!
buff.ly/GKDYnH3
Rajeev Thakur is kicking off our Advanced MPI tutorial at #SC25. It's always an honor to teach this long standing tutorial with esteemed colleagues including Bill Gropp and Pavan Balaji. Great attendance 👌, still some seats in 122.
We're looking forward to a productive session.
We're looking forward to a productive session.
November 17, 2025 at 2:36 PM
Rajeev Thakur is kicking off our Advanced MPI tutorial at #SC25. It's always an honor to teach this long standing tutorial with esteemed colleagues including Bill Gropp and Pavan Balaji. Great attendance 👌, still some seats in 122.
We're looking forward to a productive session.
We're looking forward to a productive session.
Kurt Ferreira opens the 14th addition of our ROSS workshop at #SC25! Supercomputer operating systems and middleware going strong! Packed room as always 😀.
Starting with an invited talk by NVIDIA's Jeff Hammond on communication systems.
Trivia: he probably traveled furthest ✈️
Starting with an invited talk by NVIDIA's Jeff Hammond on communication systems.
Trivia: he probably traveled furthest ✈️
November 16, 2025 at 8:15 PM
Kurt Ferreira opens the 14th addition of our ROSS workshop at #SC25! Supercomputer operating systems and middleware going strong! Packed room as always 😀.
Starting with an invited talk by NVIDIA's Jeff Hammond on communication systems.
Trivia: he probably traveled furthest ✈️
Starting with an invited talk by NVIDIA's Jeff Hammond on communication systems.
Trivia: he probably traveled furthest ✈️
Arrived at #SC25 and just moved into the SPCL den 🏠 - our homebase for the whole on-site team this week. Readying ourselves for a crazy time - my three talks for tomorrow should mainly be set 😀.
Looking forward to seeing all of you in person - for those who I have not already met 🤝.
Looking forward to seeing all of you in person - for those who I have not already met 🤝.
November 16, 2025 at 3:17 PM
Arrived at #SC25 and just moved into the SPCL den 🏠 - our homebase for the whole on-site team this week. Readying ourselves for a crazy time - my three talks for tomorrow should mainly be set 😀.
Looking forward to seeing all of you in person - for those who I have not already met 🤝.
Looking forward to seeing all of you in person - for those who I have not already met 🤝.
Very nice overview of the emerging UALink standard with nice features such as splitting packets in switches, in-network computing, high energy efficiency, and lowest silicon overhead: buff.ly/AgLvC1g
I'll be joining a panel at SC25 contrasting UALink and UEC next Wed: buff.ly/BeCMFcL Join us there
I'll be joining a panel at SC25 contrasting UALink and UEC next Wed: buff.ly/BeCMFcL Join us there
Introducing the UALink 200G 1.0 Specification Webinar
The Ultra Accelerator Link™ (UALink™) Consortium is an open industry standard group dedicated to advancing the UALink specification. The Consortium recently released the UALink 200G 1.0…
buff.ly
November 14, 2025 at 6:00 AM
Very nice overview of the emerging UALink standard with nice features such as splitting packets in switches, in-network computing, high energy efficiency, and lowest silicon overhead: buff.ly/AgLvC1g
I'll be joining a panel at SC25 contrasting UALink and UEC next Wed: buff.ly/BeCMFcL Join us there
I'll be joining a panel at SC25 contrasting UALink and UEC next Wed: buff.ly/BeCMFcL Join us there
Ram Velega (Broadcom) at OCP "The things that have confined us today that the scale-up domain is less than 100 are going to be changed very soon" (7:55 in buff.ly/9ioPH73)
I see this as a call to action for systems and software research.
UE 1.0 techniques such as LLR and CBFC play a crucial role!
I see this as a call to action for systems and software research.
UE 1.0 techniques such as LLR and CBFC play a crucial role!
2025 OCP APAC Summit Keynote - Broadcom
Ram Velaga (Broadcom)
Scale-up for AI: Balancing Compute, Memory and Networking
AI models are growing at an unprecedented pace, driving exponential increases in compute demands. As a single GPU…
buff.ly
November 12, 2025 at 6:00 AM
Ram Velega (Broadcom) at OCP "The things that have confined us today that the scale-up domain is less than 100 are going to be changed very soon" (7:55 in buff.ly/9ioPH73)
I see this as a call to action for systems and software research.
UE 1.0 techniques such as LLR and CBFC play a crucial role!
I see this as a call to action for systems and software research.
UE 1.0 techniques such as LLR and CBFC play a crucial role!
Ilya Sutskever "Three lines of math can prove all of supervised learning. That's nice" (4:33)
"I have not seen an exposition of unsupervised learning that I found satisfying" (7:50)
Its optimization objective has little relation to the actual objective you care about
Watch: buff.ly/Bnvazym
"I have not seen an exposition of unsupervised learning that I found satisfying" (7:50)
Its optimization objective has little relation to the actual objective you care about
Watch: buff.ly/Bnvazym
November 10, 2025 at 6:00 AM
Ilya Sutskever "Three lines of math can prove all of supervised learning. That's nice" (4:33)
"I have not seen an exposition of unsupervised learning that I found satisfying" (7:50)
Its optimization objective has little relation to the actual objective you care about
Watch: buff.ly/Bnvazym
"I have not seen an exposition of unsupervised learning that I found satisfying" (7:50)
Its optimization objective has little relation to the actual objective you care about
Watch: buff.ly/Bnvazym
Can we build an #AI #Climate Scientist? Asked at the ADIA Lab Symposium in Abu Dhabi last week - now online at buff.ly/6igSeyg :-).
Much work to be done - this is outlining some directions of indicative results with a lot of potential to accelerate AI for Science.
Much work to be done - this is outlining some directions of indicative results with a lot of potential to accelerate AI for Science.
November 9, 2025 at 9:24 AM
Can we build an #AI #Climate Scientist? Asked at the ADIA Lab Symposium in Abu Dhabi last week - now online at buff.ly/6igSeyg :-).
Much work to be done - this is outlining some directions of indicative results with a lot of potential to accelerate AI for Science.
Much work to be done - this is outlining some directions of indicative results with a lot of potential to accelerate AI for Science.
Collaborator and friend Dan Alistarh talks at ETH about using the new NvFP4 and MXFP4 block formats for inference.
Some going from "terrible" accuracy to acceptable using micro rotations to smoothen outliers in blocks.
arxiv.org/abs/2509.23202
Great collaboration and cool stuff
Some going from "terrible" accuracy to acceptable using micro rotations to smoothen outliers in blocks.
arxiv.org/abs/2509.23202
Great collaboration and cool stuff
November 5, 2025 at 8:32 AM
Collaborator and friend Dan Alistarh talks at ETH about using the new NvFP4 and MXFP4 block formats for inference.
Some going from "terrible" accuracy to acceptable using micro rotations to smoothen outliers in blocks.
arxiv.org/abs/2509.23202
Great collaboration and cool stuff
Some going from "terrible" accuracy to acceptable using micro rotations to smoothen outliers in blocks.
arxiv.org/abs/2509.23202
Great collaboration and cool stuff
Keren Bergman at the 2nd EFCL Workshop: "Huawei combines 3x less performant GPUs with a photonic scale-up network to build higher performance PODs than Nvidia."
Nvidia moving towards CPO 😀. Optics everywhere in scale-out.
Nice overview of optical networking at all distances.
Nvidia moving towards CPO 😀. Optics everywhere in scale-out.
Nice overview of optical networking at all distances.
November 4, 2025 at 12:49 PM
Keren Bergman at the 2nd EFCL Workshop: "Huawei combines 3x less performant GPUs with a photonic scale-up network to build higher performance PODs than Nvidia."
Nvidia moving towards CPO 😀. Optics everywhere in scale-out.
Nice overview of optical networking at all distances.
Nvidia moving towards CPO 😀. Optics everywhere in scale-out.
Nice overview of optical networking at all distances.
I was very honored to meet Carnegie Mellon University's President, Dean of the School of CS, and its famous founder Raj Reddy to present a lecture named after him.
I tremendously enjoyed speaking with young students and faculty and the evening with CMU's leadership. Thanks for the invitation Raj!
I tremendously enjoyed speaking with young students and faculty and the evening with CMU's leadership. Thanks for the invitation Raj!
November 3, 2025 at 6:00 AM
I was very honored to meet Carnegie Mellon University's President, Dean of the School of CS, and its famous founder Raj Reddy to present a lecture named after him.
I tremendously enjoyed speaking with young students and faculty and the evening with CMU's leadership. Thanks for the invitation Raj!
I tremendously enjoyed speaking with young students and faculty and the evening with CMU's leadership. Thanks for the invitation Raj!
MIT's Sandy Pentland at ADIA Lab symposium: "Modern companies need to structure their incentives to coordiate teams instead of using strict and siloed hierarchies." Enable team leaders to do "what they think is right" instead of inefficient political discussions with leadership.
October 30, 2025 at 6:01 AM
MIT's Sandy Pentland at ADIA Lab symposium: "Modern companies need to structure their incentives to coordiate teams instead of using strict and siloed hierarchies." Enable team leaders to do "what they think is right" instead of inefficient political discussions with leadership.
Reposted by Torsten Hoefler 🇨🇭
🎉 Uno accepted to SC25! Unified congestion control + reliable connectivity for intra- & inter-DC traffic to enable inter-DC AI training.
📄Paper: arxiv.org/abs/2510.15802
💻Code: github.com/spcl/Uno_SC25
🤝Collaboration with Microsoft
#SC25 #AI #SPCL @thoefler.bsky.social @csateth.bsky.social
📄Paper: arxiv.org/abs/2510.15802
💻Code: github.com/spcl/Uno_SC25
🤝Collaboration with Microsoft
#SC25 #AI #SPCL @thoefler.bsky.social @csateth.bsky.social
October 29, 2025 at 9:25 AM
🎉 Uno accepted to SC25! Unified congestion control + reliable connectivity for intra- & inter-DC traffic to enable inter-DC AI training.
📄Paper: arxiv.org/abs/2510.15802
💻Code: github.com/spcl/Uno_SC25
🤝Collaboration with Microsoft
#SC25 #AI #SPCL @thoefler.bsky.social @csateth.bsky.social
📄Paper: arxiv.org/abs/2510.15802
💻Code: github.com/spcl/Uno_SC25
🤝Collaboration with Microsoft
#SC25 #AI #SPCL @thoefler.bsky.social @csateth.bsky.social
One highlight of the ADIA Lab Symposium was Nobel Laureate Chu's talk towards Net-Zero emissions.
"China follows the US textbook from 100 years ago, when the US took products inventend in Europe, such as cars, industrialized and improved them. Now China takes things invented in the west..."
"China follows the US textbook from 100 years ago, when the US took products inventend in Europe, such as cars, industrialized and improved them. Now China takes things invented in the west..."
October 28, 2025 at 5:53 AM
One highlight of the ADIA Lab Symposium was Nobel Laureate Chu's talk towards Net-Zero emissions.
"China follows the US textbook from 100 years ago, when the US took products inventend in Europe, such as cars, industrialized and improved them. Now China takes things invented in the west..."
"China follows the US textbook from 100 years ago, when the US took products inventend in Europe, such as cars, industrialized and improved them. Now China takes things invented in the west..."
Just arrived at the ADIA Lab symposium in Abu Dhabi to listen to Horst Simon's introduction and Bjorn Stevens' keynote on how to compute the future climate! Featuring our Gordon Bell finalists 🌍🚀
Looking forward to speculating about how to create an #AI climate scientist 😀.
Looking forward to speculating about how to create an #AI climate scientist 😀.
October 27, 2025 at 6:25 AM
Just arrived at the ADIA Lab symposium in Abu Dhabi to listen to Horst Simon's introduction and Bjorn Stevens' keynote on how to compute the future climate! Featuring our Gordon Bell finalists 🌍🚀
Looking forward to speculating about how to create an #AI climate scientist 😀.
Looking forward to speculating about how to create an #AI climate scientist 😀.
Microsoft's Ultra Ethernet tutorial is now available on youtube 🎥!
Saurabh Dighe gives a brilliant motivation for Microsoft's "AI first datacenters" 🤖 followed by Abdul Kabbani and myself explaining the technical details and innovations of Ultra Ethernet supporting this goal 🫡.
buff.ly/IPN46ZR
Saurabh Dighe gives a brilliant motivation for Microsoft's "AI first datacenters" 🤖 followed by Abdul Kabbani and myself explaining the technical details and innovations of Ultra Ethernet supporting this goal 🫡.
buff.ly/IPN46ZR
October 20, 2025 at 5:00 AM
Microsoft's Ultra Ethernet tutorial is now available on youtube 🎥!
Saurabh Dighe gives a brilliant motivation for Microsoft's "AI first datacenters" 🤖 followed by Abdul Kabbani and myself explaining the technical details and innovations of Ultra Ethernet supporting this goal 🫡.
buff.ly/IPN46ZR
Saurabh Dighe gives a brilliant motivation for Microsoft's "AI first datacenters" 🤖 followed by Abdul Kabbani and myself explaining the technical details and innovations of Ultra Ethernet supporting this goal 🫡.
buff.ly/IPN46ZR
Reposted by Torsten Hoefler 🇨🇭
Next was an intriguing talk by @thoefler.bsky.social on computational architectures for more efficiently training large models at @scsatcmu.bsky.social www.youtube.com/watch?v=LnXp... (4/8)
The 2025 Raj Reddy Artificial Intelligence Lecture
YouTube video by CMU School of Computer Science
www.youtube.com
October 16, 2025 at 3:07 AM
Next was an intriguing talk by @thoefler.bsky.social on computational architectures for more efficiently training large models at @scsatcmu.bsky.social www.youtube.com/watch?v=LnXp... (4/8)
I'm excited to discuss whether we can build an "AI Climate Scientist" in my talk at the ADIA Lab Symposium 2025 🌎
Join us in Abu Dhabi or online from October 27–29!
Register here: buff.ly/gCP2K1z
#ADIALabSymposium2025
Join us in Abu Dhabi or online from October 27–29!
Register here: buff.ly/gCP2K1z
#ADIALabSymposium2025
October 15, 2025 at 10:47 AM
I'm excited to discuss whether we can build an "AI Climate Scientist" in my talk at the ADIA Lab Symposium 2025 🌎
Join us in Abu Dhabi or online from October 27–29!
Register here: buff.ly/gCP2K1z
#ADIALabSymposium2025
Join us in Abu Dhabi or online from October 27–29!
Register here: buff.ly/gCP2K1z
#ADIALabSymposium2025
I was shocked to see the first two people in my "masterclass" 🎓 on #AI networking with Ultra Ethernet at #HLF25: David Patterson and Bob Metcalfe 😅! Both Turing award winners - Bob being one of the inventors of Ethernet 🥹. Was great fun also with many enthusiastic students and great discussions 🚀.
October 13, 2025 at 5:00 AM