Lightnews — Scholar-powered news

GitHub - naver/dune: Code repository for "DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers"

2 6

Reposted by Bart Duisterhof

Philippe Weinzaepfel @weinzaepfelp.bsky.social · Jun 6

Thanks Christian for the advertisement.

github link: github.com/naver/dune

Code repository for "DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers" - naver/dune

GitHub - Duisterhof/rayst3r

3 15

Bart Duisterhof @bardienus.bsky.social · Jun 6

🔗 Project Website: rayst3r.github.io
📄 arXiv: arxiv.org/abs/2506.05285
🚀 Code: github.com/Duisterhof/...
🤗 HF Demo: Coming (very) soon!

@CMU_Robotics @SCSatCMU @nvidia @NVIDIAAI @NVIDIARobotics

Contribute to Duisterhof/rayst3r development by creating an account on GitHub.

2

Bart Duisterhof @bardienus.bsky.social · Jun 6

Big thanks to the awesome contributors to this project!👏 Jan Oberst, @bowenwen_me, @BirchfieldStan, @RamananDeva and @jeff_ichnowski. Also thanks to OctMAE author @s1wase, @nvidia for sponsoring compute 🖥️, and the scientists at @naverlabseurope for the inspiration! 🧗‍♂️

Bart Duisterhof @bardienus.bsky.social · Jun 6

We also study the impact of the confidence threshold on reconstruction quality. Our ablations suggest setting a higher confidence threshold improves accuracy, while limiting completeness and edge-bleeding. Users can tune the threshold for application-specific requirements 🎛️.

Bart Duisterhof @bardienus.bsky.social · Jun 6

We evaluate RaySt3R against the baselines on synthetic and real-world datasets. The results suggest RaySt3R achieves zero-shot generalization to the real world, and outperforms all baselines by up to 44% in 3D chamfer distance 🚀.

Bart Duisterhof @bardienus.bsky.social · Jun 6

We train RaySt3R by curating a new dataset, for a total of 12 million views 📷 with Objaverse and GSO objects. The ablations 🔍 suggest that more and more diverse data improves RaySt3R's performance. RaySt3R does not require GT meshes, paving the way for training on real-world data.

Bart Duisterhof @bardienus.bsky.social · Jun 6

💡 Our key insight is that 3D object shape completion can be recasted as a novel-view synthesis problem. RaySt3R takes a masked RGB-D image as input, and predicts depth maps and object masks for novel views. We query multiple views and merge the predictions into a consistent point cloud.

Bart Duisterhof @bardienus.bsky.social · Jun 6

We focus on multi-object 3D shape completion for robotics. Robots are commonly equipped with a RGB-D camera 📷, but their measurements are noisy and incomplete.

Using only DINOv2 features 🦖 as pretraining, we train a new model (RaySt3R) to produce accurate geometry.

Bart Duisterhof @bardienus.bsky.social · Jun 6

Imagine if robots could fill in the blanks in cluttered scenes.

✨ Enter RaySt3R: a single masked RGB-D image in, complete 3D out.
It infers depth, object masks, and confidence for novel views, and merges the predictions into a single point cloud. rayst3r.github.io

1 3 24

Bart Duisterhof @bardienus.bsky.social · Mar 26

Do you think Europe will take the opportunity? The Netherlands is even cutting research funds under the new administration... It feels like there are still significantly more opportunities in the US.

Two RI Ph.D. Students Receive Generative AI in Healthcare Fellowship - Robotics Institute Carnegie Mellon University

Bart Duisterhof @bardienus.bsky.social · Mar 26

Thanks Chris! This was a push with the entire dust3r team @naverlabseurope.bsky.social, congrats everyone!

8

Reposted by Bart Duisterhof

International Conference on 3D Vision @3dvconf.bsky.social · Mar 26

The Best Student Paper Award goes to MASt3R-SfM! #3DV2025

8 42

Reposted by Bart Duisterhof

Uksang Yoo @uksang.bsky.social · Mar 17

🎉Excited to share that our paper was a finalist for best paper at #HRI2025! We introduce MOE-Hair, a soft robot system for hair care 💇🏻💆🏼 that uses mechanical compliance and visual force sensing for safe, comfortable interaction. Check our work: moehair.github.io @cmurobotics.bsky.social 🧵1/7

1 5 10

Reposted by Bart Duisterhof

Zhenjun Zhao @ericzzj.bsky.social · Mar 4

MUSt3R: Multi-view Network for Stereo 3D Reconstruction

Yohann Cabon, Lucas Stoffl, Leonid Antsfeld, Gabriela Csurka, Boris Chidlovskii, Jerome Revaud, @vincentleroy.bsky.social

tl;dr: make DUSt3R symmetric and iterative+multi-layer memory mechanism->multi-view DUSt3R

arxiv.org/abs/2503.01661

1 4 25

Bart Duisterhof @bardienus.bsky.social · Jan 31

Great news, CMU's Center for Machine Learning and Health (CMLH) decided to fund another year of our research! If you're a PhD student at CMU, consider applying for the next iterations of the fellowship - the funding is generous and relatively unconstrained :)

CMU Robotics Institute @cmurobotics.bsky.social · Jan 30

Congrats to Ph.D. students Angela Chen and Bardienus (Bart) Duisterhof, who both received Generative AI in Healthcare fellowships from the Center for Machine Learning and Health!

Read about their research and the fellowship: www.ri.cmu.edu/two-ri-ph-d-...

The Center for Machine Learning and Health (CMLH) at Carnegie Mellon University recently announced the 2024 Generative AI in Healthcare Fellows. This round, two Robotics Institute Ph.D. students, Ange...

www.ri.cmu.edu

3

Bart Duisterhof @bardienus.bsky.social · Jan 13

😆

RI Seminar : Jeffrey Ichnowski : Learning for Dynamic Robot Manipulation of Deformable...

Bart Duisterhof @bardienus.bsky.social · Dec 5

Is the book just as good/better than the show for "The 3 body problem"?

2

Reposted by Bart Duisterhof

CMU Robotics Institute @cmurobotics.bsky.social · Nov 26

Watch Professor Jeff Ichnowski's RI seminar talk: "Learning for Dynamic Robot Manipulation of Deformable and Transparent Objects" 🦾🤖

@jeff-ichnowski.bsky.social closed out our Fall seminar series. Keep an eye out for the Spring schedule in the new year!

www.youtube.com/watch?v=DvvF...

YouTube video by CMU Robotics Institute

www.youtube.com

2 15

Reposted by Bart Duisterhof

Sarvesh Patil @nagababa.bsky.social · Nov 23

Intro Post
Hello World!
I'm a 2nd year Robotics PhD student at CMU, working on distributed dexterous manipulation, accessible soft robots and sensors, sample efficient robot learning, and causal inference.

Here are my cute robots:
PS: Videos are old and sped up. They move slower in real-world :3

3 15

Reposted by Bart Duisterhof

Kosta Derpanis @csprofkgd.bsky.social · Nov 19

My growing list of #computervision researchers on Bsky.

Missed you? Let me know.

go.bsky.app/M7HGC3Y

88 42 130

Bart Duisterhof @bardienus.bsky.social · Nov 22

My advisor @jeff-ichnowski.bsky.social! For example: github.com/BerkeleyAuto...

GitHub - BerkeleyAutomation/FogROS2: An Adaptive and Extensible Platform for Cloud and Fog Robotics Using ROS 2

An Adaptive and Extensible Platform for Cloud and Fog Robotics Using ROS 2 - BerkeleyAutomation/FogROS2

2

Bart Duisterhof @bardienus.bsky.social · Nov 22

For international students: renewing your visa asap might be a good idea.