Big thanks to Linghao Chen (author of EasyHEC) for helping out. I simply made some examples and simplified some code for reproducibility/readability.

Opensourcing a tool to calibrate camera extrinsics painlessly in a minute, no checkerboards! It's based on EasyHEC, using diff. rendering to optimize extrinsics given object meshes+poses. Crazy that even a piece of paper works too.

Code:
github.com/StoneT2000/s...
(paper example in next post)

Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works

Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...

The sim2real demo had some mixed success, hampered primarily by the lighting conditions of the outdoors.

At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle

longer uncut video 17/18 successes

increased the spawn area size and tuned it to be less occluded. Works well!

sneak peek of what Xander and I will show next week at RSS 2025 for the maniskill demo session:

Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100

~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!

I’ll be at the robot learning workshop today and giving an oral talk at 9:15 AM on ManiSkill3 in rooms Garnet 216/217 at ICLR. Come see the crazy things you can do with fast sim+rendering like fast visual RL and zero-shot RGB sim2real!

ManiSkill was also accepted at RSS! Updated paper coming soon

as someone who started in rl, I check if manipulation robots are modelled well and run fine in sim by checking if they can learn to pick up objects via RL. Slowly adding these test scripts for robot arms in ManiSkill, including SO100 here!

some exciting news, ManiSkill/SAPIEN now has experimental support for MacOS for the CPU simulation and rendering. You can now do your local debugging/development on Mac. Example shown here is a Push-T policy trained on my 4090 running on my mac!

try now: maniskill.readthedocs.io/en/latest/us...

(scripted, not autonomous), drawing of our projects logo. doubt any IL algorithm can solve this accurately.