Stone Tao
@stonet2000.bsky.social
PhDing @UCSanDiego @NVIDIA @hillbot_ai on scalable robot learning and embodied AI. Co-founded @LuxAIChallenge to build AI competitions. @NSF GRFP fellow
http://stoneztao.com
http://stoneztao.com
Big thanks to Linghao Chen (author of EasyHEC) for helping out. I simply made some examples and simplified some code for reproducibility/readability.
September 9, 2025 at 2:31 AM
Big thanks to Linghao Chen (author of EasyHEC) for helping out. I simply made some examples and simplified some code for reproducibility/readability.
Opensourcing a tool to calibrate camera extrinsics painlessly in a minute, no checkerboards! It's based on EasyHEC, using diff. rendering to optimize extrinsics given object meshes+poses. Crazy that even a piece of paper works too.
Code:
github.com/StoneT2000/s...
(paper example in next post)
Code:
github.com/StoneT2000/s...
(paper example in next post)
September 9, 2025 at 2:31 AM
Opensourcing a tool to calibrate camera extrinsics painlessly in a minute, no checkerboards! It's based on EasyHEC, using diff. rendering to optimize extrinsics given object meshes+poses. Crazy that even a piece of paper works too.
Code:
github.com/StoneT2000/s...
(paper example in next post)
Code:
github.com/StoneT2000/s...
(paper example in next post)
might be the first time I made it personally on the trending developers list on github! One of the contributing factors being the lerobot sim2real code
July 19, 2025 at 3:12 AM
might be the first time I made it personally on the trending developers list on github! One of the contributing factors being the lerobot sim2real code
Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
July 4, 2025 at 6:31 PM
Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Excited to announce that I will be interning at @nvidia research this summer on robotics/embodied AI! I’ll be in seattle for the summer, let me know if you want to meet up and chat! 🦾
June 28, 2025 at 5:37 PM
Excited to announce that I will be interning at @nvidia research this summer on robotics/embodied AI! I’ll be in seattle for the summer, let me know if you want to meet up and chat! 🦾
The sim2real demo had some mixed success, hampered primarily by the lighting conditions of the outdoors.
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
June 22, 2025 at 4:29 PM
The sim2real demo had some mixed success, hampered primarily by the lighting conditions of the outdoors.
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
I’ll be at #RSS2025 from June 21 to June 23!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
June 20, 2025 at 7:44 PM
I’ll be at #RSS2025 from June 21 to June 23!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
longer uncut video 17/18 successes
increased the spawn area size and tuned it to be less occluded. Works well!
increased the spawn area size and tuned it to be less occluded. Works well!
June 12, 2025 at 2:22 AM
longer uncut video 17/18 successes
increased the spawn area size and tuned it to be less occluded. Works well!
increased the spawn area size and tuned it to be less occluded. Works well!
sneak peek of what Xander and I will show next week at RSS 2025 for the maniskill demo session:
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
June 11, 2025 at 5:56 PM
sneak peek of what Xander and I will show next week at RSS 2025 for the maniskill demo session:
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Very good thread on how we got into this whole stem education / research cutting situation and its impacts by @davidbau.bsky.social
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
June 1, 2025 at 8:12 PM
Very good thread on how we got into this whole stem education / research cutting situation and its impacts by @davidbau.bsky.social
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Nice to see the use of ManiSkill3 in this work!
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
May 17, 2025 at 12:23 AM
Nice to see the use of ManiSkill3 in this work!
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
first thing I see on wechat channels when I get to China is a 1 hour uncut video of G1 speed walking on a track lmao
April 28, 2025 at 3:38 PM
first thing I see on wechat channels when I get to China is a 1 hour uncut video of G1 speed walking on a track lmao
talk by @ncklashansen.bsky.social at the world models workshop starting now!
April 28, 2025 at 12:59 AM
talk by @ncklashansen.bsky.social at the world models workshop starting now!
spotted at the robot learning workshop, 3 papers in a row all using maniskill! (and mujoco)
seems like this lab likes to use our parallel rendering features and prebuilt tasks
seems like this lab likes to use our parallel rendering features and prebuilt tasks
April 27, 2025 at 7:53 AM
spotted at the robot learning workshop, 3 papers in a row all using maniskill! (and mujoco)
seems like this lab likes to use our parallel rendering features and prebuilt tasks
seems like this lab likes to use our parallel rendering features and prebuilt tasks
I’ll be at the robot learning workshop today and giving an oral talk at 9:15 AM on ManiSkill3 in rooms Garnet 216/217 at ICLR. Come see the crazy things you can do with fast sim+rendering like fast visual RL and zero-shot RGB sim2real!
ManiSkill was also accepted at RSS! Updated paper coming soon
ManiSkill was also accepted at RSS! Updated paper coming soon
April 26, 2025 at 11:20 PM
I’ll be at the robot learning workshop today and giving an oral talk at 9:15 AM on ManiSkill3 in rooms Garnet 216/217 at ICLR. Come see the crazy things you can do with fast sim+rendering like fast visual RL and zero-shot RGB sim2real!
ManiSkill was also accepted at RSS! Updated paper coming soon
ManiSkill was also accepted at RSS! Updated paper coming soon
no more experiments today apparently
April 21, 2025 at 2:32 AM
no more experiments today apparently
strangely, in robotics there’s value to doing BC and RL. RL for generating super human data, BC (or vlas these days) to copy that super human data
it’s why i still do research on learning from demos, rl, sim. It’s the most underexplored area at the moment with the promise imo
it’s why i still do research on learning from demos, rl, sim. It’s the most underexplored area at the moment with the promise imo
April 20, 2025 at 8:09 PM
strangely, in robotics there’s value to doing BC and RL. RL for generating super human data, BC (or vlas these days) to copy that super human data
it’s why i still do research on learning from demos, rl, sim. It’s the most underexplored area at the moment with the promise imo
it’s why i still do research on learning from demos, rl, sim. It’s the most underexplored area at the moment with the promise imo
as someone who started in rl, I check if manipulation robots are modelled well and run fine in sim by checking if they can learn to pick up objects via RL. Slowly adding these test scripts for robot arms in ManiSkill, including SO100 here!
April 9, 2025 at 10:43 PM
as someone who started in rl, I check if manipulation robots are modelled well and run fine in sim by checking if they can learn to pick up objects via RL. Slowly adding these test scripts for robot arms in ManiSkill, including SO100 here!
so100 and koch arms are also now more or less officially supported with pretty good simulation too! Still working on making the sim2real code for so100/koch accessible so that is not released yet
April 5, 2025 at 8:49 PM
so100 and koch arms are also now more or less officially supported with pretty good simulation too! Still working on making the sim2real code for so100/koch accessible so that is not released yet
worked on a nice new gallery/docs for robots in maniskill/sapien (inspired by mujoco menagerie). Given all the github issues on robot modelling (it's hard!) it seems like we should just model robots for most people, this is an effort to organize it more maniskill.readthedocs.io/en/latest/ro...
April 5, 2025 at 8:45 PM
worked on a nice new gallery/docs for robots in maniskill/sapien (inspired by mujoco menagerie). Given all the github issues on robot modelling (it's hard!) it seems like we should just model robots for most people, this is an effort to organize it more maniskill.readthedocs.io/en/latest/ro...
some exciting news, ManiSkill/SAPIEN now has experimental support for MacOS for the CPU simulation and rendering. You can now do your local debugging/development on Mac. Example shown here is a Push-T policy trained on my 4090 running on my mac!
try now: maniskill.readthedocs.io/en/latest/us...
try now: maniskill.readthedocs.io/en/latest/us...
March 20, 2025 at 12:32 AM
some exciting news, ManiSkill/SAPIEN now has experimental support for MacOS for the CPU simulation and rendering. You can now do your local debugging/development on Mac. Example shown here is a Push-T policy trained on my 4090 running on my mac!
try now: maniskill.readthedocs.io/en/latest/us...
try now: maniskill.readthedocs.io/en/latest/us...
been learning warp recently primarily to add soft body sims back to ManiSkill. I’ve also been making some efforts to support mujoco/mjx as a physics backend earlier in the year. If these upgrades work then all of my efforts to learn warp / add mujoco will pay off a ton
March 18, 2025 at 9:09 PM
been learning warp recently primarily to add soft body sims back to ManiSkill. I’ve also been making some efforts to support mujoco/mjx as a physics backend earlier in the year. If these upgrades work then all of my efforts to learn warp / add mujoco will pay off a ton
Currently trying to do a kind of api setup where you can really write code like this to get easy sim2real real environment gym interfaces where the action/observation space is the same as the sim env, controllers are aligned etc. hoping to make sim2real more accessible
March 14, 2025 at 10:20 PM
Currently trying to do a kind of api setup where you can really write code like this to get easy sim2real real environment gym interfaces where the action/observation space is the same as the sim env, controllers are aligned etc. hoping to make sim2real more accessible
ManiSkill3 accepted as an oral presentation at the @iclr-conf.bsky.social robot learning workshop! See you all in Singapore🇸🇬, we have some cool demos demonstrating all the power of parallel rendering, heterogeneous sim and more! We will also update the arxiv with a v2 with some of these updates
March 1, 2025 at 2:25 AM
ManiSkill3 accepted as an oral presentation at the @iclr-conf.bsky.social robot learning workshop! See you all in Singapore🇸🇬, we have some cool demos demonstrating all the power of parallel rendering, heterogeneous sim and more! We will also update the arxiv with a v2 with some of these updates
(scripted, not autonomous), drawing of our projects logo. doubt any IL algorithm can solve this accurately.
February 26, 2025 at 12:20 AM
(scripted, not autonomous), drawing of our projects logo. doubt any IL algorithm can solve this accurately.