Lightnews — Scholar-powered news

Reposted by Lerrel Pinto

Robot Talk @robottalkpod.bsky.social · May 21

How is AI helping robots to generalise their skills to unfamiliar environments? 🤖 🏠

In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.

Available wherever you get your podcasts: linktr.ee/robottalkpod

1 2

Lerrel Pinto @lerrelpinto.com · Apr 18

This project, which combines hardware design with learning-based controllers was a monumental effort led by @anyazorin.bsky.social and Irmak Guzey. More links and information about RUKA are below:

Website: ruka-hand.github.io
Assembly Instructions: ruka.gitbook.io/instructions

1

Lerrel Pinto @lerrelpinto.com · Apr 18

We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced!

The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.

1 7 29

Lerrel Pinto @lerrelpinto.com · Mar 29

This would be funny! 😂

Lerrel Pinto @lerrelpinto.com · Mar 28

When life gives you lemons, you pick them up.

(trained with robotutilitymodels.com)

1 4 15

Reposted by Lerrel Pinto

Robot Talk @robottalkpod.bsky.social · Mar 18

What would you love to know about #robot learning and decision making?

Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.

Send me your questions for Lerrel: robottalk.org/ask-a-question/

7 13

Lerrel Pinto @lerrelpinto.com · Mar 2

Is there a word for the feeling when you want to cheer for the other team?

1 5

Reposted by Lerrel Pinto

Physicians for a Healthy Democracy @physiciandemocracy.medsky.social · Mar 2

Times Square

220 3.2K 17K

Reposted by Lerrel Pinto

Dustin Moskovitz @moskov.goodventures.org · Feb 28

Slava Ukraini 🇺🇦

400 4.6K 35K

Lerrel Pinto @lerrelpinto.com · Feb 28

This project was an almost solo effort from @haldarsiddhant.bsky.social. And as always, this project is fully opensourced.

Project page: point-policy.github.io
Paper: arxiv.org/abs/2502.20391

Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation

point-policy.github.io

3

Lerrel Pinto @lerrelpinto.com · Feb 28

The overall algorithm is simple:
1. Extract key points from human videos.
2. Train a transformer policy to predict future robot key points.
3. Convert predicted key points to robot actions.

1 1

Lerrel Pinto @lerrelpinto.com · Feb 28

Point Policy uses sparse key points to represent both human demonstrators and robots, bridging the morphology gap. The scene is hence encoded through semantically meaningful key points from minimal human annotations.

1

Lerrel Pinto @lerrelpinto.com · Feb 28

The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!

1 5 20

Reposted by Lerrel Pinto

Rodney Brooks @rodneyabrooks.bsky.social · Feb 27

It finally dawned on me. I couldn't figure out what was going in Washington, plus all the hype about humanoids, including the new genre of "humanoid theater" videos, unmoored from reality. Then I realized, we must indeed be living in a simulation. Scripted by an LLM. Our world is a confabulation!

1 3 24

Reposted by Lerrel Pinto

Chris Paxton @cpaxton.bsky.social · Feb 26

This is important because the humble iPhone is one of the best accessories for embodied AI out there, if not actually the best. It's got a depth sensor, good camera, built-in internet, decent compute, and -- uniquely -- it has really good slam already built in.

3 4 15

Lerrel Pinto @lerrelpinto.com · Feb 26

It should be accessible in EU now!

1 1

Lerrel Pinto @lerrelpinto.com · Feb 26

AnySense is built to empower researchers with better tools for robotics. Try it out below.

Download on App store: apps.apple.com/us/app/anyse...
Open-source code on GitHub: github.com/NYU-robot-le...
Website: anysense.app

AnySense is led by @raunaqb.bsky.social with several from NYU.

‎AnySense

‎AnySense is an open-source iPhone app that enables multi-sensory data collection by integrating the iPhone’s sensory suite with external sensors via Bluetooth and wired interfaces, enabling both offl...

apps.apple.com

3

Lerrel Pinto @lerrelpinto.com · Feb 26

With this 'wild' robot data, data collected by AnySense can then be used to train multimodal policies! In the video above, we use the Robot Utility Models framework to train Visuo-Tactile policies for a whiteboard erasing task. You can use it for so much more though!

1 3

Lerrel Pinto @lerrelpinto.com · Feb 26

We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream:

1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors

2 10 36

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Feb 23

A useful “productivity” trick is to remind yourself that research should be fun and inspiring and if it’s not that something should change.

2 7 73

Reposted by Lerrel Pinto

George Takei @georgetakei.bsky.social · Feb 22

And I am an ally. If you are too, let the world know.

1.3K 17K 80K

Lerrel Pinto @lerrelpinto.com · Feb 20

Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)

1 6 37

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Feb 20

One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies

4 27 190

Lerrel Pinto @lerrelpinto.com · Feb 18

Thanks Tucker! The timing of this is great given the uncertainty with other funding mechanisms.

Lerrel Pinto @lerrelpinto.com · Feb 18

Thank you to @sloanfoundation.bsky.social for this generous award to our lab. Hopefully this will bring us closer to building truly general-purpose robots!

Alfred P. Sloan Foundation @sloanfoundation.bsky.social · Feb 18

🎉Congrats to the 126 early-career scientists who have been awarded a Sloan Research Fellowship this year! These exceptional scholars are drawn from 51 institutions across the US and Canada, and represent the next generation of groundbreaking researchers. sloan.org/fellowships/...

3 4 22