Author | Lightnews

Akash Sharma @akashsharma02.bsky.social · May 27

This was an amazing collaboration at
@aiatmeta.bsky.social &
@cmurobotics.bsky.social : @carohiguera.bsky.social ,
@mukadammh, @francois_hogan
and many others
Code coming soon! Checkout our paper: arxiv.org/abs/2505.11420 &
Website: akashsharma02.github.io/sparsh-skin-... for more details
6/6

Self-supervised perception for tactile skin covered dexterous hands

We present Sparsh-skin, a pre-trained encoder for magnetic skin sensors distributed across the fingertips, phalanges, and palm of a dexterous robot hand. Magnetic tactile skins offer a flexible form f...

arxiv.org

Akash Sharma @akashsharma02.bsky.social · May 27

And we also show improvement in many tactile perception tasks such as force estimation, pose estimation and full-hand joystick state estimation.
5/6

1 1

Akash Sharma @akashsharma02.bsky.social · May 27

With this we see a 75% improvement in real-world tactile plug insertion over end-to-end using vision and tactile:
4/6

1

Akash Sharma @akashsharma02.bsky.social · May 27

We pretrain Sparsh-skin with 4 hours of unlabeled data via self-distillation, and make several changes to get highly performant reps:

Decorrelate signals by tokenizing 1s window of tactile data.
Condition the encoder on robot hand configurations via sensor positions as input
3/6

1

Akash Sharma @akashsharma02.bsky.social · May 27

Sparsh-skin is an approach to pretrain encoders for magnetic skin sensors on a dexterous robot hand.

It improves tactile tasks by over 56% in end-to-end methods and by over 41% in prior work.
It is trained via self-supervision for the Xela sensor, so no labeled data needed!
2/6

1 1

Akash Sharma @akashsharma02.bsky.social · May 27

Robots need touch for human-like hands to reach the goal of general manipulation. However, approaches today don’t use tactile sensing or use specific architectures per tactile task.

Can 1 model improve many tactile tasks?
🌟Introducing Sparsh-skin: tinyurl.com/y935wz5c

1/6

1 3

Akash Sharma @akashsharma02.bsky.social · May 12

I might sound salty, but I never got how 'outstanding reviewers' are chosen. Til then, part of the 'mediocre reviewers' gang it is 🤣

1 2

Akash Sharma @akashsharma02.bsky.social · May 11

Check out my work to know more:

1. Sparsh: tactile reps for vision based sensors sparsh-ssl.github.io

2. [Releasing soon] Sparsh-skin: Tactile reps for full hand magnetic skins

3. [Coming soon] Reps for multimodal touch fusing tactile-images, audio, motion and pressure

Sparsh | Self-supervised touch representations for vision-based tactile sensing

Sparsh: Self-supervised touch representations for vision-based tactile sensing

sparsh-ssl.github.io

Akash Sharma @akashsharma02.bsky.social · May 11

Last week I passed my thesis proposal, and I'm now officially a Ph.D. candidate!

I'm grateful to my committee, and everyone who supported me.

My proposed thesis "Self supervised perception for tactile dexterity" will explore ways to improve dexterous manipulation using tactile reps.

1 2

Akash Sharma @akashsharma02.bsky.social · May 11

We took a matter of fact approach for a robotics conference, and it backfired too.

Reposted by Akash Sharma

François Fleuret @francois.fleuret.org · Apr 28

I asked "on the other platform" what were the most important improvements to the original 2017 transformer.

That was quite popular and here is a synthesis of the responses:

4 43 210

Reposted by Akash Sharma

Kashyap Chitta @kashyap7x.bsky.social · Apr 28

⏰ Heads up! The deadline for two #CVPR2025 Autonomous Grand Challenge tracks is May 10th, 2025:

1️⃣ NAVSIM v2 Challenge: huggingface.co/spaces/AGC20...

2️⃣ World Model Challenge by 1X: huggingface.co/spaces/1x-te...

1 6 9

Reposted by Akash Sharma

Christian Wolf @chriswolfvision.bsky.social · Mar 27

I love situations like this: in the pre-deep era (and following classical learning theory), people would have stopped training the white model at the red arrow, as the validation error increases. But, no, the model first seems to learns unwanted short cuts (overfitting wildly) but finds a way out.

6 9 52

Akash Sharma @akashsharma02.bsky.social · Mar 26

Some pictures of the Pittsburgh spring to reduce the spiciness of the bsky feed!

a6700 w/ 17-70mm Tamron lens

2

Reposted by Akash Sharma

Terence Tao @teorth.bsky.social · Mar 24

A new #CosmicDistanceLadder post on why lunar and solar eclipses tend to come in pairs (for instance, the solar eclipse next week is paired with the lunar eclipse from last week). www.instagram.com/p/DHkS3EcA40L

1 5 29

Reposted by Akash Sharma

Robot Talk @robottalkpod.bsky.social · Mar 18

What would you love to know about #robot learning and decision making?

Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.

Send me your questions for Lerrel: robottalk.org/ask-a-question/

7 13

Reposted by Akash Sharma

Tengda Han @tengda.bsky.social · Mar 5

We are looking for a student researcher to work on video understanding plus 3D, in Google DeepMind London. DM/Email me or pass it to someone if you feel it may be a good fit!

6 20

Reposted by Akash Sharma

Katie Mack @astrokatie.com · Feb 26

The first measles death in the US in a decade -- the tragic, preventable death of a child whose parents chose not to protect them with vaccination -- should spark an immediate nation-wide campaign to ensure all children are protected against preventable diseases. Anything less is unconscionable.

1.1K 9.5K 39K

Akash Sharma @akashsharma02.bsky.social · Feb 26

Congrats Eric, really cool stuff!

1

Reposted by Akash Sharma

Tarasha Khurana @tarashakhurana.bsky.social · Feb 12

Gearing up for our workshop on 4D Vision at @CVPR this June! Check out our line up of speakers and submit your work by Mar 28. Spread the word!

Elliott / Shangzhe Wu @elliottwu.bsky.social · Feb 12

Really excited to put together this #CVPR2025 workshop on "4D Vision: Modeling the Dynamic World" -- one of the most fascinating areas in computer vision today!

We've invited incredible researchers who are leading fantastic work at various related fields.

4dvisionworkshop.github.io

1 5

Akash Sharma @akashsharma02.bsky.social · Feb 9

At first I thought poisson or log-normal, but after a bit of searching, maybe Gumbel distribution: en.m.wikipedia.org/wiki/Gumbel_...

Gumbel distribution - Wikipedia

en.m.wikipedia.org

1 2

Reposted by Akash Sharma

Trajan Hammonds @starlord37.bsky.social · Feb 8

Last night I found out that the NSF math postdoctoral fellowship I applied for is being deleted because it does not comply with Trump’s executive orders on DEI in the federal government. I’m going to answer some FAQs and share some thoughts about this ordeal in this thread 1/n

50 500 1.3K

Akash Sharma @akashsharma02.bsky.social · Feb 8

Now, see how life changes when you swap control and caps lock! 😆

2

Akash Sharma @akashsharma02.bsky.social · Feb 8

Seeing some of the early results from DexterityGen were definitely a wow moment for me!

It doesn't take a lot to realize all the new opportunities a strong teleop system like this enables! 🚀

X thread: x.com/zhaohengyin/...
Link: zhaohengyin.github.io/dexteritygen/

DexGen

zhaohengyin.github.io

1 2

Reposted by Akash Sharma

Peyman Milanfar @docmilanfar.bsky.social · Jan 28

Not one VC would ever fund a startup to do the kind of hardcore optimization work that DeepSeek did.

Every VC firm should be asking themselves why.

5 11 110