Synthetic data, specifically sim2real works like a charm

But robustness becomes an issue

In my experience adding a little bit of consistency regularization between simulated and real data helps a lot!

Almost all the ML enthusiasts focus a lot of architectures and fancy models but seldom touch the intricacies of optimization techniques and learning

As I’ve said many times, learning rate is the single most important hyperparameter that can help you so much in training.

I come across this graph every couple of months.

It's so beautiful.

The intuition and maths intersect perfectly.
The absolute minimum of a loss curve is often not the best so you choose a higher but flatter point.

First post on Bluesky. Let's see how it goes :')

Connect with me if you're interested in software dev, ML, AI, dev ops, mobile development, or anything in tech.