Whether through multi-task learning, auxiliary objectives, or simply smarter input design, giving models context unlocks generalization, robustness, and sometimes surprising insights.

It’s a good reminder: The best models don’t just predict, they understand

Auxiliary Tasks: When training for sentiment analysis, add an auxiliary task like predicting part-of-speech tags. A better understanding of grammar leads to a better understanding of sentiment.

Additional Contextual Data e.g. search queries in recommendations models: A user's search history is pure gold. A streaming service that sees you're searching for "Oscar-winning movies" can offer far more relevant suggestions than one relying on watch history alone.

Multi-Objective Training: Don't just predict customer purchase; also predict the likelihood of a return and a positive review. This creates a more holistic and useful e-commerce model.

Link: januverma.substack.com/p/the-protei...

Covers the significance of Anfinsen’s experiment, the role of the CASP competition, and why protein structure prediction was considered an AI-complete problem. This sets the stage for understanding how AlphaFold-2 achieved its breakthrough.

Part III involves using frontier models to generate (synthetic) ‘reasoning’ for user engagement based on past interactions and then use the reasoning-augmented data to SFT Qwen 1.5B model. Comparable or better results with just 10% of the interaction open.substack.com/pub/januverma/…

Part II of my explorations with LLMs for Recommendation tasks involves experimenting with base models of varying sizes from 0.5B to 14B params(Qwen 2.5 Series) and incorporating user attributes.
januverma.substack.com/p/large-language-models-for-recommender-35c

First experiment is on building a proof of concept for LLM recommender by supervised fine-tuning (SFT) a small-scale LLM (Llama 1B). januverma.substack.com/p/large-lang...

Or they are too narcissistic to even notice the work/life of others. I feel there could be a coping mechanism to make their view quite myopic - ignorance is a bliss, I guess.

Can’t wait for new stuff in the RLHF book. Part of my holidays reading plan.

And a never ending pit. Where do you stop with prompt refinement and based on what criteria is surely messy.