I find making your agents safe is just as important as making them smart. 🔒

A good read for building secure AI!

arxiv.org/pdf/2503.18813

There will be one day ... in 🇺🇸 or 🇻🇳

Have you ever wondered what are the differences between 2 types of knowledge distillation (for LLMs) these days?

Link: developer.nvidia.com/blog/how-to-...

Thanks for sharing with us, Seb!

Surprisingly, when I used 'deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B', the output contained tokens/words introduced in the S1 paper. LOL! 🤣

My learning project: Implement a Reasoning Model with GRPO from scratch

Code: github.com/ducnh279/grp...

- Base model: Qwen2.5-1.5B
- Dataset: GSM8K (math)
- Reward functions: Format, Accuracy
- Objective: GRPO
- PEFT: LoRA

𝐂𝐨𝐬𝐢𝐧𝐞 𝐫𝐞𝐰𝐚𝐫𝐝 stabilizes training & incentivizes the model to generate responses that are both accurate and reasonably lengthy.

To prevent reward hacking, a repetition penalty is added to the cosine reward.

Paper: arxiv.org/pdf/2502.03373
Hackable Code: github.com/eddycmu/demy...

The best way to understand GRPO is to code it from scratch without HF Trainer. 😂

Profile Summary on Twitter is so interesting! Can't agree more on this.

🎁 Jason Wei: "agents"

🎁 Sebastian Raschka: "reasoning models"

Microsoft Phi team avoids overfitting on benchmarks using a decontamination algo, removing data overlap via 7&13-gram counts. I want to implement this for my future work, must research efficient way. Looking at scikit-learn's implementation CountVectorizer may be a good start!

The Phi-4 technical report is a bag of techniques for synthetic data in pre & post-training. The 𝗣𝗶𝘃𝗼𝘁𝗮𝗹 𝗧𝗼𝗸𝗲𝗻 𝗦𝗲𝗮𝗿𝗰𝗵 (PTS) method for generating DPO pairs is fascinating but computationally expensive for sampling completions to calculate p(success).

Interestingly, when a new model is released these days, checking whether it is compared to Qwen models can be a good indicator of its quality. 😂