It was wonderful to continue our personal and professional exchange outside the walls of the department and with other colleagues! Thanks for stopping by :)

Then at our weekly @gronlp.bsky.social Reading Group she talked about “𝐖𝐡𝐚𝐭 𝐝𝐨 𝐬𝐞𝐥𝐟-𝐬𝐮𝐩𝐞𝐫𝐯𝐢𝐬𝐞𝐝 𝐬𝐩𝐞𝐞𝐜𝐡 𝐦𝐨𝐝𝐞𝐥𝐬 𝐥𝐞𝐚𝐫𝐧 𝐚𝐛𝐨𝐮𝐭 𝐃𝐮𝐭𝐜𝐡?”, which is also part of an ongoing research project. 📊

Marianne presented her work on “𝐇𝐮𝐦𝐚𝐧-𝐥𝐢𝐤𝐞 𝐥𝐢𝐧𝐠𝐮𝐢𝐬𝐭𝐢𝐜 𝐛𝐢𝐚𝐬𝐞𝐬 𝐢𝐧 𝐧𝐞𝐮𝐫𝐚𝐥 𝐬𝐩𝐞𝐞𝐜𝐡 𝐦𝐨𝐝𝐞𝐥𝐬” at the monthly edition of the Linguistic Lunch I organize in our CLCG department.

...ii) a direct quality reward from a teacher model, and iii) a reward based on the log probabilities of a teacher model (and its dialogue continuations). While these rewards did not improve our models performance, two different DPO approaches did!

Don’t hesitate to reach out with any questions or doubts :)
Every contribution is more than welcome! 🙏😍

These are living datasets that researchers around the world can enrich as new resources become available
On the website of the project babylm.github.io/babybabellm/
you can find information about the dataset creation pipeline, training, and evaluation.

This was made possible thanks to the dedication of each language-specific expert and the coordination of senior researchers, @jumelet.bsky.social and @lchoshen.bsky.social among others.

This was made possible thanks to the dedication of each language-specific expert and the coordination of senior researchers, in particular @jumelet.bsky.social ! 🤩

Thanks to the entire CIRM team and the @ilcb.bsky.social for organizing this truly special week !