I have been wanting to create a public reading list for a while

Starting today with ASHA for hyper parameter optimisation

blog.ml.cmu.edu/2018/12/12/m...

TL;DR
- For a compute budget, run hyper param configs in parallel
- Allocate more budget to promising runs, early stop others

CamemBERT 2.0: A Smarter French 🇫🇷 Language Model Aged to Perfection 👌

We release a much-needed update for the previous. SOTA French encoder LM.

We introduce two new models CamemBERTa-v2 and CamemBERT-v2, based on the DeBERTaV3 and RoBERTa recipe.

So what's new?

[1/8]

just learned the other day that it's possible to copy to your clipboard over SSH:

- it's a core neovim feature: neovim.io/doc/user/pro...
- @cyberdemon.org told me you can write a 1-line shell script that will take stdin and put it in your clipboard, script is here jvns.ca/til/vim-osc52/

1/ Introducing ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge 📚
@uwnlp.bsky.social & Ai2
With open models & 45M-paper datastores, it outperforms proprietary systems & match human experts.
Try out our demo!
openscholar.allen.ai

Gradient ascent on dual problems. A powerful idea that spawned tens of thousands of papers fits in a single blog post. www.argmin.net/p/dual-decom...

Talk: Speculations on Test-Time Scaling

A tutorial on the technical aspects behind OpenAI's o1 and open research questions in this space.

youtu.be/6PEJ96k1kiw
Slides+bibliography: github.com/srush/awesom...