LightNews
lauraruis.bsky.social
Laura
@lauraruis.bsky.social
1.5K followers 110 following 56 posts
PhD supervised by Tim Rocktäschel and Ed Grefenstette, part time at Cohere. Language and LLMs. Spent time at FAIR, Google, and NYU (with Brenden Lake). She/her.
Posts Media Videos Starter Packs
Reposted by Laura
maxbartolo.bsky.social
Max Bartolo @maxbartolo.bsky.social · Mar 19
I really enjoyed my MLST chat with Tim @neuripsconf.bsky.social about the research we've been doing on reasoning, robustness and human feedback. If you have an hour to spare and are interested in AI robustness, it may be worth a listen 🎧

Check it out at youtu.be/DL7qwmWWk88?...
3 8
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 22
arxiv.org/abs/2009.14050
Understanding Human Intelligence through Human Limitations
Recent progress in artificial intelligence provides the opportunity to ask the question of what is unique about human intelligence, but with a new comparison class. I argue that we can understand huma...
arxiv.org
6
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 22
"Rather than being animals that *think*, we are *animals* that think"; the last sentence of Tom Griffiths's characterisation of human intelligence through limited time, compute, and communication hits different today than it did 4 years ago.
1 1 15
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 15
leave parrots alone!!
3
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 15
Sometimes o1's thinking time almost feels like a slight. o1 is like "oh I thought about this uninvolved question of yours for 7 seconds and here is my 20 page essay on it"
1 2 19
Reposted by Laura
lampinen.bsky.social
Andrew Lampinen @lampinen.bsky.social · Dec 10
What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7
The broader spectrum of in-context learning
The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervised few-shot learning...
arxiv.org
2 31 120
Reposted by Laura
dnnslmr.bsky.social
Dennis Ulmer @dnnslmr.bsky.social · Dec 10
Big congratulations to Dr. @jumelet.bsky.social for obtaining his PhD today and crafting a beautiful thesis full of original and insightful work!! 🎉 arxiv.org/pdf/2411.16433?
3 15
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 8
I'll be at NeurIPS tues-sun, send me a message if you'd like to chat!
7
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
Cool that those experiments changed your mind :) The referenced appendix was important to convince myself of what we eventually concluded (that the correlations indicate procedural knowledge). And thank you for the praise!! What kind of ideas did you get?
1
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
If that's how you define retrieval, then they are doing retrieval under your definition. The heavy lifting is of course done by the word "synthesize", how do they do that? That's what we're characterising in the paper
1 2
Reposted by Laura
stellaathena.bsky.social
Stella Biderman @stellaathena.bsky.social · Nov 30
This is an incredible paper that I've longed to do for a long time. However the engineering challenges were far too daunting, so my collaborators and I settled for indirect evidence for this hypothesis instead (or did other things).
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 20
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:

Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢

🧵⬇️
2 2 53
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
It should be much less computationally expensive to do for fine tuning data
2
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
To do this you can use Juhan's recent work on Source! arxiv.org/abs/2405.12186
Training Data Attribution via Approximate Unrolled Differentiation
Many training data attribution (TDA) methods aim to estimate how a model's behavior would change if one or more data points were removed from the training set. Methods based on implicit differentiatio...
arxiv.org
1 1
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
Just want to add to Stella's responses that the reason we went with procedural knowledge very much came from the correlation results; documents influence each query with the same underlying task similarly, even though the task is applied to different numbers for different queries.
1 2
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
Definitely! Next time will be Christmas so I presume that's not ideal, but I can reach out when I know the next time I will be there?
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Dec 1
What did you think
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 28
Yeah. I do think as you become more senior you become better at determining from the intro whether a paper is likely to be good or bad. The point is just that we should still actively keep an open mind when reading the rest of the paper
1
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
I learn so much from reviewing, it’s the papers I review that I keep coming back to for my own ideas and citations. They broaden and deepen my view on the field. Let’s give it the time it deserves.
1 2 21
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
It’s actually pretty cool if you as a reviewer get to make papers better by suggesting improvements. This cycle, I’ve given an 8 where all other reviewers gave a rejecting rating. Now, the scores are 8, 5, 8, 6, 8. Pretty exciting, if you ask me.
1 11
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
You don’t have to add these to the review (unless it’s TMLR). But hold yourself accountable when you are rejecting it. What could the authors do to lift your scores? If the answer is nothing, be sure to have a good reason for this. If there is something, tell the authors.
1 1 15
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
There’s an easy way to hold yourself accountable (thanks TMLR guidelines ✌️): "make a list of proposed adjustments to the submission, specifying for each whether they are critical to securing your recommendation for acceptance or would simply strengthen the work in your view."
2 1 19
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
The art of rebuttal is to learn how to stick firmly to the points you believe are important, while at the same time allowing yourself to be wrong. Admitting when you might be misunderstanding (after all, the authors probably spent about ~1000x more time thinking about it).
1 2 16
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
The hardest part is to keep an open mind all the way down 🐢. The rebuttal phase is the kicker. If you don’t spend enough time in this phase, just don’t sign up to be a reviewer, because it’s incredibly demoralising to people who work months to years on a submission.
2 1 17
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
I’ve heard people say they know whether they will accept or reject a paper after reading the abstract/intro. That’s great, but what is even greater is when you realise that is *just presentation*, and the soundness and contribution are *not* going to be determined by that part.
2 2 23
lauraruis.bsky.social
Laura @lauraruis.bsky.social · Nov 27
Reviewing requires constant questioning of the motive behind your responses, every step of the way. Which btw, according to chatty, will help you become a better scientist yourself.
1 16