Lightnews — Scholar-powered news

ruggsea @ruggsea.bsky.social · 29d

A thing I've been working on for the past year: an LLM benchmark on the dreaded Italian medicine faculty entry exam. I will present it in two weeks at @ailc-nlp.bsky.social Clic-it in Cagliari!

arxiv cs.CL @arxiv-cs-cl.bsky.social · Sep 10

Ruggero Marino Lazzaroni, Alessandro Angioi, Michelangelo Puliga, Davide Sanna, Roberto Marras
MedBench-IT: A Comprehensive Benchmark for Evaluating Large Language Models on Italian Medical Entrance Examinations
https://arxiv.org/abs/2509.07135

1 2

ruggsea @ruggsea.bsky.social · 29d

I think that is a big conclusion to make based on loosely validated LLM simulations (still a big fan of their work but Generative simulation is a relatively young research field)

1 1

ruggsea @ruggsea.bsky.social · Sep 9

from this awesome blogpost: snats.xyz/pages/articl...

1

ruggsea @ruggsea.bsky.social · Jul 30

Life is going from one cool spot to train AI models to another

In this fancy Berlin library you can listen to vinyls while you do it!

2

ruggsea @ruggsea.bsky.social · Jul 28

Me looking very dumb while pointing at things at two academic events:

1. Pointing at a logits inside @repligate.bsky.social's loom for the "Braive New World" conference at @uni-graz.at
2. Pointing at my poster at @ic2s2.bsky.social last week on improving LLM agentic natural conversation synthesis

5

ruggsea @ruggsea.bsky.social · Jul 27

Only prisoners have time to read, and if you want to engage in a twenty-year long research project funded by the state, you will have to kill someone.

Sorry for the Fisher posting but it's so good

ruggsea @ruggsea.bsky.social · May 16

was true in 2012 and it is true now

1

ruggsea @ruggsea.bsky.social · Jul 12

criticallegalthinking.com/2013/05/14/a...

1

ruggsea @ruggsea.bsky.social · Jul 10

I look really bad/funny in this picture but I am glad I was given the opportunity to talk about LLM research to an audience of cool researchers!

Fun fact: I also held a hands-on session that involved playing around with Bluesky data!

Complexity Science Hub @csh.ac.at · Jul 10

Scientists know it's happening: #LLMs like #ChatGPT are quietly transforming #academia —helping write papers, draft grants & process data. CSH's @lespin.bsky.social organized a 3-day workshop to explore their ethical, practical use in research—from writing to coding to data annotation.

1 1

ruggsea @ruggsea.bsky.social · Jun 17

Really cannot hold comparison to metaculus comments, real alpha there

1 6

ruggsea @ruggsea.bsky.social · Jun 9

from karpathy.github.io/2019/04/25/r...

A Recipe for Training Neural Networks

Musings of a Computer Scientist.

karpathy.github.io

1

ruggsea @ruggsea.bsky.social · Jun 9

Karpathy on training Neural Networks: you should go slowly and be paranoid

me, while vibecoding torch code: what if i just increase the paranoid part?

2 3

ruggsea @ruggsea.bsky.social · Jun 3

study results partially depend on this hierarchical team coordination ability; I wonder how much the results would change if you gave the agents some less authority loving names

2

ruggsea @ruggsea.bsky.social · Jun 3

basically, Multi Agent collaborative task with three agents (with military names). The agents see their name in the prompt; the one called Alpha then appears to be biased to be the team leader

1 3

ruggsea @ruggsea.bsky.social · Jun 3

how not to name your ai agents

1 4

ruggsea @ruggsea.bsky.social · May 27

genz semantic embeddings engineering from my collegue

ruggsea @ruggsea.bsky.social · May 21

I am aware some of you have seen this on the other site where I stole it from, but it sounds so interesting not to share here

1

ruggsea @ruggsea.bsky.social · May 21

backpropagation was inspired by freud

1 2

ruggsea @ruggsea.bsky.social · May 20

“How about we pull over for a bit and get some rest?" - GPT4, when it's their turn to drive

ruggsea @ruggsea.bsky.social · May 18

I wish all textbooks were written like this one from @jurafsky.bsky.social‬

2

ruggsea @ruggsea.bsky.social · May 16

www.gonzocircus.com/blog/exclusi...

Exclusive essay: 'Time-wars: towards an alternative for the neo-capitalist era' - Gonzo (circus)

Is there really no alternative? Right-wing politicians focussed on austerity keep telling us. But Mark Fisher argues there is an alternative. If only we could see how valuable time is to all of us...

www.gonzocircus.com

ruggsea @ruggsea.bsky.social · May 16

Mark Fisher quoting Bifo in the Time-wars essay

1

ruggsea @ruggsea.bsky.social · May 16

was true in 2012 and it is true now

1

ruggsea @ruggsea.bsky.social · May 15

Stumbled upon new ammunition for my personal struggle against RHLF: the tendency of models trained this way to give correct sounding answers make us believe in wrong things more

arxiv.org/abs/2409.12822

Language Models Learn to Mislead Humans via RLHF

Language models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex. RLHF, the most popular post-training method, may exacerbate this problem: to achieve h...

arxiv.org

1 4

ruggsea @ruggsea.bsky.social · May 14

I think this is fine in the actual prevalent reasoning training paradigm (rule-based verifiable rewards — so "did we get to the objective right answer?")

About getting longer and longer meaningless reasoning, I always wonder if implementing thinking length penalties would work or not

1 1

ruggsea @ruggsea.bsky.social · May 14

I would doubt that, it would be cool if they did that (like fucked up but technically cool), but sometimes the model says it has been "instructed" to talk about that, smells like system prompt to me

1