Lightnews — Scholar-powered news

Ahmad Beirami @abeirami.bsky.social · 13d

The math that LLMs can do today is novel enough to be considered publishable, but it's not the kind of math that would be consequential.

4

Ahmad Beirami @abeirami.bsky.social · 17d

My thoughts on the broken state of AI conference reviewing.

www.linkedin.com/feed/update/...

My thoughts on the broken state of AI conference reviewing: Years ago, when I was in graduate school and a postdoc in Information Theory, I always felt fortunate to be invited to review for IEEE… | A...

My thoughts on the broken state of AI conference reviewing: Years ago, when I was in graduate school and a postdoc in Information Theory, I always felt fortunate to be invited to review for IEEE Tran...

www.linkedin.com

7

Ahmad Beirami @abeirami.bsky.social · 27d

Let's regress from here to AGI!

nagpalchirag.bsky.social @nagpalchirag.bsky.social · 27d

Kernel regression is the only way to achieve AGI

3

Ahmad Beirami @abeirami.bsky.social · 28d

This is the conclusion slide of a talk I gave more than a year ago on RL/Alignment! It still holds true today.

Slide titled “Takeaways (alignment recipe).”

Step 1: Perform Best-of-n and make sure it works as desired.
– Inspect a few responses and verify the reward-induced ranking makes sense.
– Best-of-n gives the best trade-offs; if it doesn’t work, no fancy method will.
– You can debug best-of-n much faster.

Step 2: Only then train your favorite alignment method.
– Track KL(π‖p) throughout training:
• KL > 100: results are unlikely to be useful.
• KL > 15: inspect outcomes for reward hacking.
• KL < 8: you are probably OK.

Bottom banner in a black box repeats “(1) Look at your data! (2) Look at your data! (3) Look at your data!” in blue, green, and red.

3

Ahmad Beirami @abeirami.bsky.social · 29d

This also applies to telling your story (e.g., in a CV, bio, interview, etc).

Focus on what you have accomplished and what you are excited about doing next; not just where you did it!

Ahmad Beirami @abeirami.bsky.social · 29d

I occasionally get messages asking how to follow my path and get into Meta, DeepMind, or similar places. That is the wrong question. Do not focus on the brand! Focus on what you want to work on, then find the opportunity that fits your goals best.

3

Ahmad Beirami @abeirami.bsky.social · 29d

Haha. the content was:

If a paper is great, the credit goes to the first author.

If a paper has any flaws, the responsibility falls on the last author.

1 3

Reposted by Ahmad Beirami

Shubhendu Trivedi @shubhendu.bsky.social · Sep 7

The actual unpopular opinion is that the notion of senior and junior authors should be abolished. It has completely diluted the notion of scientific authorship and created this entire industry of free-riding, head-in-the-clouds, incompetent PIs/managers. List down exact contributions instead. [+]

Ahmad Beirami @abeirami.bsky.social · Sep 6

Unpopular opinion:
When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar.

2 2 10

Ahmad Beirami @abeirami.bsky.social · 29d

Glad you asked.
bsky.app/profile/abei...

Ahmad Beirami @abeirami.bsky.social · Sep 6

Corollary: If you lack bandwidth or expertise to act as the verifier, then you shouldn't sign up to be the senior author of a paper!

Ahmad Beirami @abeirami.bsky.social · Sep 6

Unpopular opinion:
When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar.

1 1

Ahmad Beirami @abeirami.bsky.social · 29d

I occasionally get messages asking how to follow my path and get into Meta, DeepMind, or similar places. That is the wrong question. Do not focus on the brand! Focus on what you want to work on, then find the opportunity that fits your goals best.

2

Reposted by Ahmad Beirami

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Sep 7

Related to this if a paper turns out to have a major error in it, you’re supposed to throw yourself under the bus not your students.

Ahmad Beirami @abeirami.bsky.social · Sep 6

Unpopular opinion:
When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar.

2 1 24

Ahmad Beirami @abeirami.bsky.social · Sep 7

I think for every deliverable, there has to be one person who is responsible (gets it done) and one person who is accountable (makes sure it's done correctly).

Middle authors can be responsible or accountable for a subset of tasks.

1 1 4

Ahmad Beirami @abeirami.bsky.social · Sep 7

proposed contribution breakdown by Atlas Wang makes a lot of sense imo:
www.linkedin.com/feed/update/...

Corollary: If you lack bandwidth or expertise to act as the verifier, then you shouldn't sign up to be the senior author of a paper! | Ahmad Beirami

Corollary: If you lack bandwidth or expertise to act as the verifier, then you shouldn't sign up to be the senior author of a paper!

www.linkedin.com

2

Ahmad Beirami @abeirami.bsky.social · Sep 6

Not really. I've been saying variants of the same thing for a long time:

x.com/abeirami/sta...

Ahmad Beirami on X: "Good time to remind ourselves that: If a paper is great, the credit goes to the first author. If a paper has any flaws, the responsibility falls on the last author." / X

Good time to remind ourselves that: If a paper is great, the credit goes to the first author. If a paper has any flaws, the responsibility falls on the last author.

x.com

2

Ahmad Beirami @abeirami.bsky.social · Sep 6

Corollary: If you lack bandwidth or expertise to act as the verifier, then you shouldn't sign up to be the senior author of a paper!

Ahmad Beirami @abeirami.bsky.social · Sep 6

Unpopular opinion:
When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar.

5

Ahmad Beirami @abeirami.bsky.social · Sep 6

The junior author is the generator. The senior author is the verifier. The verifier should teach/distill some checks to the generator, but the verifier keeps final responsibility. If a wrong claim gets out, it is on the verifier!

2 3

Ahmad Beirami @abeirami.bsky.social · Sep 6

Unpopular opinion:
When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar.

4 3 19

Ahmad Beirami @abeirami.bsky.social · Sep 6

This is the recipe for many provable claims:

Make enough assumptions and narrow down the claim, then prove a narrow result with caveats. Present it as broad, hide the caveats, and declare “XYZ is provable!”

3

Ahmad Beirami @abeirami.bsky.social · Sep 5

Today, Industry research is focused on short term (3-6months) bets. Academics have an opportunity to balance their portfolio with medium term (1-2 years) and long term (5-10 years) bets. Putting all academic efforts in short-term basket is suboptimal!

7

Ahmad Beirami @abeirami.bsky.social · Sep 2

When I worked in corporate, I was often first in the office because that routine worked for me. It was a personal preference, not a benchmark for anyone else.

We should not judge commitment by hours, especially in research. We should look for thoughtful work and steady progress.

3

Ahmad Beirami @abeirami.bsky.social · Sep 2

True, this sounds obvious but it is more common than we'd hope, unfortunately.

Ahmad Beirami @abeirami.bsky.social · Sep 1

Common mistake in LLM prompting projects: jumping into full-scale pipelines (datasets and inference) without testing feasibility. Iterating at scale is expensive and time-consuming.

Start with ONE example to validate the hypothesis, verify context, debug the design, then scale.

1 1 7

Ahmad Beirami @abeirami.bsky.social · Aug 31

I have to admit that I embarrassingly didn't know about this history. Nice reading material for the long weekend, thank you :)

1 1

Ahmad Beirami @abeirami.bsky.social · Aug 31

But I don't disagree that maybe it is already better than an average human, because we humans are pretty bad at this too.

1 1

Ahmad Beirami @abeirami.bsky.social · Aug 31

And audience modeling is a hard problem and won't be solved unless we effectively solve memory.

On the clarity point: I think we are still not there yet on info seeking queries, e.g., GPT5 Thinking returns a lot of info (high recall) but is unable to put the info in a clear and logical structure.

1 1

Ahmad Beirami @abeirami.bsky.social · Aug 31

The open question is how to choose annotators who can teach that skill. We need people who understand the point in the first place, and who can tell when a point is vague, what context is missing, and how to fix it.

1 1