Nathan Lambert
@natolambert.bsky.social
A LLN - large language Nathan - (RL, RLHF, society, robotics), athlete, yogi, chef
Writes http://interconnects.ai
At Ai2 via HuggingFace, Berkeley, and normal places
Writes http://interconnects.ai
At Ai2 via HuggingFace, Berkeley, and normal places
Pinned
Nathan Lambert
@natolambert.bsky.social
· Apr 16
First draft online version of The RLHF Book is DONE. Recently I've been creating the advanced discussion chapters on everything from Constitutional AI to evaluation and character training, but I also sneak in consistent improvements to the RL specific chapter.
rlhfbook.com
rlhfbook.com
Opus?
Sorry, living under rocks today.
Sorry, living under rocks today.
November 24, 2025 at 10:41 PM
Opus?
Sorry, living under rocks today.
Sorry, living under rocks today.
Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai/p/latest-ope...
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai/p/latest-ope...
Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai
November 23, 2025 at 7:35 PM
Latest open artifacts (#16): Who's building models in the U.S., China's model release playbook, and a resurgence of truly open models
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai/p/latest-ope...
A month with SOTA releases with (truly) open model releases left and right.
www.interconnects.ai/p/latest-ope...
I asked (on ChinaTalk) the head of product at Z ai, one of the leading Chinese companies building open models, how long it takes them to get their model out the door once its done training. Incredible stuff:
"a few hours" and the model is on HuggingFace.
www.chinatalk.media/p/the-zai-pl...
"a few hours" and the model is on HuggingFace.
www.chinatalk.media/p/the-zai-pl...
November 21, 2025 at 5:05 PM
I asked (on ChinaTalk) the head of product at Z ai, one of the leading Chinese companies building open models, how long it takes them to get their model out the door once its done training. Incredible stuff:
"a few hours" and the model is on HuggingFace.
www.chinatalk.media/p/the-zai-pl...
"a few hours" and the model is on HuggingFace.
www.chinatalk.media/p/the-zai-pl...
People: "You must be so relaxed, proud, and happy that the model you worked on all year is out."
Me:
Me:
November 21, 2025 at 12:29 AM
People: "You must be so relaxed, proud, and happy that the model you worked on all year is out."
Me:
Me:
the Epstein files have been trending on HuggingFace.
> This dataset is provided for...
> Evaluating information retrieval and retrieval augmented generation (RAG) systems.
> It is not intended for: Fine-tuning language models.
??
> This dataset is provided for...
> Evaluating information retrieval and retrieval augmented generation (RAG) systems.
> It is not intended for: Fine-tuning language models.
??
November 20, 2025 at 9:49 PM
the Epstein files have been trending on HuggingFace.
> This dataset is provided for...
> Evaluating information retrieval and retrieval augmented generation (RAG) systems.
> It is not intended for: Fine-tuning language models.
??
> This dataset is provided for...
> Evaluating information retrieval and retrieval augmented generation (RAG) systems.
> It is not intended for: Fine-tuning language models.
??
Happy Olmo day to all who celebrate.
Sorry to all who delayed releases today to get out of our way.
We're hiring.
Sorry to all who delayed releases today to get out of our way.
We're hiring.
November 20, 2025 at 6:40 PM
Happy Olmo day to all who celebrate.
Sorry to all who delayed releases today to get out of our way.
We're hiring.
Sorry to all who delayed releases today to get out of our way.
We're hiring.
We present Olmo 3, our next family of fully open, leading language models.
This family of 7B and 32B models represents:
1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.
This family of 7B and 32B models represents:
1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.
November 20, 2025 at 2:32 PM
We present Olmo 3, our next family of fully open, leading language models.
This family of 7B and 32B models represents:
1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.
This family of 7B and 32B models represents:
1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.
Chinese models are enabling AI research. US progress needs to be accelerated
November 20, 2025 at 1:16 AM
Chinese models are enabling AI research. US progress needs to be accelerated
Conspiracy theory: OpenAI's serving costs were too high because GPT 5 Thinking overthought too much on easy queries, and GPT 5.1 was a patch to fix that.
5.1 just feels weird, can't quite place it.
5.1 just feels weird, can't quite place it.
November 19, 2025 at 4:08 AM
Conspiracy theory: OpenAI's serving costs were too high because GPT 5 Thinking overthought too much on easy queries, and GPT 5.1 was a patch to fix that.
5.1 just feels weird, can't quite place it.
5.1 just feels weird, can't quite place it.
A new tab on Google Scholar???
scholar.google.com/scholar_labs...
scholar.google.com/scholar_labs...
November 18, 2025 at 5:48 PM
A new tab on Google Scholar???
scholar.google.com/scholar_labs...
scholar.google.com/scholar_labs...
Why AI writing is mid
How the current way of training language models destroys any voice (and hope of good writing).
www.interconnects.ai/p/why-ai-wri...
How the current way of training language models destroys any voice (and hope of good writing).
www.interconnects.ai/p/why-ai-wri...
Why AI writing is mid
How the current way of training language models destroys any voice (and hope of good writing).
www.interconnects.ai
November 17, 2025 at 3:41 PM
Why AI writing is mid
How the current way of training language models destroys any voice (and hope of good writing).
www.interconnects.ai/p/why-ai-wri...
How the current way of training language models destroys any voice (and hope of good writing).
www.interconnects.ai/p/why-ai-wri...
Updating to GPT 5.1 in codex made the model do crazy shit when I ask it to stage changes, make a PR, revert to XYZ branch, etc.
Very shocking because even the first claude code could do this. Am I the only one?
Very shocking because even the first claude code could do this. Am I the only one?
November 16, 2025 at 11:55 PM
Updating to GPT 5.1 in codex made the model do crazy shit when I ask it to stage changes, make a PR, revert to XYZ branch, etc.
Very shocking because even the first claude code could do this. Am I the only one?
Very shocking because even the first claude code could do this. Am I the only one?
GPT 5.1 Thinking sometimes responds so much faster than GPT 5 Thinking would that it makes me think "shit I must've used the wrong model" and doubt it's answer a bit.
Funny dynamic. TBD if quality dropped at all, quality at least didn't nosedive. Hard to tell when style changes a bit.
Funny dynamic. TBD if quality dropped at all, quality at least didn't nosedive. Hard to tell when style changes a bit.
November 16, 2025 at 11:47 PM
GPT 5.1 Thinking sometimes responds so much faster than GPT 5 Thinking would that it makes me think "shit I must've used the wrong model" and doubt it's answer a bit.
Funny dynamic. TBD if quality dropped at all, quality at least didn't nosedive. Hard to tell when style changes a bit.
Funny dynamic. TBD if quality dropped at all, quality at least didn't nosedive. Hard to tell when style changes a bit.
I'm excited to announce my RLHF Book is now in pre-order for the @manning.com Early Access Program (MEAP), and for this milestone it's 50% off.
Excited to land in print in early 2026! Lots of improvements coming soon.
Thanks for the support!
hubs.la/Q03Tc37Q0
Excited to land in print in early 2026! Lots of improvements coming soon.
Thanks for the support!
hubs.la/Q03Tc37Q0
November 14, 2025 at 9:02 PM
I'm excited to announce my RLHF Book is now in pre-order for the @manning.com Early Access Program (MEAP), and for this milestone it's 50% off.
Excited to land in print in early 2026! Lots of improvements coming soon.
Thanks for the support!
hubs.la/Q03Tc37Q0
Excited to land in print in early 2026! Lots of improvements coming soon.
Thanks for the support!
hubs.la/Q03Tc37Q0
Many people are sleeping on, or even making fun of this plot in the GPT 5.1 release. This is a crucial plot for anyone serving a thinking model in real world use-cases. Latency to an answer is a huge cause of user churn and not thinking enough is a fast track to having your model's output be bad.
November 13, 2025 at 7:18 PM
Many people are sleeping on, or even making fun of this plot in the GPT 5.1 release. This is a crucial plot for anyone serving a thinking model in real world use-cases. Latency to an answer is a huge cause of user churn and not thinking enough is a fast track to having your model's output be bad.
OpenAI showing very clearly why you should care about Character Training with GPT 5.1: It's the leading selling point of the release.
November 13, 2025 at 2:45 AM
OpenAI showing very clearly why you should care about Character Training with GPT 5.1: It's the leading selling point of the release.
Lol currently every X account with 2fa auth enabled is locked out, while if you don't have 2fa you can use the app as usual. Iconic levels of broken.
November 12, 2025 at 6:28 PM
Lol currently every X account with 2fa auth enabled is locked out, while if you don't have 2fa you can use the app as usual. Iconic levels of broken.
I’m starting a new series of interviews with all the leading open model labs around the world to show why people are doing this, how people train great models, and where the ecosystem is going.
November 12, 2025 at 3:12 PM
I’m starting a new series of interviews with all the leading open model labs around the world to show why people are doing this, how people train great models, and where the ecosystem is going.
If you're working on character training research, what're you working on? What is limiting your ability to do the research you want here?
Surely there are more people studying how to modify & steer model personality after the GPT 4o sycophancy incident.
Surely there are more people studying how to modify & steer model personality after the GPT 4o sycophancy incident.
November 10, 2025 at 9:57 PM
If you're working on character training research, what're you working on? What is limiting your ability to do the research you want here?
Surely there are more people studying how to modify & steer model personality after the GPT 4o sycophancy incident.
Surely there are more people studying how to modify & steer model personality after the GPT 4o sycophancy incident.
Opening the black box of character training
Some new research from me!
Exploring how easy it is to craft personalities like sycophantic chatbots, and exploring how this will change as we move from chat to agents.
www.interconnects.ai/p/opening-th...
Some new research from me!
Exploring how easy it is to craft personalities like sycophantic chatbots, and exploring how this will change as we move from chat to agents.
www.interconnects.ai/p/opening-th...
Opening the character training pipeline
Some new research from me!
www.interconnects.ai
November 10, 2025 at 3:40 PM
Opening the black box of character training
Some new research from me!
Exploring how easy it is to craft personalities like sycophantic chatbots, and exploring how this will change as we move from chat to agents.
www.interconnects.ai/p/opening-th...
Some new research from me!
Exploring how easy it is to craft personalities like sycophantic chatbots, and exploring how this will change as we move from chat to agents.
www.interconnects.ai/p/opening-th...
The DeepSeek moment underestimated the talent portion & overfocused on training capital in N of GPUs.
The rest of 2025 has been living through that reality with Kimi, GLM, Ant Ling, Meituan... The burden of proof is back on scaling if AI will be in the hands of a few companies.
The rest of 2025 has been living through that reality with Kimi, GLM, Ant Ling, Meituan... The burden of proof is back on scaling if AI will be in the hands of a few companies.
November 8, 2025 at 1:05 PM
The DeepSeek moment underestimated the talent portion & overfocused on training capital in N of GPUs.
The rest of 2025 has been living through that reality with Kimi, GLM, Ant Ling, Meituan... The burden of proof is back on scaling if AI will be in the hands of a few companies.
The rest of 2025 has been living through that reality with Kimi, GLM, Ant Ling, Meituan... The burden of proof is back on scaling if AI will be in the hands of a few companies.
I appreciate the shoutout from @simonwillison.net
I'm building up a much richer (and direct) understanding of Chinese AI labs. Excited to share more here soon :)
I'm building up a much richer (and direct) understanding of Chinese AI labs. Excited to share more here soon :)
November 7, 2025 at 6:13 PM
I appreciate the shoutout from @simonwillison.net
I'm building up a much richer (and direct) understanding of Chinese AI labs. Excited to share more here soon :)
I'm building up a much richer (and direct) understanding of Chinese AI labs. Excited to share more here soon :)
Reposted by Nathan Lambert
The Chinese Kimi K2 thinking model beats GPT and Claude on some benchmarks. This analysis from @natolambert.bsky.social is a good overview iew of what is going on www.interconnects.ai/p/kimi-k2-th...
5 Thoughts on Kimi K2 Thinking
Quick thoughts on another fantastic open model from a rapidly rising Chinese lab.
www.interconnects.ai
November 7, 2025 at 12:07 AM
The Chinese Kimi K2 thinking model beats GPT and Claude on some benchmarks. This analysis from @natolambert.bsky.social is a good overview iew of what is going on www.interconnects.ai/p/kimi-k2-th...
Thoughts on Kimi K2 Thinking
Congrats to the Moonshot AI team on the awesome open release. For close followers of Chinese AI models, this isn't shocking, but more inflection points are coming. Pressure is building on US labs with more expensive models.
www.interconnects.ai/p/kimi-k2-th...
Congrats to the Moonshot AI team on the awesome open release. For close followers of Chinese AI models, this isn't shocking, but more inflection points are coming. Pressure is building on US labs with more expensive models.
www.interconnects.ai/p/kimi-k2-th...
November 6, 2025 at 6:53 PM
Thoughts on Kimi K2 Thinking
Congrats to the Moonshot AI team on the awesome open release. For close followers of Chinese AI models, this isn't shocking, but more inflection points are coming. Pressure is building on US labs with more expensive models.
www.interconnects.ai/p/kimi-k2-th...
Congrats to the Moonshot AI team on the awesome open release. For close followers of Chinese AI models, this isn't shocking, but more inflection points are coming. Pressure is building on US labs with more expensive models.
www.interconnects.ai/p/kimi-k2-th...