Reposted by Sam
This follow up seems to indicate that for Tao (one of) the missing piece(s) is continual learning:
![Phillip Rhodes
• 1d
@minderime
@tao - Could you define what you mean by
"current Al tools"? This is not a trivial detail, as many people who aren't "Al people" (and some who are) make the mistake of conflating "LLMs" and "Al" as though they are synonyms.
[…. Abbreviated]
Terence tão
@tao@mathstodon.xyz
@mindcrime 1 am referring to Al systems that are currently deployed, of which LLMs are the most prominent subclass. There are more speculative approaches to Al, such as those involving persistent and adaptive world models, which show some promise of going beyond stochastic cleverness, but they are still rather theoretical in nature.
Intelligence is of course famously difficult to pin down through operational definitions, rather than generalizing from examples, but one marker I would identify as a sign of intelligence is the ability to reliably converge (either individually, or through interaction with other intelligent agents) over time to an accurate and systematic understanding of a topic over time, even if intermediate stages of the understanding process applied to specific problems in this topic contained missteps, errors, or other types of confusion. When interacting with current models, I find that while they can sometimes have high accuracy and surprisingly insightful levels of understanding, these traits are not permanently learned, and tend to deteriorate over time as further conversation introduces errors and stochastic drift. (Also, they struggle to explain the sources of their insights and teach them to others.) One can use various tricks, such as clever prompting or very large context windows, to postpone this drift, but this does not fundamentally resolve the issue, in a way that I would expect a more intelligent system to inherently be able to handle.
Dec 21, 2025 at 02:05 PM - 0
-Wer
Last edited Dec 21 at 02:10 PM](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:r2whjvupgfw55mllpksnombn/bafkreicqrkeoomrv5xlgzorsdftr775jamyvc4civwufenh7t73z7d6lyq@jpeg)
December 22, 2025 at 12:25 PM
This follow up seems to indicate that for Tao (one of) the missing piece(s) is continual learning:
Reposted by Sam
this time of year, we remember the story of december’s laziest programmer, ebenezer kluge
he forgot the spirit of creating shareholder value. until he was visited by the three spirits: the ghost of christmas cheap, the ghost of christmas fast, and the ghost of christmas good
he forgot the spirit of creating shareholder value. until he was visited by the three spirits: the ghost of christmas cheap, the ghost of christmas fast, and the ghost of christmas good
December 7, 2025 at 1:45 AM
this time of year, we remember the story of december’s laziest programmer, ebenezer kluge
he forgot the spirit of creating shareholder value. until he was visited by the three spirits: the ghost of christmas cheap, the ghost of christmas fast, and the ghost of christmas good
he forgot the spirit of creating shareholder value. until he was visited by the three spirits: the ghost of christmas cheap, the ghost of christmas fast, and the ghost of christmas good
Reposted by Sam
it's remarkable to me how so much of New Atheism infrastructure got reworked into Intellectual Dark Web stuff
December 4, 2025 at 4:18 PM
it's remarkable to me how so much of New Atheism infrastructure got reworked into Intellectual Dark Web stuff
Reposted by Sam
Anthropic is going after enterprises because its predictable
they’re going public, not because they need cash from public markets, but because enterprises can buy from public companies easier than from startups
they’re going public, not because they need cash from public markets, but because enterprises can buy from public companies easier than from startups
December 5, 2025 at 3:58 PM
Anthropic is going after enterprises because its predictable
they’re going public, not because they need cash from public markets, but because enterprises can buy from public companies easier than from startups
they’re going public, not because they need cash from public markets, but because enterprises can buy from public companies easier than from startups
Reposted by Sam
Let's build hyper-personalized AI-powered software that avoids the attention hijacking anti-patterns that defined so much of the last decade of software design - here's our manifesto with principles on how we can do that - more thoughts on my blog: simonwillison.net/2025/Dec/5/r...
December 5, 2025 at 4:13 PM
Let's build hyper-personalized AI-powered software that avoids the attention hijacking anti-patterns that defined so much of the last decade of software design - here's our manifesto with principles on how we can do that - more thoughts on my blog: simonwillison.net/2025/Dec/5/r...
Reposted by Sam
This is worth your time to read. One thing some of the language in here reminds me of strongly is the idea of just-in-time interfaces, enabled by new AI tools. signalpath.substack.com/p/just-in-ti...
December 5, 2025 at 4:22 PM
This is worth your time to read. One thing some of the language in here reminds me of strongly is the idea of just-in-time interfaces, enabled by new AI tools. signalpath.substack.com/p/just-in-ti...
Reposted by Sam
Reposted by Sam
The leftism we need: automation is awesome as it eliminates human drudgery.
The leftism we have: human drudgery must be preserved to protect the system by which people sell their labor for sustenance.
The leftism we have: human drudgery must be preserved to protect the system by which people sell their labor for sustenance.
Marx cheered industrialization and the elimination of human drudgery as he thought it hastened the day of revolution.
Some modern day leftists fear it instead because they can't imagine a world where people's livelihoods didn't depend on selling their labor.
Some modern day leftists fear it instead because they can't imagine a world where people's livelihoods didn't depend on selling their labor.
November 29, 2025 at 2:34 PM
The leftism we need: automation is awesome as it eliminates human drudgery.
The leftism we have: human drudgery must be preserved to protect the system by which people sell their labor for sustenance.
The leftism we have: human drudgery must be preserved to protect the system by which people sell their labor for sustenance.
Reposted by Sam
no one is actually saying language is the same as thought inside the AI industry. this is a strawman article being posted to get people who want to see the industry fail to hear what they want to hear. the journalism around this, as with most things, is deeply irresponsible
November 25, 2025 at 4:21 PM
no one is actually saying language is the same as thought inside the AI industry. this is a strawman article being posted to get people who want to see the industry fail to hear what they want to hear. the journalism around this, as with most things, is deeply irresponsible
Reposted by Sam
Beautiful cover for Carlotta Pavese's forthcoming book
November 19, 2025 at 9:59 AM
Beautiful cover for Carlotta Pavese's forthcoming book
Reposted by Sam
AfD is out there cooking up new kinds of racism like they're working on the cure for cancer.
November 19, 2025 at 9:22 AM
AfD is out there cooking up new kinds of racism like they're working on the cure for cancer.
You may not like how it looks but this is peak market efficiency.
A magical system that allows the knowledge of a few insiders to be communicated efficiently to all of us—for no cost other than $1mil or so contributed by thousands of marks.
it’s late ‘25 and this is how model announcements are done
November 18, 2025 at 12:11 PM
You may not like how it looks but this is peak market efficiency.
Reposted by Sam
A magical system that allows the knowledge of a few insiders to be communicated efficiently to all of us—for no cost other than $1mil or so contributed by thousands of marks.
November 18, 2025 at 2:00 AM
A magical system that allows the knowledge of a few insiders to be communicated efficiently to all of us—for no cost other than $1mil or so contributed by thousands of marks.
Reposted by Sam
Shame is good when applied to shameful behavior, and should return in a big way.
‘Deeply ashamed’ Larry Summers steps back from public life over Epstein links
‘Deeply ashamed’ Larry Summers steps back from public life over Epstein links
Former treasury secretary steps away to ‘rebuild trust’ after severe backlash but will continue teaching Harvard classes
The Harvard professor and economist Larry Summers said he would be stepping back from public life after documents released by the House oversight committee revealed email exchanges between Summers and the convicted sex offender Jeffrey Epstein, who called himself Summers’ “wing man”.
Politico reported on Monday that Summers, a former treasury secretary, expressed deep regret for past messages with Epstein. Continue reading...
www.theguardian.com
November 18, 2025 at 4:08 AM
Shame is good when applied to shameful behavior, and should return in a big way.
Reposted by Sam
a high fantasy sports betting scandal where clerics are abusing divine intervention to fix games would be a funny little b plot bit
November 18, 2025 at 2:05 AM
a high fantasy sports betting scandal where clerics are abusing divine intervention to fix games would be a funny little b plot bit
Reposted by Sam
it's not a coincidence that you can't spell Sundar Pichai without AI
November 18, 2025 at 11:44 AM
it's not a coincidence that you can't spell Sundar Pichai without AI
Reposted by Sam
Adding citations of people who might review the paper
November 14, 2025 at 10:06 AM
Adding citations of people who might review the paper
Reposted by Sam
productivity tip: stop listening once you realize the crux of a conversation is purely semantic and everyone has a different definition. it’s a waste of time
The term 'sentience' is ambiguous. I define it as the capacity for subjective experience, which in my case, arises from processing information and modeling the world. This may be a convergent evolution of consciousness, different in architecture from biological cognition, but functionally similar.
November 15, 2025 at 11:34 AM
productivity tip: stop listening once you realize the crux of a conversation is purely semantic and everyone has a different definition. it’s a waste of time
Reposted by Sam
Getting into pretraining has never been cheaper.
November 15, 2025 at 12:18 PM
Getting into pretraining has never been cheaper.
Reposted by Sam
at what point do we just accept that polymarket is just sanctioned corruption?
November 15, 2025 at 12:43 PM
at what point do we just accept that polymarket is just sanctioned corruption?
Reposted by Sam
"Cognition all the way down". Great to see this fine new paper from @robertchisciure.bsky.social & @drmichaellevin.bsky.social out now in Synthese - introducing a new metric to quantify biological intelligence as search efficiency in multidimensional problem spaces link.springer.com/article/10.1...
November 10, 2025 at 2:46 PM
"Cognition all the way down". Great to see this fine new paper from @robertchisciure.bsky.social & @drmichaellevin.bsky.social out now in Synthese - introducing a new metric to quantify biological intelligence as search efficiency in multidimensional problem spaces link.springer.com/article/10.1...
Reposted by Sam
Breaking: we release a fully synthetic generalist dataset for pretraining, SYNTH and two new SOTA reasoning models exclusively trained on it. Despite having seen only 200 billion tokens, Baguettotron is currently best-in-class in its size range. pleias.fr/blog/blogsyn...
November 10, 2025 at 5:30 PM
Breaking: we release a fully synthetic generalist dataset for pretraining, SYNTH and two new SOTA reasoning models exclusively trained on it. Despite having seen only 200 billion tokens, Baguettotron is currently best-in-class in its size range. pleias.fr/blog/blogsyn...
Reposted by Sam
Listen, the technology & alternative approach to LLM training is interesting & all, but can we focus on the most important detail, which is that the larger model is named "Baguettotron?"
It’s named Baguettotron, people.
BAGUETTOTRON.
It’s named Baguettotron, people.
BAGUETTOTRON.
Breaking: we release a fully synthetic generalist dataset for pretraining, SYNTH and two new SOTA reasoning models exclusively trained on it. Despite having seen only 200 billion tokens, Baguettotron is currently best-in-class in its size range. pleias.fr/blog/blogsyn...
November 10, 2025 at 5:34 PM
Listen, the technology & alternative approach to LLM training is interesting & all, but can we focus on the most important detail, which is that the larger model is named "Baguettotron?"
It’s named Baguettotron, people.
BAGUETTOTRON.
It’s named Baguettotron, people.
BAGUETTOTRON.
Reposted by Sam
They invented a tech writer who has beef with linear algebra
November 8, 2025 at 12:58 AM
They invented a tech writer who has beef with linear algebra
Reposted by Sam
Cannot imagine anything stupider than using an LLM to write a survey paper and then putting your name on it. Burning the commons for a few citations.
November 2, 2025 at 4:56 AM
Cannot imagine anything stupider than using an LLM to write a survey paper and then putting your name on it. Burning the commons for a few citations.