deen
@sir-deenicus.bsky.social
tinkering on intelligence amplification. there are only memories of stories; formed into the right shape, the stories can talk back.
Moravec paradox: The robot can do a wall flip but will completely fall apart trying to do the basic beginner "six-step" move.
October 11, 2025 at 2:12 PM
Moravec paradox: The robot can do a wall flip but will completely fall apart trying to do the basic beginner "six-step" move.
One minor note, as observed by @maxine.science in the comments, is that LLMs are glorified markov chains (or more precisely, they try to approximate one) and the interesting thing is a glorified markov chain is quite powerful.
to be clear i would be much less stressed out if we actually existed in the world where LLMs were glorified markov chains/autocomplete and we understood exactly how they worked
October 11, 2025 at 12:58 PM
One minor note, as observed by @maxine.science in the comments, is that LLMs are glorified markov chains (or more precisely, they try to approximate one) and the interesting thing is a glorified markov chain is quite powerful.
Reposted by deen
Here’s some Gemini gaslighting:
• giving a wrong answer to a puzzle
• giving Python code that could test its claim
• asserting it obtained results from the code supporting its claim
• but when I ran the code myself, it showed the claim was false.
g.co/gemini/share...
• giving a wrong answer to a puzzle
• giving Python code that could test its claim
• asserting it obtained results from the code supporting its claim
• but when I ran the code myself, it showed the claim was false.
g.co/gemini/share...
Gemini - Five Dimensions: A Dimensional Puzzle
Created with Gemini
g.co
August 23, 2025 at 11:29 AM
Here’s some Gemini gaslighting:
• giving a wrong answer to a puzzle
• giving Python code that could test its claim
• asserting it obtained results from the code supporting its claim
• but when I ran the code myself, it showed the claim was false.
g.co/gemini/share...
• giving a wrong answer to a puzzle
• giving Python code that could test its claim
• asserting it obtained results from the code supporting its claim
• but when I ran the code myself, it showed the claim was false.
g.co/gemini/share...
Indeed and beyond that, the counter-intuitive thing about LLMs is that they're bad at general computation, they merely run on computers. Computing strategy profiles for imperfect info games with combinatorial state spaces, bayes inference, convex optimization solvers all still need improving/writing
Thinking carefully about optimization goals and evaluation methods not only “is still relevant” but is becoming practically necessary for a wider and wider group of students.
Should we still teach machine learning classes? Let me justify my existence.
August 22, 2025 at 9:54 PM
Indeed and beyond that, the counter-intuitive thing about LLMs is that they're bad at general computation, they merely run on computers. Computing strategy profiles for imperfect info games with combinatorial state spaces, bayes inference, convex optimization solvers all still need improving/writing
It is interesting that, not only does deep learning suffer from catastrophic forgetting, it also suffers from "plasticity loss", ie performance degrades tthat of a shallow learner under a shifting data distribution (or training against multiple). arxiv.org/abs/2404.00781
Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
Deep representation learning methods struggle with continual learning, suffering from both catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful units. While ...
arxiv.org
August 22, 2025 at 5:45 AM
It is interesting that, not only does deep learning suffer from catastrophic forgetting, it also suffers from "plasticity loss", ie performance degrades tthat of a shallow learner under a shifting data distribution (or training against multiple). arxiv.org/abs/2404.00781
That's the wrong takeaway from this outcome. Even with an LLM driving, quality of retrieval will still matter due to needing to prioritize across tens to hundreds of millions of documents/records.
LLMs are bad at computation; side-effect of Faustian bargain for better usage of parallel flops
LLMs are bad at computation; side-effect of Faustian bargain for better usage of parallel flops
they killed a benchmark
@letta.com figured out how to game memory benchmark LoCoMo — and then they wrote about it!
the takeaways are actually quite incredible. like, vectors & graphs are cool, but you’re better off just giving an agent better tools
www.letta.com/blog/benchma...
@letta.com figured out how to game memory benchmark LoCoMo — and then they wrote about it!
the takeaways are actually quite incredible. like, vectors & graphs are cool, but you’re better off just giving an agent better tools
www.letta.com/blog/benchma...
August 13, 2025 at 6:51 PM
That's the wrong takeaway from this outcome. Even with an LLM driving, quality of retrieval will still matter due to needing to prioritize across tens to hundreds of millions of documents/records.
LLMs are bad at computation; side-effect of Faustian bargain for better usage of parallel flops
LLMs are bad at computation; side-effect of Faustian bargain for better usage of parallel flops
This paper also, is not the complete picture. An uncareful reading might mistakenly lead one into thinking LLMs learn complete algorithms, but they do not! The more subtle picture is, exhibited algorithms are implemented by the interactions of lots of heuristics/functions. arxiv.org/abs/2410.212...
August 12, 2025 at 6:40 PM
This paper also, is not the complete picture. An uncareful reading might mistakenly lead one into thinking LLMs learn complete algorithms, but they do not! The more subtle picture is, exhibited algorithms are implemented by the interactions of lots of heuristics/functions. arxiv.org/abs/2410.212...
379709726798671623170154914653099124690369479097664990658075118650615429539243340414911636796350741652250491582069972052351496710185354320898239363164310274804151650208096406757467682054257956383712571315143975925420275697670234838615414782231861448256037612500
February 21, 2025 at 7:40 PM
379709726798671623170154914653099124690369479097664990658075118650615429539243340414911636796350741652250491582069972052351496710185354320898239363164310274804151650208096406757467682054257956383712571315143975925420275697670234838615414782231861448256037612500
λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙ξ⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙ξ⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙ξ⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩
February 21, 2025 at 10:11 AM
λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙ξ⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙ξ⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙ξ⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩⟩⟩⟩∙⟨λφ.φ∙⟨⊥⟩∙⟨λφ.φ∙⟨λφ.λξ.φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙⟨φ∙ξ⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩⟩
❔ 🐟 🎣 ⛵
Sonnet: Dave the diver, Raft, Subnautica
Gemini Exp 1206: Dredge, Dave the Diver, Moonglow Bay
Gemini Flash Thinker: Fishing: North Atlantic /A Fishing sim, Sailwind, Subnautica
GPT4o: Fishing: North Atlantic, Dredge, Sailwind
o1: Dredge, Moonglow Bay, Call of the Sea
Sonnet: Dave the diver, Raft, Subnautica
Gemini Exp 1206: Dredge, Dave the Diver, Moonglow Bay
Gemini Flash Thinker: Fishing: North Atlantic /A Fishing sim, Sailwind, Subnautica
GPT4o: Fishing: North Atlantic, Dredge, Sailwind
o1: Dredge, Moonglow Bay, Call of the Sea
December 24, 2024 at 1:34 AM
❔ 🐟 🎣 ⛵
Sonnet: Dave the diver, Raft, Subnautica
Gemini Exp 1206: Dredge, Dave the Diver, Moonglow Bay
Gemini Flash Thinker: Fishing: North Atlantic /A Fishing sim, Sailwind, Subnautica
GPT4o: Fishing: North Atlantic, Dredge, Sailwind
o1: Dredge, Moonglow Bay, Call of the Sea
Sonnet: Dave the diver, Raft, Subnautica
Gemini Exp 1206: Dredge, Dave the Diver, Moonglow Bay
Gemini Flash Thinker: Fishing: North Atlantic /A Fishing sim, Sailwind, Subnautica
GPT4o: Fishing: North Atlantic, Dredge, Sailwind
o1: Dredge, Moonglow Bay, Call of the Sea
I think it's possible (and based on our apparent trajectory, likely that) LLMs will be able to discover or identify new frontiers but will struggle when tasked to venture into them.
On the Limits of Large Language Models in Operating in vs Discovering New Fields of Study
This post originated in response to a post by Victor Taelin on X: https://x.com/VictorTaelin/status/1865144235227517053
metarecursive.substack.com
December 11, 2024 at 12:31 AM
I think it's possible (and based on our apparent trajectory, likely that) LLMs will be able to discover or identify new frontiers but will struggle when tasked to venture into them.
Reposted by deen
More evidence that protein folding NNs have learned a crude approximation of an energy function, though the authors here suggest it is at the local level exclusively, because it is only able to make incremental refinements in the absence of coevolutionary data academic.oup.com/bioinformati...
December 4, 2024 at 5:10 PM
More evidence that protein folding NNs have learned a crude approximation of an energy function, though the authors here suggest it is at the local level exclusively, because it is only able to make incremental refinements in the absence of coevolutionary data academic.oup.com/bioinformati...
Thread to gather thoughts on how Transformers work. Two aspects. One is the sense in which we can informally analogize feed forward portions with 3D meshes and formally as tropical hypersurfaces.
For attention, based on a 2023 paper, the inhomogenous mixtures of vMFs on a hypersphere perspective.
For attention, based on a 2023 paper, the inhomogenous mixtures of vMFs on a hypersphere perspective.
December 1, 2024 at 5:14 PM
Thread to gather thoughts on how Transformers work. Two aspects. One is the sense in which we can informally analogize feed forward portions with 3D meshes and formally as tropical hypersurfaces.
For attention, based on a 2023 paper, the inhomogenous mixtures of vMFs on a hypersphere perspective.
For attention, based on a 2023 paper, the inhomogenous mixtures of vMFs on a hypersphere perspective.
I wonder if there's extra psychological pressure for this year's FIDE World Chess Championship players, given the symbolism of the countries they're both from.
December 1, 2024 at 4:28 PM
I wonder if there's extra psychological pressure for this year's FIDE World Chess Championship players, given the symbolism of the countries they're both from.
Reposted by deen
That's why fair use expert @pamelasamuelson.bsky.social urges nonprofit AI researchers to consider filing amicus briefs in the current lawsuits against AI companies (is EleutherAI on that?): A techlash-fueled broad anti fair use ruling could harm noncommercial AI too
www.youtube.com/watch?v=S7Zp...
www.youtube.com/watch?v=S7Zp...
Will Copyright Derail Generative AI Technologies?
YouTube video by Simons Institute
www.youtube.com
November 30, 2024 at 7:24 AM
That's why fair use expert @pamelasamuelson.bsky.social urges nonprofit AI researchers to consider filing amicus briefs in the current lawsuits against AI companies (is EleutherAI on that?): A techlash-fueled broad anti fair use ruling could harm noncommercial AI too
www.youtube.com/watch?v=S7Zp...
www.youtube.com/watch?v=S7Zp...
In an open-source model, we can inject into the unrolling dependent sequences trace (or reasoning trace for short) and say things like "ping. msg: it's almost time to wrap up this session".
There are all kinds of injection possibilities; including web search, tool use, nesting and fork + join.
There are all kinds of injection possibilities; including web search, tool use, nesting and fork + join.
November 29, 2024 at 4:52 AM
In an open-source model, we can inject into the unrolling dependent sequences trace (or reasoning trace for short) and say things like "ping. msg: it's almost time to wrap up this session".
There are all kinds of injection possibilities; including web search, tool use, nesting and fork + join.
There are all kinds of injection possibilities; including web search, tool use, nesting and fork + join.
The fixation on next token prediction by both detractors and proponents of LLMs strikes me as odd. Some dismissively say, "It's just next token prediction," while proponents say, "Isn't it amazing that next token prediction works so well?!" But both perspectives, to me, seem to miss a deeper point.
Moving from Next Token Prediction to The Sculpted Geometry of Language
Why next token prediction is such a shallow and uninteresting frame
metarecursive.substack.com
November 28, 2024 at 5:45 PM
The fixation on next token prediction by both detractors and proponents of LLMs strikes me as odd. Some dismissively say, "It's just next token prediction," while proponents say, "Isn't it amazing that next token prediction works so well?!" But both perspectives, to me, seem to miss a deeper point.
Reposted by deen
Rather than restricting data to only the richest and most powerful (as reddit, facebook, and twitter do), bsky makes it available to everyone.
Personally, I think that's a good thing.
Personally, I think that's a good thing.
November 28, 2024 at 9:56 AM
Rather than restricting data to only the richest and most powerful (as reddit, facebook, and twitter do), bsky makes it available to everyone.
Personally, I think that's a good thing.
Personally, I think that's a good thing.
Reposted by deen
My new novel MORPHOTROPHIC is available now!
You can read the first two chapters here:
www.gregegan.net/MORPHOTROPHI...
You can read the first two chapters here:
www.gregegan.net/MORPHOTROPHI...
November 25, 2024 at 1:14 PM
My new novel MORPHOTROPHIC is available now!
You can read the first two chapters here:
www.gregegan.net/MORPHOTROPHI...
You can read the first two chapters here:
www.gregegan.net/MORPHOTROPHI...
The main disagreement I have with this image is, I'd push "human makes all high level decisions" to Level 3.
Given that, I think we're somewhere around Level 3 to 4 (3 > level < 5), with most of the useful stuff < level 4. As of today, level 5 does not appear to be in sight.
Given that, I think we're somewhere around Level 3 to 4 (3 > level < 5), with most of the useful stuff < level 4. As of today, level 5 does not appear to be in sight.
November 25, 2024 at 10:04 PM
The main disagreement I have with this image is, I'd push "human makes all high level decisions" to Level 3.
Given that, I think we're somewhere around Level 3 to 4 (3 > level < 5), with most of the useful stuff < level 4. As of today, level 5 does not appear to be in sight.
Given that, I think we're somewhere around Level 3 to 4 (3 > level < 5), with most of the useful stuff < level 4. As of today, level 5 does not appear to be in sight.