bastian bunzeck
@bbunzeck.bsky.social
wondering how humans and computers learn and use language 👶🧠🗣️🖥️💬
the work is mysterious and important, see bbunzeck.github.io
phd at @clausebielefeld.bsky.social
the work is mysterious and important, see bbunzeck.github.io
phd at @clausebielefeld.bsky.social
Reposted by bastian bunzeck
waiting for some experiments to run, so a quick thread about base models and pretraining contamination, with some weird & interesting base model generations i've collected over time.
or, why do open source models claim to be claude or chatgpt?
or, why do open source models claim to be claude or chatgpt?
January 29, 2026 at 1:12 AM
waiting for some experiments to run, so a quick thread about base models and pretraining contamination, with some weird & interesting base model generations i've collected over time.
or, why do open source models claim to be claude or chatgpt?
or, why do open source models claim to be claude or chatgpt?
Reposted by bastian bunzeck
If you, as a scientist, cannot be bothered to engage in the intellectual work of science, please quit your job and leave it to someone with skill and integrity.
“The idea is to put ChatGPT front and center inside software that scientists use to write up their work in much the same way that chatbots are now embedded into popular programming editors.
It’s vibe coding, but for science.”
It’s vibe coding, but for science.”
OpenAI’s latest product lets you vibe code science
Prism is a ChatGPT-powered text editor that automates much of the work involved in writing scientific papers.
www.technologyreview.com
January 28, 2026 at 5:10 AM
If you, as a scientist, cannot be bothered to engage in the intellectual work of science, please quit your job and leave it to someone with skill and integrity.
Reposted by bastian bunzeck
Reposted by bastian bunzeck
Interviewer: what's your biggest strength ?
Me as an developer: Machine Learning
Interviewer: what's 10 + 20 ?
Me: 0
Interviewer: Incorrect. It is 30
Me: It's 30
Interviewer: what's 50 + 30 ?
Me: it's 30
Me as an developer: Machine Learning
Interviewer: what's 10 + 20 ?
Me: 0
Interviewer: Incorrect. It is 30
Me: It's 30
Interviewer: what's 50 + 30 ?
Me: it's 30
January 26, 2026 at 3:00 PM
Interviewer: what's your biggest strength ?
Me as an developer: Machine Learning
Interviewer: what's 10 + 20 ?
Me: 0
Interviewer: Incorrect. It is 30
Me: It's 30
Interviewer: what's 50 + 30 ?
Me: it's 30
Me as an developer: Machine Learning
Interviewer: what's 10 + 20 ?
Me: 0
Interviewer: Incorrect. It is 30
Me: It's 30
Interviewer: what's 50 + 30 ?
Me: it's 30
Reposted by bastian bunzeck
The "if it works, it works" camp is big, and we welcome both the True Believers and the Understanding Haters.
It thinks the question of "Does this LLM understand anything" is a question it doesnt find interesting at all. It focuses on "Can this LLM do a thing or not? If no, can it be made to do it anyways?" and figuring out how to make it do the thing it couldn't before.
January 24, 2026 at 4:59 PM
The "if it works, it works" camp is big, and we welcome both the True Believers and the Understanding Haters.
Reposted by bastian bunzeck
One more week to apply for the PhD position on curiosity in early development (B4) in my group!
📢Applications are open for the second cohort of our RTG! Learn more about the opportunities: www.uni-goettingen.de/de/open+posi...
🔔Join our info meeting on Dec 10, 1 p.m. CET. Register via: survey.academiccloud.de/index.php/34...
🔔Join our info meeting on Dec 10, 1 p.m. CET. Register via: survey.academiccloud.de/index.php/34...
Open Positions - Georg-August-Universität Göttingen
Webseiten der Georg-August-Universität Göttingen
www.uni-goettingen.de
January 23, 2026 at 9:21 AM
One more week to apply for the PhD position on curiosity in early development (B4) in my group!
Reposted by bastian bunzeck
Constructing language: A framework for explaining acquisition. Final version by @carorowland.bsky.social & al.
doi.org/10.1016/j.tics.2025.05.015
doi.org/10.1016/j.tics.2025.05.015
January 23, 2026 at 7:44 AM
Constructing language: A framework for explaining acquisition. Final version by @carorowland.bsky.social & al.
doi.org/10.1016/j.tics.2025.05.015
doi.org/10.1016/j.tics.2025.05.015
Reposted by bastian bunzeck
Edward Gibson's new book has arrived on my desk – 350 pages on cognitively-oriented dependency syntax from MIT Press! mitpress.mit.edu/978026255357... For me, this is an exciting development, as I came to love dependency syntax in the 1980s, through European authors such as Tesnière and Mel'čuk.
January 22, 2026 at 5:27 PM
Edward Gibson's new book has arrived on my desk – 350 pages on cognitively-oriented dependency syntax from MIT Press! mitpress.mit.edu/978026255357... For me, this is an exciting development, as I came to love dependency syntax in the 1980s, through European authors such as Tesnière and Mel'čuk.
»pink is the new black«
Happening right now — @stefanhartmann.bsky.social presenting an extremely interesting case study on snowclones like »x is the new y«. 🗣️
January 22, 2026 at 2:02 PM
»pink is the new black«
Reposted by bastian bunzeck
I'm very excited about this paper with @yngwienielsen.bsky.social just out in @nathumbehav.nature.com in which we provide evidence for the mental representation of non-hierarchical linguistic structure in language use.
🧵 1/4
Read the paper here: rdcu.be/eZ26u
🧵 1/4
Read the paper here: rdcu.be/eZ26u
Evidence for the representation of non-hierarchical structures in language
Nature Human Behaviour - Language is often thought to be represented through hierarchically structured units. Nielsen and Christiansen find that non-hierarchical structures are present across...
rdcu.be
January 21, 2026 at 10:07 PM
I'm very excited about this paper with @yngwienielsen.bsky.social just out in @nathumbehav.nature.com in which we provide evidence for the mental representation of non-hierarchical linguistic structure in language use.
🧵 1/4
Read the paper here: rdcu.be/eZ26u
🧵 1/4
Read the paper here: rdcu.be/eZ26u
claude code just helped me add tons of stuff to my website in like 10 minutes. and it’s good. good good. scary good. 👀
January 21, 2026 at 8:40 PM
claude code just helped me add tons of stuff to my website in like 10 minutes. and it’s good. good good. scary good. 👀
Reposted by bastian bunzeck
🔥I am super excited for the official release of an open-source library we've been working on for about a year!
🪄interpreto is an interpretability toolbox for HF language models🤗. In both generation and classification!
Why do you need it, and for what?
1/8 (links at the end)
🪄interpreto is an interpretability toolbox for HF language models🤗. In both generation and classification!
Why do you need it, and for what?
1/8 (links at the end)
January 20, 2026 at 4:03 PM
🔥I am super excited for the official release of an open-source library we've been working on for about a year!
🪄interpreto is an interpretability toolbox for HF language models🤗. In both generation and classification!
Why do you need it, and for what?
1/8 (links at the end)
🪄interpreto is an interpretability toolbox for HF language models🤗. In both generation and classification!
Why do you need it, and for what?
1/8 (links at the end)
Reposted by bastian bunzeck
Thrilled to announce the 1st Workshop on Computational Developmental Linguistics (CDL) at ACL 2026 🎉 A new venue at the intersection of development linguistics × modern NLP, spearheaded by @fredashi.bsky.social @marstin.bsky.social, and and outstanding team of colleagues!
A thread 🧵
A thread 🧵
January 20, 2026 at 11:26 AM
Thrilled to announce the 1st Workshop on Computational Developmental Linguistics (CDL) at ACL 2026 🎉 A new venue at the intersection of development linguistics × modern NLP, spearheaded by @fredashi.bsky.social @marstin.bsky.social, and and outstanding team of colleagues!
A thread 🧵
A thread 🧵
Reposted by bastian bunzeck
The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted more—which of these new ideas work, and which work for SLMs? jurgens.people.si.umich.edu/CSE598-004/
CSE 598-004 - Building Small Language Models
jurgens.people.si.umich.edu
January 19, 2026 at 9:29 PM
The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted more—which of these new ideas work, and which work for SLMs? jurgens.people.si.umich.edu/CSE598-004/
🙂↕️🙂↕️🙂↕️
..what LLMs have learned when they learn language, the inappropriateness of parrot metaphors, the (non)autonomy of syntax, and the sheer power of pattern matching (construction grammars had it right!)
Pattern matching is not an alternative to “true” thinking, it is its key ingredient.
Pattern matching is not an alternative to “true” thinking, it is its key ingredient.
January 19, 2026 at 7:55 AM
🙂↕️🙂↕️🙂↕️
Reposted by bastian bunzeck
'stochastic parrot' is technically defensible and conceptually useless. it's the same move as referring to humans as 'meat bags' or a masterpiece as 'daubs of pigment': true at one level, distorting at every level that matters.
Erdos problems, a set of famous difficult math challenges, are a clear example of AI models breaching a threshold. The idea that an AI could solve one, let alone many, would have been insane a year ago (o1 was brand new). Now we have multiple Erdos problems solved by GPT-5.2 in the last couple weeks
January 18, 2026 at 7:34 AM
'stochastic parrot' is technically defensible and conceptually useless. it's the same move as referring to humans as 'meat bags' or a masterpiece as 'daubs of pigment': true at one level, distorting at every level that matters.
Reposted by bastian bunzeck
Using a combination of glitches in Super Mario World, Yoshi can be taken into the final battle, where he can eat Peach upon Bowser's defeat. With Peach gone, the ending will not play, and Mario and Yoshi will simply stare at each other forever.
January 14, 2026 at 5:41 PM
Using a combination of glitches in Super Mario World, Yoshi can be taken into the final battle, where he can eat Peach upon Bowser's defeat. With Peach gone, the ending will not play, and Mario and Yoshi will simply stare at each other forever.
Reposted by bastian bunzeck
Releasing v. 2.3 of ManyNames, an object naming dataset with 25K objects in real world images (English, plus partial coverage in Catalan and Mandarin Chinese). Check it out!
amore-upf.github.io/manynames/
(New in this version: further data cleaning, speaker ID, more lexical info)
amore-upf.github.io/manynames/
(New in this version: further data cleaning, speaker ID, more lexical info)
January 15, 2026 at 2:19 PM
Releasing v. 2.3 of ManyNames, an object naming dataset with 25K objects in real world images (English, plus partial coverage in Catalan and Mandarin Chinese). Check it out!
amore-upf.github.io/manynames/
(New in this version: further data cleaning, speaker ID, more lexical info)
amore-upf.github.io/manynames/
(New in this version: further data cleaning, speaker ID, more lexical info)
Highly motivated to start side project no. 27 and pivot to multimodal BabyLMs now 😮💨
This week we’re having @ecekt.bsky.social as our guest in Bielefeld. She gave a highly timely talk on language+vision models, how they process images under noise conditions, and about how to train a highly effective multimodal BabyLM with model merging. 🗣️👀💻
January 13, 2026 at 10:44 AM
Highly motivated to start side project no. 27 and pivot to multimodal BabyLMs now 😮💨
Reposted by bastian bunzeck
my competitive advantage is that i'm having fun
January 13, 2026 at 5:21 AM
my competitive advantage is that i'm having fun
Reposted by bastian bunzeck
Excited to share that our book "History of Ideas in the Science of AI" (co-authored with Luc Steels and Ann Dooms) is now freely available as #OpenAccess!
#OpenScience, #AIResearch, #HistoryOfAI
doi.org/10.5281/zeno...
#OpenScience, #AIResearch, #HistoryOfAI
doi.org/10.5281/zeno...
January 12, 2026 at 9:33 AM
Excited to share that our book "History of Ideas in the Science of AI" (co-authored with Luc Steels and Ann Dooms) is now freely available as #OpenAccess!
#OpenScience, #AIResearch, #HistoryOfAI
doi.org/10.5281/zeno...
#OpenScience, #AIResearch, #HistoryOfAI
doi.org/10.5281/zeno...
Reposted by bastian bunzeck
Introducing DroPE: Extending Context by Dropping Positional Embeddings
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
January 12, 2026 at 4:07 AM
Introducing DroPE: Extending Context by Dropping Positional Embeddings
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
Reposted by bastian bunzeck
google: we have invented agi but have hidden it in such an obscure website no one will ever find it
anthropic: through our interpretability research, we discovered claude imagines himself wearing a bow tie at all times
openai: we added slot machines
anthropic: through our interpretability research, we discovered claude imagines himself wearing a bow tie at all times
openai: we added slot machines
December 29, 2025 at 7:49 PM
google: we have invented agi but have hidden it in such an obscure website no one will ever find it
anthropic: through our interpretability research, we discovered claude imagines himself wearing a bow tie at all times
openai: we added slot machines
anthropic: through our interpretability research, we discovered claude imagines himself wearing a bow tie at all times
openai: we added slot machines
Reposted by bastian bunzeck
A fascinating recent development is that the ML research community -- as the earliest adopters of "AI for research" -- are at the frontlines of dealing with all the problems that come with that (ie. reduced trust in results & reviewers, increased submission load etc).
Every other field is next! 😭
Every other field is next! 😭
We need new rules for publishing AI-generated research. The teams developing automated AI scientists have customarily submitted their papers to standard refereed venues (journals and conferences) and to arXiv. Often, acceptance has been treated as the dependent variable. 1/
December 27, 2025 at 7:46 PM
A fascinating recent development is that the ML research community -- as the earliest adopters of "AI for research" -- are at the frontlines of dealing with all the problems that come with that (ie. reduced trust in results & reviewers, increased submission load etc).
Every other field is next! 😭
Every other field is next! 😭
Look what Santa has slipped unter my virtual Christmas tree🎄🤩
New book! I have written a book, called Syntax: A cognitive approach, published by MIT Press.
This is open access; MIT Press will post a link soon, but until then, the book is available on my website:
tedlab.mit.edu/tedlab_websi...
This is open access; MIT Press will post a link soon, but until then, the book is available on my website:
tedlab.mit.edu/tedlab_websi...
tedlab.mit.edu
December 24, 2025 at 10:35 PM
Look what Santa has slipped unter my virtual Christmas tree🎄🤩