Lightnews — Scholar-powered news

Reposted by Alexander Huth

Rodrigo Braga @rodbraga.bsky.social · 1d

📣 New preprint from the Braga Lab! 📣

The ventral visual stream for reading converges on the transmodal language network

Congrats to Dr. Joe Salvo for this epic set of results

Big Q: What brain systems support the translation of writing to concepts and meaning?

Thread 🧵 ⬇️

1 14 53

Alexander Huth @alexanderhuth.bsky.social · 7d

Always include some stimuli that are permissively licensed so they can be used as examples! E.g. we have a video stimulus set that's mostly Pixar short films, but also includes a segment from the Blender movie Sintel (en.wikipedia.org/wiki/Sintel), which is licensed CC-BY.

Sintel - Wikipedia

en.wikipedia.org

2

Alexander Huth @alexanderhuth.bsky.social · 25d

Happy and proud to see @rjantonello.bsky.social’s work awarded by SNL!

MIT Press @mitpress.bsky.social · 25d

🏆 Best Paper — Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data: doi.org/10.1162/nol_...

Richard Antonello and @alexanderhuth.bsky.social

Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

Abstract. Many recent studies have shown that representations drawn from neural network language models are extremely effective at predicting brain responses to natural language. But why do these…

doi.org

1 4 29

Reposted by Alexander Huth

Alejandro de la Vega @neurozorro.bsky.social · 27d

Our latest paper outlining our ecosystem of tools for mining the neuroimaging literature, is finally officially published in eLife! doi.org/10.7554/eLif...

Mining the neuroimaging literature

New tools for literature mining, such as automated analysis of the research literature, are accessible, scalable, and reliable.

doi.org

1 4 18

Alexander Huth @alexanderhuth.bsky.social · 28d

Someone on the aggies subreddit posted the course description and it seems pretty thorough! www.reddit.com/r/aggies/s/C...

Wang_Lung_1921's comment on "Regardless of your stance on the politics behind the Welsh/LGBT situation…"

Explore this conversation and more from the aggies community

www.reddit.com

4

Reposted by Alexander Huth

Heejung Jung @jungheejung.bsky.social · Sep 4

New Open dataset alert:
🧠 Introducing "Spacetop" – a massive multimodal fMRI dataset that bridges naturalistic and experimental neuroscience!

N = 101 x 6 hours each = 606 functional iso-hours combining movies, pain, faces, theory-of-mind and other cognitive tasks!

🧵below

3 58 120

Alexander Huth @alexanderhuth.bsky.social · Aug 18

And of course check out the paper! www.biorxiv.org/content/10.1...

Evaluating scientific theories as predictive models in language neuroscience

Modern data-driven encoding models are highly effective at predicting brain responses to language stimuli. However, these models struggle to explain the underlying phenomena, i.e. what features of the...

www.biorxiv.org

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

or @csinva.bsky.social: bsky.app/profile/csin...

Chandan Singh @csinva.bsky.social · Aug 14

New paper: Ask 35 simple questions about sentences in a story and use the answers to predict brain responses. Interpretable, compact, & surprisingly high performance in both fMRI and ECoG. 🧵 biorxiv.org/content/10.1...

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

I'm posting this thread to highlight some things I thought cool, but if you're interested you should also check out what @rjantonello.bsky.social wrote: bsky.app/profile/rjan...

RJ Antonello @rjantonello.bsky.social · Aug 18

In our new paper, we explore how we can build encoding models that are both powerful and understandable. Our model uses an LLM to answer 35 questions about a sentence's content. The answers linearly contribute to our prediction of how the brain will respond to that sentence. 1/6

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

Cortical weight maps were also reasonably correlated between ECoG and fMRI data, at least for the dimensions well-captured in the ECoG coverage.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

Finally, we tested whether the same interpretable embeddings could also be used to model ECoG data from Nima Mesgarani's lab. Despite the fact that our features are less well-localized in time than LLM embeddings, this still works quite well!

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

To validate the maps we get from this model we also compared them to expectations derived from NeuroSynth and results from experiments targeting specific semantic categories, and also looked at inter-subject reliability. All quite successful.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

The model and experts were well-aligned, but there were some surprises, like "Does the input include technical or specialized terminology?" (32), which was much more important than expected.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

This method lets us quantitatively assess how much variance different theories explain about brain responses to natural language. So to figure out how well this aligns with what scientists think, we polled experts to see which questions/theories they thought would be important.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

"Does the input include dialogue?" (27) has high weights in a smattering of small regions in temporal cortex. And "Does the input contain a negation?" (35) has high weights in anterior temporal lobe and a few prefrontal areas. I think there's a lot of drilling-down we can do here.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

The fact that each dimension in the embedding thus corresponds to a specific question means that the encoding model weights are interpretable right out-of-the-box. "Does the input describe a visual experience?" has high weight all along the boundary of visual cortex, for example.

Left hemisphere cortical flatmap showing regression weights for the feature "Does the input describe a visual experience or scene?"

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

But the wilder thing is how we get the embeddings: by just asking LLMs questions. Each theory is cast as a yes/no question. We then have GPT-4 answer each question about each 10-gram in our natural language dataset. We did this for ~600 theories/questions.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

And it works REALLY well! Prediction performance for encoding models is on a par with uninterpretable Llama3 embeddings! Even with just 35 dimensions!!! I find this fairly wild.

Average test encoding performance across cortex for the QA model and baselines on the three original subjects (20
hours of fMRI data each) and 5 additional subjects (5 hours each). The 35-question QA model outperformed the state-of-the-art
black-box model (which uses hidden representations from the LLaMA family of LLMs) by 12.0% when trained on all the story
data. The model’s compactness yields greater relative improvements in data-limited scenarios; when trained on only 5 stories per
subject it outperforms the baseline LLaMA model by 43.3%.

1

Alexander Huth @alexanderhuth.bsky.social · Aug 18

New paper with @rjantonello.bsky.social @csinva.bsky.social, Suna Guo, Gavin Mischler, Jianfeng Gao, & Nima Mesgarani: We use LLMs to generate VERY interpretable embeddings where each dimension corresponds to a scientific theory, & then use these embeddings to predict fMRI and ECoG. It WORKS!

bioRxiv Neuroscience @biorxiv-neursci.bsky.social · Aug 13

Evaluating scientific theories as predictive models in language neuroscience https://www.biorxiv.org/content/10.1101/2025.08.12.669958v1

1 8 16

Reposted by Alexander Huth

RJ Antonello @rjantonello.bsky.social · Aug 18

In our new paper, we explore how we can build encoding models that are both powerful and understandable. Our model uses an LLM to answer 35 questions about a sentence's content. The answers linearly contribute to our prediction of how the brain will respond to that sentence. 1/6

1 9 25

Reposted by Alexander Huth

Jianing Mu @mujianing.bsky.social · Aug 1

The preprint of my 1st project in grad school is up 🙌 We propose a simple, information-theoretic model of how humans remember narratives. We tested it with the help of open-source LLMs. Plz check out this thread for details ➡️ Many thanks to my wonderful advisors! It's been a fun adventure!!

Alexander Huth @alexanderhuth.bsky.social · Aug 1

New paper with @mujianing.bsky.social & @prestonlab.bsky.social! We propose a simple model for human memory of narratives: we uniformly sample incoming information at a constant rate. This explains behavioral data much better than variable-rate sampling triggered by event segmentation or surprisal.

bioRxiv Neuroscience @biorxiv-neursci.bsky.social · Aug 1

Efficient uniform sampling explains non-uniform memory of narrative stories https://www.biorxiv.org/content/10.1101/2025.07.31.667952v1

2 4 16

Alexander Huth @alexanderhuth.bsky.social · Aug 1

So if you're interested in human memory for complex stimuli, please check the paper out! www.biorxiv.org/content/10.1...

Efficient uniform sampling explains non-uniform memory of narrative stories

Humans do not remember all experiences uniformly. We remember certain moments better than others, and central gist better than detail. Current theories focus exclusively on surprise to explain why som...

www.biorxiv.org

5

Alexander Huth @alexanderhuth.bsky.social · Aug 1

This work was a really fun departure for me. Nothing data-driven (and no fMRI!), we just sat down and devised a theory, then tested it. It feels surprisingly good :D

1 6

Alexander Huth @alexanderhuth.bsky.social · Aug 1

Our model also has interesting linguistic consequences. Speech tends to have uniform information density over time, but there are local variations. We argue that these variations (at least around event boundaries) are actually in service of more uniform _memory_ density.

1 1

Alexander Huth @alexanderhuth.bsky.social · Aug 1

We also devised a new way to model and study gist with LLMs, which is to measure (or manipulate) the entropy of attention weights for specific "induction" heads within the LLM. Higher entropy attention weights more evenly sample information from the input, and lead to gist-like behavior.

1 1