Lightnews — Scholar-powered news

Rodrigo Braga @rodbraga.bsky.social · 28m

Tagging co-authors:
@josephsalvo.bsky.social
Maya Lakshman
@aniaholubecki.bsky.social
@zeynepsaygin.bsky.social
Marsel Mesulam

1

Rodrigo Braga @rodbraga.bsky.social · 35m

...whatever the visual properties of those things.

Our work suggests that this specialization is shaped by connections to large-scale brain networks that aren't visual

So the interplay between visual and non-visual networks seems to shape the specialization of these late-stage visual areas

Rodrigo Braga @rodbraga.bsky.social · 38m

Excellent Q - see the paper below about how the specialization in this part of the brain is influenced by early life experience:
www.nature.com/articles/nn....

Could be this part of the brain specializes for processing visual things that are behaviorally important:
arxiv.org/abs/2411.08251

1

Rodrigo Braga @rodbraga.bsky.social · 17h

We welcome feedback - Read the full preprint here:
👉 www.biorxiv.org/content/10.1...

And congrats to Joe Salvo and the team for a careful and comprehensive (5y+) study!

🧵 The end

The ventral visual stream for reading converges on the transmodal language network

Reading bridges sensation and cognition. To derive meaning from written words, visual input is first processed in unimodal (i.e., sensory-specific) visual streams and then engages a distributed langua...

www.biorxiv.org

1 1

Rodrigo Braga @rodbraga.bsky.social · 17h

We propose that thinking about this region a “nexus”, or a hand-off point, between visual and transmodal language systems, is useful for understanding why it is so critical for reading.

1

Rodrigo Braga @rodbraga.bsky.social · 17h

We think this can help us understand the weird properties of the VWFA: it shows a mixture of responses (visual, speech) because it is at the confluence of two brain systems: the transmodal LANG network and unimodal (orthographic) visual stream.

1

Rodrigo Braga @rodbraga.bsky.social · 17h

Conclusion: Our results support that the network connections to LANG can explain functional differences along the ventral visual stream.

More anterior stream = more transmodal, more selective for word-like visual input
More posterior stream = more visual, less discriminant about visual stimuli

1

Rodrigo Braga @rodbraga.bsky.social · 17h

This suggests that the basal LANG region seems to care about visual stimuli that look like they might be words and readable

1

Rodrigo Braga @rodbraga.bsky.social · 17h

AND – we replicated that familiar (Roman) letter strings activate the whole orthographic stream, but in contrast, strings of numbers only activated the more posterior regions!

This was very surprising – numbers are also highly learned, yet their activity is like the foreign scripts map above:

1

Rodrigo Braga @rodbraga.bsky.social · 17h

Using the NSD fLOC task data, we replicated that the anterior part of the orthographic stream (teal) converges on that basal LANG region:

1

Rodrigo Braga @rodbraga.bsky.social · 17h

We next replicated the key results in the Natural Scenes Dataset (NSD; Allen et al. 2018).

In three subjects, the FC map of LANG confirmed the ventral temporal region:

(there was a lot more dropout in this non multi-echo data so the other subs were inconclusive).

1

Rodrigo Braga @rodbraga.bsky.social · 17h

The network connections to LANG therefore do differentiate visual responses along the stream, but “Meaning” does not seem to be a deciding factor – consonant strings and pseudowords also activated the whole stream, including the LANG region.

1

Rodrigo Braga @rodbraga.bsky.social · 17h

We saw that the whole stream activated for stimuli in a familiar (Roman) script, replicating what we saw with pseudowords.

BUT: the foreign (Japanese) scripts only activated the more posterior regions – specifically missing the anterior region that is within LANG! 😱

1

Rodrigo Braga @rodbraga.bsky.social · 17h

To test this further, we had the same subs view Pseudowords again, but also Real Words, Consonant Strings, Symbols and Line Drawings.

Some of these involved a familiar (Roman) script, whereas the Symbols category had similar visual features but was in an unfamiliar script (modified Japanese).

1

Rodrigo Braga @rodbraga.bsky.social · 17h

So: the results suggest that the visual stream for reading converges precisely on a region of the distributed LANG network.

The LANG network connections therefore predict functional differences within the visual stream.

2 1

Rodrigo Braga @rodbraga.bsky.social · 17h

Indeed, this anterior orthographic stream region also activates during listening to speech (but the Face and Scene streams don’t overlap with speech regions as consistently; see graph).

In almost all cases, the overlap was right where the FC-defined basal LANG region was (black lines).

1

Rodrigo Braga @rodbraga.bsky.social · 17h

This implies that the orthographic visual stream, used for reading, culminates exactly in the LANG network!

The anterior part of the stream actually has a transmodal function (even though it was defined here using a visual Iocalizer task, using pseudowords that aren’t meaningful)

1

Rodrigo Braga @rodbraga.bsky.social · 17h

Intriguingly, in many subjects the orthographic stream (teal) extends in an arc that ends exactly where the basal LANG region is (see black lines)!

The other streams for Faces (blue) & Scenes (purple) didn’t overlap with LANG as reliably.

1

Rodrigo Braga @rodbraga.bsky.social · 17h

In each subject, viewing Pseudowords activated multiple regions extening from near the occipital pole to the middle of the ventral surface.

We called it an ‘orthographic stream’, nodding to the “multiple VWFAs” of:

Yeatman 2021: doi.org/10.1146/annu...
Woolnough 2021: doi.org/10.1038/s415...

1

Rodrigo Braga @rodbraga.bsky.social · 17h

If this region is transmodal, what is its contribution to reading? Does it have different properties to the VWFA? Is it the VWFA?

In the same subjects, we next mapped category-preferring visual areas using a classic visual categories task:

1

Rodrigo Braga @rodbraga.bsky.social · 17h

Because this region is connected to LANG, it should have a transmodal function, despite being near the visual hierarchy.

And it does! Listening to speech (red-yellow) activates it.

(This is known but v. under-appreciated).
doi.org/10.1093/brai...
doi.org/10.1093/brai...
doi.org/10.1162/imag...

1 1

Rodrigo Braga @rodbraga.bsky.social · 18h

Using precision fMRI, we first mapped LANG using functional connectivity (FC) in 8 individuals.

We observed that LANG reliably contains a basal temporal region that is not a “classic” language region:

(We used multi-echo to overcome dropout)

1

Rodrigo Braga @rodbraga.bsky.social · 18h

In this preprint, we traced activity along the “orthographic stream” — a set of regions in ventral temporal cortex that respond when someone views text, including real words, pseudowords, letter strings, etc,

….& includes the “visual word form area” (VWFA) 👻
Dehaene 2011: doi.org/10.1016/j.ti...

1

Rodrigo Braga @rodbraga.bsky.social · 18h

So what happens when we read? 📚

Reading begins with vision, so visual sensory input must interface with the transmodal LANG network to produce understanding.

Where is this interface? What is its nature?

1

Rodrigo Braga @rodbraga.bsky.social · 18h

What this transmodal function is, is unclear:

LANG is probably helping convert sensory input into meaning, somehow. See this recent review for a useful framing:

Fedorenko et al. 2024: www.nature.com/articles/s41...

The language network as a natural kind within the broader landscape of the human brain - Nature Reviews Neuroscience

Many brain areas support complex language processing behaviours. In this Review, Fedorenko et al. disentangle the ‘core’ language system as functionally distinct from the perceptual and motor brain areas and knowledge and reasoning systems it closely interacts with during language comprehension and production.

www.nature.com

1