Lightnews — Scholar-powered news

Hudson Golino

@hudsongolino.bsky.social

1.6K followers 620 following 280 posts

Associate Professor of Quantitative Methods at the Department of Psychology - University of Virginia. NLP/LLMs, Network Science, Info Theory, Psychometrics. Web: tinyurl.com/HudsonGolino Web 2: https://r-ega.net

r-ega.net

Posts Media Videos Starter Packs

Pinned

Hudson Golino @hudsongolino.bsky.social · Apr 10

A new approach for LLM Interpretability: Mapping the Emergence of Semantic Structure in GPT-2. See the project details and initial findings below:

1 1

Hudson Golino @hudsongolino.bsky.social · May 9

BREAKTHROUGH COMING NEXT WEEK! After months of very intensive work, we are about to release the results of our "Exploratory Graph Analysis Item Parameters" simulation. We've successfully derived IRT-compatible parameters directly from EGA! Stay tuned, folks!

Hudson Golino @hudsongolino.bsky.social · Apr 22

Pre-print: osf.io/preprints/ps...

OSF

osf.io

Hudson Golino @hudsongolino.bsky.social · Apr 22

This work exemplifies how innovative psychometric methods can reshape our understanding of fundamental psychological constructs. Its implications extend beyond personality assessment to potentially revolutionize how we conceptualize and measure human traits across psychology.

1 1

Hudson Golino @hudsongolino.bsky.social · Apr 22

Our research effectively integrates empirical findings scattered across personality literature into a coherent hierarchical structure. This data-driven framework demonstrates TGA's value for investigating complex psychological constructs and offers a rigorous new perspective on personality taxonomy.

Hudson Golino @hudsongolino.bsky.social · Apr 22

a "Disinhibition" meta-trait at the third level represents a major departure from the traditional five-factor model. 7/n

Hudson Golino @hudsongolino.bsky.social · Apr 22

While some dimensions aligned with traditional IPIP-NEO structure, we found significant deviations. The emergence of novel dimensions like "Sociability," "Integrity," and "Impulsivity" at the second level and 6/n

Hudson Golino @hudsongolino.bsky.social · Apr 22

What makes this approach very interesting is how it addresses longstanding methodological challenges in personality assessment:

Local independence violations
Wording effects
Dimensionality assessment
Structural robustness 5/n

Hudson Golino @hudsongolino.bsky.social · Apr 22

28 first-level dimensions (facets)
6 second-level dimensions (traits)
3 third-level dimensions (meta-traits) 4/n

Hudson Golino @hudsongolino.bsky.social · Apr 22

imposing existing theoretical models. We applied TGA to the open-source 300-item IPIP-NEO dataset with over 149,000 participants and reveal a three-level structure of personality:

1 1

Hudson Golino @hudsongolino.bsky.social · Apr 22

In our preprint (link in the comments) we introduce a new approach to understanding personality structure: Taxonomic Graph Analysis (TGA), is a comprehensive network psychometrics approach that identifies hierarchical personality structures from the bottom up rather than 2/n

Hudson Golino @hudsongolino.bsky.social · Apr 22

Are you interested in hierarchical dimensionality analysis? Here's our new "Taxonomic Graph Analysis" used to model the IPIP-NEO Personality Hierarchy. The project is led by Andrew Samo and Alexander Christensen, with the collaboration of Luis Garrido, Paco Abad, Sam McAbee, and me! 1/n

1 2 15

Hudson Golino @hudsongolino.bsky.social · Apr 10

and
Develop interactive dashboards to explore these findings dynamically.

1 1

Hudson Golino @hudsongolino.bsky.social · Apr 10

Looking ahead, I plan to extend this approach:
Implement a large-scale simulation to see the effects of varying the number of items per dimension
Investigate how adversarial inputs or fine-tuning might shift these representational landscapes,

1 1

Hudson Golino @hudsongolino.bsky.social · Apr 10

By linking internal representations to cognitive functions—ranging from shallow syntactic analysis to deep conceptual abstraction—we can better understand and eventually improve LLM performance, interpretability, and safety.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Implications and Future Directions
This investigation not only advances our understanding of how GPT-2 (and by extension, other large language models) processes language internally but also opens up new avenues for interpretability research.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Global peaks in NMI values,
Functional labels such as “Token Feature Extraction,” “Contextual Integration,” and “Optimal Abstraction Peak,” and
A horizontal transition boundary, precisely indicating where the network’s representational strategy shifts.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Visualization Innovations
One of the project’s central achievements is the development of a mind-blowing NMI heatmap that visually narrates these transitions. The heatmap uses a non-linear Viridis color scale to highlight regions of high semantic coherence and overlays annotations that mark:

Hudson Golino @hudsongolino.bsky.social · Apr 10

Layer 13: Pre-Generation Compression
The final layer reverts to lower NMI values. This “compression” is indicative of the model preparing its internal state for next-token prediction rather than maintaining a rich semantic structure.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Layers 10–12: Conceptual Refinement Layers
The network continues to process and refine these representations. Although NMI values remain high, they reflect a fine-tuning process in conceptual understanding rather than the initial burst of abstraction seen in Layer 9.

Hudson Golino @hudsongolino.bsky.social · Apr 10

The robust, disentangled representations here capture personality constructs in a way that aligns with theoretical expectations of deep semantic abstraction.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Layers 7–9: Optimal Abstraction Layers
This zone is where the magic happens—Layer 9, in particular, consistently shows the highest semantic coherence (bright yellow on the heatmap).

1 1

Hudson Golino @hudsongolino.bsky.social · Apr 10

Layers 4–6: Contextual Integration Zone
A transition begins in these layers as context starts to shape token representations. Notably, Layer 6 marks the first significant rise in NMI (approaching 0.91), suggesting a shift toward meaningful semantic integration.

Hudson Golino @hudsongolino.bsky.social · Apr 10

Layers 1–3: Token Feature Extraction
The earliest layers capture raw token-level properties. Here, the low NMI values (deep blue on the heatmap) indicate that the model primarily encodes lexical-syntactic details without forming robust abstract representations.

Hudson Golino @hudsongolino.bsky.social · Apr 10

By processing personality-related items through GPT-2 and applying DynEGA to the activation patterns, I discovered a striking organization:

Hudson Golino @hudsongolino.bsky.social · Apr 10

and: TEFI (Total Entropy Fit Index): model fit
New Interpretability Roadmap
The result? A Roadmap with embedding dimensions on the x-axis, GPT-2 layers on the y-axis, and NMI as the color gradient — showing where and how semantic structure emerges.