Studying the intersection of AI, social media, and politics.
Polarization, misinformation, radicalization, digital platforms, social complexity.
Worth every penny! (It’s open access 😉)
Worth every penny! (It’s open access 😉)
Really happy you liked the book! :)
Really happy you liked the book! :)
Democrats still have users but rarely visit the site.
I would point you to the preprint I put up for more details and better versions of the figures: www.arxiv.org/abs/2510.25417
Democrats still have users but rarely visit the site.
I would point you to the preprint I put up for more details and better versions of the figures: www.arxiv.org/abs/2510.25417
Colleagues who do excellent work in this field, and might find these results interesting:
@mbernst.bsky.social
@robbwiller.bsky.social
@joon-s-pk.bsky.social
@janalasser.bsky.social
@dgarcia.eu
@aaronshaw.bsky.social
Colleagues who do excellent work in this field, and might find these results interesting:
@mbernst.bsky.social
@robbwiller.bsky.social
@joon-s-pk.bsky.social
@janalasser.bsky.social
@dgarcia.eu
@aaronshaw.bsky.social
Paper (preprint): arxiv.org/abs/2511.04195
Happy to share prompts, configs, and analysis scripts.
Paper (preprint): arxiv.org/abs/2511.04195
Happy to share prompts, configs, and analysis scripts.
• LLMs are worse stand-ins for humans than they may appear.
• Don’t rely on human judges.
• Measure detectability and meaning.
• Expect a style–meaning trade-off.
• Use examples + context, not personas.
• Affect is still the biggest giveaway.
• LLMs are worse stand-ins for humans than they may appear.
• Don’t rely on human judges.
• Measure detectability and meaning.
• Expect a style–meaning trade-off.
• Use examples + context, not personas.
• Affect is still the biggest giveaway.
🎭 When models sound more human, they drift from what people actually say.
🧠 When they match meaning better, they sound less human.
Style or meaning — you have to pick one.
🎭 When models sound more human, they drift from what people actually say.
🧠 When they match meaning better, they sound less human.
Style or meaning — you have to pick one.
Not personas. And fine-tuning? Not always.
The real improvements came from:
✅ Providing stylistic examples of the user
✅ Adding context retrieval from past posts
Together, these reduced detectability by 4-16 percentage points.
Not personas. And fine-tuning? Not always.
The real improvements came from:
✅ Providing stylistic examples of the user
✅ Adding context retrieval from past posts
Together, these reduced detectability by 4-16 percentage points.
⚙️ Instruction-tuned models — the ones fine-tuned to follow prompts — are easier to detect than their base counterparts.
📏 Model size doesn’t help: even 70B models don’t sound more human.
⚙️ Instruction-tuned models — the ones fine-tuned to follow prompts — are easier to detect than their base counterparts.
📏 Model size doesn’t help: even 70B models don’t sound more human.
❤️ Affective tone and emotion — the clearest tell.
✍️ Stylistic markers — average word length, toxicity, hashtags, emojis.
🧠 Topic profiles — especially on Reddit, where conversations are more diverse and nuanced.
❤️ Affective tone and emotion — the clearest tell.
✍️ Stylistic markers — average word length, toxicity, hashtags, emojis.
🧠 Topic profiles — especially on Reddit, where conversations are more diverse and nuanced.
Even short social media posts written by LLMs are readily distinguishable.
Our BERT-based classifier spots AI with 70–80% accuracy across X, Bluesky, and Reddit.
LLMs are much less human-like than they may seem.
Even short social media posts written by LLMs are readily distinguishable.
Our BERT-based classifier spots AI with 70–80% accuracy across X, Bluesky, and Reddit.
LLMs are much less human-like than they may seem.
We benchmark 9 open-weight LLMs across 5 calibration strategies:
👤 Persona
✍️ Stylistic examples
🧩 Context retrieval
⚙️ Fine-tuning
🎯 Post-generation selection
We benchmark 9 open-weight LLMs across 5 calibration strategies:
👤 Persona
✍️ Stylistic examples
🧩 Context retrieval
⚙️ Fine-tuning
🎯 Post-generation selection
We use data from X (Twitter), Bluesky, and Reddit.
This task is arguably what LLMs should do best: they are literally trained on this data!
We use data from X (Twitter), Bluesky, and Reddit.
This task is arguably what LLMs should do best: they are literally trained on this data!
🕵️♂️ Detectability — can an ML classifier tell AI from human?
🧠 Semantic fidelity — does it mean the same thing?
✍️ Interpretable linguistic features — style, tone, topics.
🕵️♂️ Detectability — can an ML classifier tell AI from human?
🧠 Semantic fidelity — does it mean the same thing?
✍️ Interpretable linguistic features — style, tone, topics.
But humans are actually really bad at this task: we are subjective, scale poorly, and very easy to fool.
We need something more rigorous.
But humans are actually really bad at this task: we are subjective, scale poorly, and very easy to fool.
We need something more rigorous.