Nari Johnson
@narijohnson.bsky.social
researching AI [evaluation, governance, accountability]
Reposted by Nari Johnson
PhD admissions visits/open houses are starting to happen, and I got a comment on an old Reddit post where I was offering advice, and realized that it's actually really good advice. So here it is! (And this applies whether you've already been admitted to the program or not.) 🧵
February 5, 2026 at 5:26 PM
PhD admissions visits/open houses are starting to happen, and I got a comment on an old Reddit post where I was offering advice, and realized that it's actually really good advice. So here it is! (And this applies whether you've already been admitted to the program or not.) 🧵
Reposted by Nari Johnson
Yep and it gets worse! Owner doesn't even care to remove hundreds of skills which directly instruct the model to install malware
opensourcemalware.com/blog/clawdbo...
opensourcemalware.com/blog/clawdbo...
February 6, 2026 at 1:05 AM
Yep and it gets worse! Owner doesn't even care to remove hundreds of skills which directly instruct the model to install malware
opensourcemalware.com/blog/clawdbo...
opensourcemalware.com/blog/clawdbo...
Reposted by Nari Johnson
Our call for craft and tutorial sessions for #FAccT2026 is now live!
▶️ Craft CfP: facctconference.org/2026/cfpcraf...
▶️ Tutorials CfP: facctconference.org/2026/cft.html
Both kinds of proposals are due March 25!
▶️ Craft CfP: facctconference.org/2026/cfpcraf...
▶️ Tutorials CfP: facctconference.org/2026/cft.html
Both kinds of proposals are due March 25!
February 5, 2026 at 5:39 PM
Our call for craft and tutorial sessions for #FAccT2026 is now live!
▶️ Craft CfP: facctconference.org/2026/cfpcraf...
▶️ Tutorials CfP: facctconference.org/2026/cft.html
Both kinds of proposals are due March 25!
▶️ Craft CfP: facctconference.org/2026/cfpcraf...
▶️ Tutorials CfP: facctconference.org/2026/cft.html
Both kinds of proposals are due March 25!
Reposted by Nari Johnson
🎭 How do LLMs (mis)represent culture?
🧮 How often?
🧠 Misrepresentations = missing knowledge? spoiler: NO!
At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India
📜 arxiv.org/abs/2511.21322
1/10
🧮 How often?
🧠 Misrepresentations = missing knowledge? spoiler: NO!
At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India
📜 arxiv.org/abs/2511.21322
1/10
February 2, 2026 at 9:38 PM
🎭 How do LLMs (mis)represent culture?
🧮 How often?
🧠 Misrepresentations = missing knowledge? spoiler: NO!
At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India
📜 arxiv.org/abs/2511.21322
1/10
🧮 How often?
🧠 Misrepresentations = missing knowledge? spoiler: NO!
At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India
📜 arxiv.org/abs/2511.21322
1/10
Reposted by Nari Johnson
Microsoft Research NYC is hiring a researcher in the space of AI and society!
January 29, 2026 at 11:27 PM
Microsoft Research NYC is hiring a researcher in the space of AI and society!
Reposted by Nari Johnson
A new report by the Center for Tech Responsibility at Brown University and the ACLU uses computational tools to analyze legislative trends on AI across 1,804 state and federal bills, while offering recommendations for how to integrate the technology into policy analysis.
Making Sense of AI Policy Using Computational Tools | TechPolicy.Press
A new report examines how to use computational tools to evaluate policy, with AI policy as a case study.
www.techpolicy.press
January 8, 2026 at 8:56 PM
A new report by the Center for Tech Responsibility at Brown University and the ACLU uses computational tools to analyze legislative trends on AI across 1,804 state and federal bills, while offering recommendations for how to integrate the technology into policy analysis.
Reposted by Nari Johnson
We are studying the sentiments of visual artists towards generative AI in the workplace and their impacts on creative careers. If you're an artist, please consider filling out this recruitment form for access to our survey!
cmu.ca1.qualtrics.com/jfe/form/SV_...
cmu.ca1.qualtrics.com/jfe/form/SV_...
December 19, 2025 at 1:58 AM
We are studying the sentiments of visual artists towards generative AI in the workplace and their impacts on creative careers. If you're an artist, please consider filling out this recruitment form for access to our survey!
cmu.ca1.qualtrics.com/jfe/form/SV_...
cmu.ca1.qualtrics.com/jfe/form/SV_...
Reposted by Nari Johnson
Most LLM evals use API calls or offline inference, testing models in a memory-less silo. Our new Patterns paper shows this misses how LLMs actually behave in real user interfaces, where personalization and interaction history shape responses: arxiv.org/abs/2509.19364
December 12, 2025 at 8:42 PM
Most LLM evals use API calls or offline inference, testing models in a memory-less silo. Our new Patterns paper shows this misses how LLMs actually behave in real user interfaces, where personalization and interaction history shape responses: arxiv.org/abs/2509.19364
Reposted by Nari Johnson
US CAISI is hiring -- the internal govt name for the role is "IT Specialist" but it is effectively a research scientist role!
Salary is $120,579 to - $195,200 per year, and you get to work on AI evaluation within government agencies!
Job posting (**closes EOD 12/28/2025**): lnkd.in/exJgkqr5
Salary is $120,579 to - $195,200 per year, and you get to work on AI evaluation within government agencies!
Job posting (**closes EOD 12/28/2025**): lnkd.in/exJgkqr5
December 11, 2025 at 10:02 PM
US CAISI is hiring -- the internal govt name for the role is "IT Specialist" but it is effectively a research scientist role!
Salary is $120,579 to - $195,200 per year, and you get to work on AI evaluation within government agencies!
Job posting (**closes EOD 12/28/2025**): lnkd.in/exJgkqr5
Salary is $120,579 to - $195,200 per year, and you get to work on AI evaluation within government agencies!
Job posting (**closes EOD 12/28/2025**): lnkd.in/exJgkqr5
Reposted by Nari Johnson
December 8, 2025 at 2:47 PM
Reposted by Nari Johnson
Our team at NIST's Center for AI Standards and Innovation (CAISI) just released a blog post with open questions for AI measurement science:
www.nist.gov/blogs/caisi-...
www.nist.gov/blogs/caisi-...
Accelerating AI Innovation Through Measurement Science
Building gold-standard AI systems requires gold-standard AI measurement science – the scientific study of methods used to assess AI systems’ properties and impacts. NIST works to improve measurements ...
www.nist.gov
December 4, 2025 at 8:17 PM
Our team at NIST's Center for AI Standards and Innovation (CAISI) just released a blog post with open questions for AI measurement science:
www.nist.gov/blogs/caisi-...
www.nist.gov/blogs/caisi-...
Reposted by Nari Johnson
Did you know that one base model is responsible for 94% of model-tagged NSFW AI videos on CivitAI?
This new paper studies how a small number of models power the non-consensual AI video deepfake ecosystem and why their developers could have predicted and mitigated this.
This new paper studies how a small number of models power the non-consensual AI video deepfake ecosystem and why their developers could have predicted and mitigated this.
December 4, 2025 at 5:32 PM
Did you know that one base model is responsible for 94% of model-tagged NSFW AI videos on CivitAI?
This new paper studies how a small number of models power the non-consensual AI video deepfake ecosystem and why their developers could have predicted and mitigated this.
This new paper studies how a small number of models power the non-consensual AI video deepfake ecosystem and why their developers could have predicted and mitigated this.
Reposted by Nari Johnson
I appreciate this sympathetic position
people's feelings of emotional dependency on these "human-like" bots is real. ridiculing them doesn't help anyone
people's feelings of emotional dependency on these "human-like" bots is real. ridiculing them doesn't help anyone
November 28, 2025 at 11:48 PM
I appreciate this sympathetic position
people's feelings of emotional dependency on these "human-like" bots is real. ridiculing them doesn't help anyone
people's feelings of emotional dependency on these "human-like" bots is real. ridiculing them doesn't help anyone
Reposted by Nari Johnson
Can public involvement in AI evaluation improve the science? Or does it compromise quality, speed, cost?
In @pnas.org, Megan Price & I summarize challenges of AI evaluation, review strengths/weaknesses, & suggest how participatory methods can improve the science of AI
www.pnas.org/doi/10.1073/...
In @pnas.org, Megan Price & I summarize challenges of AI evaluation, review strengths/weaknesses, & suggest how participatory methods can improve the science of AI
www.pnas.org/doi/10.1073/...
How public involvement can improve the science of AI | PNAS
As AI systems from decision-making algorithms to generative AI are deployed more widely,
computer scientists and social scientists alike are being ...
www.pnas.org
November 17, 2025 at 12:47 PM
Can public involvement in AI evaluation improve the science? Or does it compromise quality, speed, cost?
In @pnas.org, Megan Price & I summarize challenges of AI evaluation, review strengths/weaknesses, & suggest how participatory methods can improve the science of AI
www.pnas.org/doi/10.1073/...
In @pnas.org, Megan Price & I summarize challenges of AI evaluation, review strengths/weaknesses, & suggest how participatory methods can improve the science of AI
www.pnas.org/doi/10.1073/...
Reposted by Nari Johnson
Can LLMs accurately aggregate information over long, information-dense texts? Not yet…
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
November 7, 2025 at 5:07 PM
Can LLMs accurately aggregate information over long, information-dense texts? Not yet…
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
Reposted by Nari Johnson
📣 Our method for conducting community-based algorithmic impact assessments is now available! We’ve just launched a new section on our website where you can find an extensive toolkit, documentation of our pilots, and a series of reflections on lessons learned. datasociety.net/research/alg...
October 29, 2025 at 7:10 PM
📣 Our method for conducting community-based algorithmic impact assessments is now available! We’ve just launched a new section on our website where you can find an extensive toolkit, documentation of our pilots, and a series of reflections on lessons learned. datasociety.net/research/alg...
Reposted by Nari Johnson
𝐒𝐨𝐜𝐢𝐞𝐭𝐚𝐥 𝐈𝐦𝐩𝐚𝐜𝐭 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 𝐟𝐨𝐫 𝐈𝐧𝐝𝐮𝐬𝐭𝐫𝐲 𝐂𝐨𝐦𝐩𝐮𝐭𝐢𝐧𝐠 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡𝐞𝐫𝐬
🏅 Best Paper Honorable Mention (Top 3% Submissions)
🔗 dl.acm.org/doi/10.1145/...
📆 Wed, 22 Oct | 9:00 AM, CET: Toward More Ethical and Transparent Systems and Environments
🏅 Best Paper Honorable Mention (Top 3% Submissions)
🔗 dl.acm.org/doi/10.1145/...
📆 Wed, 22 Oct | 9:00 AM, CET: Toward More Ethical and Transparent Systems and Environments
Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work | Proceedings of the ACM on Human-Computer Interaction
Recent years have witnessed increasing calls for computing researchers to grapple with the societal impacts of their work. Tools such as impact assessments have gained prominence as a method to uncover potential impacts, and a number of publication ...
dl.acm.org
October 19, 2025 at 1:49 PM
𝐒𝐨𝐜𝐢𝐞𝐭𝐚𝐥 𝐈𝐦𝐩𝐚𝐜𝐭 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 𝐟𝐨𝐫 𝐈𝐧𝐝𝐮𝐬𝐭𝐫𝐲 𝐂𝐨𝐦𝐩𝐮𝐭𝐢𝐧𝐠 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡𝐞𝐫𝐬
🏅 Best Paper Honorable Mention (Top 3% Submissions)
🔗 dl.acm.org/doi/10.1145/...
📆 Wed, 22 Oct | 9:00 AM, CET: Toward More Ethical and Transparent Systems and Environments
🏅 Best Paper Honorable Mention (Top 3% Submissions)
🔗 dl.acm.org/doi/10.1145/...
📆 Wed, 22 Oct | 9:00 AM, CET: Toward More Ethical and Transparent Systems and Environments
Reposted by Nari Johnson
💡Can we trust synthetic data for statistical inference?
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
October 10, 2025 at 4:12 PM
💡Can we trust synthetic data for statistical inference?
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
We show that synthetic data (e.g., LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moment residuals of synthetic data and those of real data
Reposted by Nari Johnson
Our Responsible AI team at Apple is looking for spring/summer 2026 PhD research interns! Please apply at jobs.apple.com/en-us/detail... and email [email protected]. Do not send extra info (e.g., CV), just drop us a line so we can find your application in the central pool!
Machine Learning / AI Internships - Jobs - Careers at Apple
Apply for a Machine Learning / AI Internships job at Apple. Read about the role and find out if it’s right for you.
jobs.apple.com
October 10, 2025 at 2:28 AM
Our Responsible AI team at Apple is looking for spring/summer 2026 PhD research interns! Please apply at jobs.apple.com/en-us/detail... and email [email protected]. Do not send extra info (e.g., CV), just drop us a line so we can find your application in the central pool!
Reposted by Nari Johnson
✨I’m on the academic job market ✨
I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻💻💪🏽💥
I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action
More info: cella.io
I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻💻💪🏽💥
I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action
More info: cella.io
Cella M. Sum –
cella.io
October 9, 2025 at 2:39 PM
✨I’m on the academic job market ✨
I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻💻💪🏽💥
I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action
More info: cella.io
I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻💻💪🏽💥
I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action
More info: cella.io
Reposted by Nari Johnson
🌟 If you’re applying to CMU SCS PhD programs, and come from a background that would bring additional dimensions to the CMU community, our PhD students are here to help!
Apply to the Graduate Applicant Support Program by Oct 13 to receive feedback on your application materials:
Apply to the Graduate Applicant Support Program by Oct 13 to receive feedback on your application materials:
September 24, 2025 at 4:00 PM
🌟 If you’re applying to CMU SCS PhD programs, and come from a background that would bring additional dimensions to the CMU community, our PhD students are here to help!
Apply to the Graduate Applicant Support Program by Oct 13 to receive feedback on your application materials:
Apply to the Graduate Applicant Support Program by Oct 13 to receive feedback on your application materials:
Reposted by Nari Johnson
📌📌📌
I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com
I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com
Stephen Casper
Visit the post for more.
stephencasper.com
September 4, 2025 at 3:39 AM
📌📌📌
I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com
I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com
Reposted by Nari Johnson
📢2026 Fellowship applications are OPEN!📢
If you are someone looking to inform technology policy through rigorous original reporting or policy analyses, we want to hear from you!
Apply here: airtable.com/appIrc1F9M5d...
If you are someone looking to inform technology policy through rigorous original reporting or policy analyses, we want to hear from you!
Apply here: airtable.com/appIrc1F9M5d...
September 4, 2025 at 11:47 AM
📢2026 Fellowship applications are OPEN!📢
If you are someone looking to inform technology policy through rigorous original reporting or policy analyses, we want to hear from you!
Apply here: airtable.com/appIrc1F9M5d...
If you are someone looking to inform technology policy through rigorous original reporting or policy analyses, we want to hear from you!
Apply here: airtable.com/appIrc1F9M5d...
Reposted by Nari Johnson
What can #CSCW learn from tech workers who have been involved in collective action and unionization about how to make transformative change within our field?
My new #CSCW2025 paper with Mona Wang, Anna Konvicka, and Sarah Fox seeks to answer this question.
Pre-print: arxiv.org/pdf/2508.12579
My new #CSCW2025 paper with Mona Wang, Anna Konvicka, and Sarah Fox seeks to answer this question.
Pre-print: arxiv.org/pdf/2508.12579
August 28, 2025 at 2:14 PM
What can #CSCW learn from tech workers who have been involved in collective action and unionization about how to make transformative change within our field?
My new #CSCW2025 paper with Mona Wang, Anna Konvicka, and Sarah Fox seeks to answer this question.
Pre-print: arxiv.org/pdf/2508.12579
My new #CSCW2025 paper with Mona Wang, Anna Konvicka, and Sarah Fox seeks to answer this question.
Pre-print: arxiv.org/pdf/2508.12579
Reposted by Nari Johnson
The exchanges between Adam and ChatGPT are devastating. This, in my mind, is the worst one.
One of his last messages was a photo of the noose hung in his bedroom closet, asking if it was "good." ChatGPT offered a technical analysis of the set up and told him it 'could potentially suspend a human."
One of his last messages was a photo of the noose hung in his bedroom closet, asking if it was "good." ChatGPT offered a technical analysis of the set up and told him it 'could potentially suspend a human."
August 26, 2025 at 1:37 PM
The exchanges between Adam and ChatGPT are devastating. This, in my mind, is the worst one.
One of his last messages was a photo of the noose hung in his bedroom closet, asking if it was "good." ChatGPT offered a technical analysis of the set up and told him it 'could potentially suspend a human."
One of his last messages was a photo of the noose hung in his bedroom closet, asking if it was "good." ChatGPT offered a technical analysis of the set up and told him it 'could potentially suspend a human."