liuyulu.bsky.social
Yu Lu Liu
@liuyulu.bsky.social
1K followers 890 following 34 posts
PhD student at Johns Hopkins University Alumni from McGill University & MILA Working on NLP Evaluation, Responsible AI, Human-AI interaction she/her 🇨🇦
Posts Media Videos Starter Packs
Pinned
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Dec 16
Human-centered Evalulation and Auditing of Language models (HEAL) workshop is back for #CHI2025, with this year's special theme: “Mind the Context”! Come join us on this bridge between #HCI and #NLProc!

Workshop submission deadline: Feb 17 AoE
More info at heal-workshop.github.io.
The image includes a shortened call for participation that reads: "We welcome participants who work on topics related to supporting human-centered evaluation and auditing of language models. Topics of interest include, but not limited to: - Empirical understanding of stakeholders' needs and goals of LLM evaluation and auditing - Human-centered evaluation and auditing methods for LLMs - Tools, processes, and guidelines for LLM evaluation and auditing - Discussion of regulatory measures and public policies for LLM auditing - Ethics in LLM evaluation and auditing Special Theme: Mind the Context. We invite authors to engage with specific contexts in LLM evaluation and auditing. This theme could involve various topics: the usage contexts of LLMs, the context of the evaluation/auditing itself, and more! The term ''context'' is purposefully left open for interpretation! The image also includes pictures of workshop organizers, who are: Yu Lu Liu, Wesley Hanwen Deng, Michelle S. Lam, Motahhare Eslami, Juho Kim, Q. Vera Liao, Wei Xu, Jekaterina Novikova, and Ziang Xiao.
2 10 44
Reposted by Yu Lu Liu
aolteanu.bsky.social
Alexandra Olteanu @aolteanu.bsky.social · 9d
This was accepted to #NeurIPS 🎉🎊

TL;DR Impoverished notions of rigor can have a formative impact on AI work. We argue for a broader conception of what rigorous work should entail & go beyond methodological issues to include epistemic, normative, conceptual, reporting & interpretative considerations
aolteanu.bsky.social
Alexandra Olteanu @aolteanu.bsky.social · Jun 18
We have to talk about rigor in AI work and what it should entail. The reality is that impoverished notions of rigor do not only lead to some one-off undesirable outcomes but can have a deeply formative impact on the scientific integrity and quality of both AI research and practice 1/
Print screen of the first page of a paper pre-print titled "Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor" by Olteanu et al. Paper abstract: "In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception -- in addition to a more expansive understanding of (1) methodological rigor -- should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community's work by researchers, policymakers, journalists, and other stakeholders."
1 8 25
Reposted by Yu Lu Liu
ziangxiao.bsky.social
Ziang Xiao @ziangxiao.bsky.social · Apr 25
We are excited to kick off the 2nd HEAL workshop tomorrow at #CHI2025. Dr. Su Lin Blodgett and Dr. Gagan Bansal from MSR will be our keynote speakers!

Welcome new and old friends! See you at G221!

All accepted papers: tinyurl.com/bdfpjcr4
Dr. Su Lin Blodgett and Dr. Gagan Bansal will be the keynote speakers of the 2nd HEAL workshop @CHI25
2 6
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Apr 3
😅
1
Reposted by Yu Lu Liu
jhuclsp.bsky.social
JHU CLSP @jhuclsp.bsky.social · Mar 7
Bringing together our incredible current and admitted students—future leaders, innovators, and changemakers!
1 6
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Feb 13
📣 DEADLINE EXTENSION 📣

By popular request, HEAL workshop submission deadline is extended to Feb 24 AOE!

Reminder that we welcome a wide range of submissions: position papers, lit reviews, encore of published work, etc.

Looking forward to your submissions!
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Dec 16
Human-centered Evalulation and Auditing of Language models (HEAL) workshop is back for #CHI2025, with this year's special theme: “Mind the Context”! Come join us on this bridge between #HCI and #NLProc!

Workshop submission deadline: Feb 17 AoE
More info at heal-workshop.github.io.
The image includes a shortened call for participation that reads: "We welcome participants who work on topics related to supporting human-centered evaluation and auditing of language models. Topics of interest include, but not limited to: - Empirical understanding of stakeholders' needs and goals of LLM evaluation and auditing - Human-centered evaluation and auditing methods for LLMs - Tools, processes, and guidelines for LLM evaluation and auditing - Discussion of regulatory measures and public policies for LLM auditing - Ethics in LLM evaluation and auditing Special Theme: Mind the Context. We invite authors to engage with specific contexts in LLM evaluation and auditing. This theme could involve various topics: the usage contexts of LLMs, the context of the evaluation/auditing itself, and more! The term ''context'' is purposefully left open for interpretation! The image also includes pictures of workshop organizers, who are: Yu Lu Liu, Wesley Hanwen Deng, Michelle S. Lam, Motahhare Eslami, Juho Kim, Q. Vera Liao, Wei Xu, Jekaterina Novikova, and Ziang Xiao.
4
Reposted by Yu Lu Liu
nikhilsksharma.bsky.social
Nikhil Sharma ༗ @nikhilsksharma.bsky.social · Jan 31
Thrilled that our paper Faux Polyglot has been accepted to #NAACL2025 main! 🚀
We show that multilingual RAG creates language-specific information cocoons and amplifies perspectives and facts in the dominant language, especially when handling knowledge conflicts.
📜 arxiv.org/abs/2407.05502
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models
With Retrieval Augmented Generation (RAG), Large Language Models (LLMs) are playing a pivotal role in information search and are being adopted globally. Although the multilingual capability of LLMs of...
arxiv.org
1 3 10
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Jan 22
The submission deadline is in less than a month! We welcome encore submissions, so consider submitting your work regardless of whether it's been accepted or not #chi2025 😉
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Dec 16
Human-centered Evalulation and Auditing of Language models (HEAL) workshop is back for #CHI2025, with this year's special theme: “Mind the Context”! Come join us on this bridge between #HCI and #NLProc!

Workshop submission deadline: Feb 17 AoE
More info at heal-workshop.github.io.
The image includes a shortened call for participation that reads: "We welcome participants who work on topics related to supporting human-centered evaluation and auditing of language models. Topics of interest include, but not limited to: - Empirical understanding of stakeholders' needs and goals of LLM evaluation and auditing - Human-centered evaluation and auditing methods for LLMs - Tools, processes, and guidelines for LLM evaluation and auditing - Discussion of regulatory measures and public policies for LLM auditing - Ethics in LLM evaluation and auditing Special Theme: Mind the Context. We invite authors to engage with specific contexts in LLM evaluation and auditing. This theme could involve various topics: the usage contexts of LLMs, the context of the evaluation/auditing itself, and more! The term ''context'' is purposefully left open for interpretation! The image also includes pictures of workshop organizers, who are: Yu Lu Liu, Wesley Hanwen Deng, Michelle S. Lam, Motahhare Eslami, Juho Kim, Q. Vera Liao, Wei Xu, Jekaterina Novikova, and Ziang Xiao.
1 8
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Dec 16
An awesome team of organizers: @wesleydeng.bsky.social, @mlam.bsky.social, @juhokim.bsky.social, @qveraliao.bsky.social, @cocoweixu.bsky.social, @ziangxiao.bsky.social, Motahhare Eslami, and Jekaterina Novikova!
7
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Dec 16
Human-centered Evalulation and Auditing of Language models (HEAL) workshop is back for #CHI2025, with this year's special theme: “Mind the Context”! Come join us on this bridge between #HCI and #NLProc!

Workshop submission deadline: Feb 17 AoE
More info at heal-workshop.github.io.
The image includes a shortened call for participation that reads: "We welcome participants who work on topics related to supporting human-centered evaluation and auditing of language models. Topics of interest include, but not limited to: - Empirical understanding of stakeholders' needs and goals of LLM evaluation and auditing - Human-centered evaluation and auditing methods for LLMs - Tools, processes, and guidelines for LLM evaluation and auditing - Discussion of regulatory measures and public policies for LLM auditing - Ethics in LLM evaluation and auditing Special Theme: Mind the Context. We invite authors to engage with specific contexts in LLM evaluation and auditing. This theme could involve various topics: the usage contexts of LLMs, the context of the evaluation/auditing itself, and more! The term ''context'' is purposefully left open for interpretation! The image also includes pictures of workshop organizers, who are: Yu Lu Liu, Wesley Hanwen Deng, Michelle S. Lam, Motahhare Eslami, Juho Kim, Q. Vera Liao, Wei Xu, Jekaterina Novikova, and Ziang Xiao.
2 10 44
Reposted by Yu Lu Liu
hannawallach.bsky.social
Hanna Wallach @hannawallach.bsky.social · Dec 2
Super excited to announce that @msftresearch.bsky.social's FATE group, Sociotechnical Alignment Center, and friends have several workshop papers at next week's @neuripsconf.bsky.social. A short thread about (some of) these papers below... #NeurIPS2024
1 13 58
Reposted by Yu Lu Liu
aolteanu.bsky.social
Alexandra Olteanu @aolteanu.bsky.social · Dec 5
📣 📣 Interested in an internship on human-centred AI, human agency, AI evaluation & the impacts of AI systems? Our team/FATE MLT (Su Lin Blodgett, @qveraliao.bsky.social & I) is looking for a few summer interns 🎉 Apply by Jan 10 for full consideration: jobs.careers.microsoft.com/global/en/jo...
10 22
Reposted by Yu Lu Liu
Dr. Arjun @arjunsubgraph.bsky.social · Dec 4
Seeing cool works on metrology and measurement modeling for NLP!

So I wanted to port over the thread our ACL 2023 Findings paper (arxiv.org/abs/2305.09022) on conceptualizations of NLP tasks and measurements of performance! Work with Eric Yuan, @haldaume3.bsky.social, and Su Lin Blodgett. (1/n)
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Progress in NLP is increasingly measured through benchmarks; hence, contextualizing progress requires understanding when and why practitioners may disagree about the validity of benchmarks. We develop...
arxiv.org
1 7 18
Reposted by Yu Lu Liu
eryk.bsky.social
Eryk Salvaggio @eryk.bsky.social · Dec 2
I am collecting examples of the most thoughtful writing about generative AI published in 2024. What’s yours? They can be insightful for commentary, smart critique, or just because it shifted the conversation. I’ll post some of mine below as I go through them. #criticalAI
98 120 350
Reposted by Yu Lu Liu
aolteanu.bsky.social
Alexandra Olteanu @aolteanu.bsky.social · Nov 30
Created a small starter pack including folks whose work I believe contributes to more rigorous and grounded AI research -- I'll grow this slowly and likely move it to a list at some point :) go.bsky.app/P86UbQw
1 5 12
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Nov 28
Added!
1
Reposted by Yu Lu Liu
cfiesler.bsky.social
Dr. Casey Fiesler @cfiesler.bsky.social · Nov 27
Hi, so I've spent the past almost-decade studying research uses of public social media data, like e.g. ML researchers using content from Twitter, Reddit, and Mastodon.

Anyway, buckle up this is about to be a VERY long thread with lots of thoughts and links to papers. 🧵
danielvanstrien.bsky.social
Daniel van Strien @danielvanstrien.bsky.social · Nov 26
First dataset for the new @huggingface.bsky.social @bsky.app community organisation: one-million-bluesky-posts 🦋

📊 1M public posts from Bluesky's firehose API
🔍 Includes text, metadata, and language predictions
🔬 Perfect to experiment with using ML for Bluesky 🤗

huggingface.co/datasets/blu...
bluesky-community/one-million-bluesky-posts · Datasets at Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
59 450 970
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Nov 27
Added!
1
Reposted by Yu Lu Liu
qveraliao.bsky.social
Vera Liao @qveraliao.bsky.social · Nov 26
Had a lot of fun teaching a tutorial on Human-Centered Evaluation of Language Technologies at #EMNLP2024, w/ @ziangxiao.bsky.social, Su Lin Blodgett, and Jackie Cheung

We just posted the slides on our tutorial website: human-centered-eval.github.io
Human-Centered Eval@EMNLP24
human-centered-eval.github.io
3 13
Reposted by Yu Lu Liu
ankareuel.bsky.social
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Nov 25
🚨 NeurIPS 2024 Spotlight
Did you know we lack standards for AI benchmarks, despite their role in tracking progress, comparing models, and shaping policy? 🤯 Enter BetterBench–our framework with 46 criteria to assess benchmark quality: betterbench.stanford.edu 1/x
5 25 140
Reposted by Yu Lu Liu
mcgill-nlp.bsky.social
McGill NLP @mcgill-nlp.bsky.social · Nov 24
It turns out we had even more papers at EMNLP!

Let's complete the list with three more🧵
mcgill-nlp.bsky.social
McGill NLP @mcgill-nlp.bsky.social · Nov 23
Our lab members recently presented 3 papers at @emnlpmeeting.bsky.social in Miami ☀️ 📜

From interpretability to bias/fairness and cultural understanding -> 🧵
1 4 14
Reposted by Yu Lu Liu
mcgill-nlp.bsky.social
McGill NLP @mcgill-nlp.bsky.social · Nov 23
Our lab members recently presented 3 papers at @emnlpmeeting.bsky.social in Miami ☀️ 📜

From interpretability to bias/fairness and cultural understanding -> 🧵
1 6 19
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Nov 23
Added!
1
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Nov 23
Added!
liuyulu.bsky.social
Yu Lu Liu @liuyulu.bsky.social · Nov 23
Added!
1