Lightnews — Scholar-powered news

Reposted by Robin Jia

Yanai Elazar @yanai.bsky.social · Aug 2

I had a lot of fun contemplating about memorization questions at the @l2m2workshop.bsky.social panel yesterday together with Niloofar Mireshghallah and Reza Shokri, moderated by
@pietrolesci.bsky.social who did a fantastic job!
#ACL2025

1 2 12

Robin Jia @robinjia.bsky.social · Jul 30

Paper link: arxiv.org/abs/2501.14883

Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics

Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge this optimism by th...

arxiv.org

Robin Jia @robinjia.bsky.social · Jul 30

Automatic metrics for assessing factuality are easy to run and commonly used, but do they work? In < 1 hour, come find the answer at poster 349 in Hall X4, where I’ll be presenting @ameyagodbole.bsky.social ‘s work uncovering inconsistencies, errors, and biases of factuality metrics!

1 2

Robin Jia @robinjia.bsky.social · Jul 25

I’ll be at ACL 2025 next week where my group has papers on evaluating evaluation metrics, watermarking training data, and mechanistic interpretability. I’ll also be co-organizing the first Workshop on LLM Memorization @l2m2workshop.bsky.social on Friday. Hope to see lots of folks there!

2

Reposted by Robin Jia

Jesse Thomason @thomason.bsky.social · May 1

Come by @naaclmeeting.bsky.social Poster 6 in Hall 3 from 4-530pm today to see @billzhu.bsky.social's and Ishika Singh's work with me and @robinjia.bsky.social on PSALM: autonomously inducing symbolic pre- and post-conditions of actions with LLMs, symbolic planning, and text environment interaction!

LLMs can propose plans and generate action semantics, but struggle with state tracking. Symbolic planners leverage specialized search algorithms, but require predefined action semantics for the environment.
PSALM integrates the strengths of both.

1 1 5

Robin Jia @robinjia.bsky.social · Apr 30

Check out @billzhu.bsky.social ‘s excellent work on combining LLMs with symbolic planners at NAACL on Thursday! I will also be at NAACL Friday-Sunday, looking forward to chatting about LLM memorization, interpretability, evaluation, and more

Wang Bill Zhu @billzhu.bsky.social · Apr 30

At @naaclmeeting.bsky.social this week! I’ll be presenting our work on LLM domain induction with @thomason.bsky.social on Thu (5/1) at 4pm in Hall 3, Section I.

Would love to connect and chat about LLM planning, reasoning, AI4Science, multimodal stuff, or anything else. Feel free to DM!

3

Reposted by Robin Jia

Wang Bill Zhu @billzhu.bsky.social · Apr 30

At @naaclmeeting.bsky.social this week! I’ll be presenting our work on LLM domain induction with @thomason.bsky.social on Thu (5/1) at 4pm in Hall 3, Section I.

Would love to connect and chat about LLM planning, reasoning, AI4Science, multimodal stuff, or anything else. Feel free to DM!

3 4

Robin Jia @robinjia.bsky.social · Feb 17

Sounds like arxiv.org/abs/2102.07033

PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

Open-domain Question Answering models which directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in terms of speed and memory compare...

arxiv.org

1

Reposted by Robin Jia

Deqing Fu @deqing.bsky.social · Feb 8

Excited to share that my intern work at Meta GenAI is accepted to @iclr-conf.bsky.social #ICLR2025

Introducing TLDR: Token-Level Detective Reward Model For Large Vision Language Models.

TLDR provides fine-grained annotations to
each text token.

🔗arXiv: arxiv.org/abs/2410.04734

1 1 5

Robin Jia @robinjia.bsky.social · Jan 27

Our workshop on LLM Memorization is coming to ACL 2025! The call for papers is out, please submit both archival and non-archival (work in progress or already published) papers

The First Workshop on Large Language Model Memorization (L2M2) @l2m2workshop.bsky.social · Jan 27

📢 The First Workshop on Large Language Model Memorization (L2M2) will be co-located with
@aclmeeting.bsky.social in Vienna 🎉

💡 L2M2 brings together researchers to explore memorization from multiple angles. Whether it's text-only LLMs or Vision-language models, we want to hear from you! 🌍

3 9

Robin Jia @robinjia.bsky.social · Dec 9

Links & presentation times:
1. Fourier Features: arxiv.org/abs/2406.03445 Thu, 4:30pm
2. TF + ICL: arxiv.org/abs/2310.17086 Fri, 11am
3. Backdoor detection: arxiv.org/abs/2409.00399 Sat, 1:44pm at AdvML Frontiers
4. LLMs + PDDL: arxiv.org/abs/2406.02791 Sun, 2:30pm at OWA workshop

Pre-trained Large Language Models Use Fourier Features to Compute Addition

Pre-trained large language models (LLMs) exhibit impressive mathematical reasoning capabilities, yet how they compute basic arithmetic, such as addition, remains unclear. This paper shows that pre-tra...

arxiv.org

Robin Jia @robinjia.bsky.social · Dec 9

I'll be at #NeurIPS2024! My group has papers analyzing how LLMs use Fourier Features for arithmetic and how TFs learn higher-order optimization for ICL (led by @deqing.bsky.social), plus workshop papers on backdoor detection and LLMs + PDDL (led by @billzhu.bsky.social)

1 3 23

Reposted by Robin Jia

Maria Antoniak @mariaa.bsky.social · Nov 4

A starter pack for #NLP #NLProc researchers! 🎉

go.bsky.app/SngwGeS

45 100 250

Reposted by Robin Jia

Matthew Finlayson @mattf1n.bsky.social · Nov 12

USC NLP folks are on Bluesky!
Follow my amazing colleagues here

go.bsky.app/KUwSZ6W

3 5 17

Reposted by Robin Jia

Sameer Singh @sameer-singh.bsky.social · Nov 19

Started a SoCal AI/ML/NLP researchers starter pack! It's a bit sparse right now, and perhaps more NLP heavy, but hey, nominate yourself and others! go.bsky.app/6QckPj9

17 8 43