Lightnews — Scholar-powered news

Akari Asai @akariasai.bsky.social · May 16

Honored to be named to the Forbes 30 Under 30 Asia 2025 in Science!
Grateful for the recognition of my Ph.D. work on Retrieval-Augmented LMs, and excited to keep pushing the boundaries of reliable and efficient language models.
🔗 forbes.com/30-under-30/...
More updates soon… 👀

Forbes 30 Under 30 2025: Healthcare & Science

Discovering new worlds, in our cells and outer space.

forbes.com

9

Akari Asai @akariasai.bsky.social · Apr 22

Sad to miss #ICLR2025 this year, but my amazing co-authors will be there in person to present Pangea!
neulab.github.io/Pangea/
I’ll be at the Foundation Models for Science conference at Simons Foundation, NYC next week, then heading to NAACL (more details soon).
Let’s catch up if you’re around!✨

1 7

Akari Asai @akariasai.bsky.social · Apr 22

Real user queries often look different from the clean, concise ones in academic benchmarks - ambiguity, full of typos, and much less readable.
We show that even strong RAG systems quickly break under these conditions.
Awesome project led by
@neelbhandari.bsky.social and @tianyucao.bsky.social!!

Neel Bhandari @neelbhandari.bsky.social · Apr 17

1/🚨 𝗡𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗮𝗹𝗲𝗿𝘁 🚨
RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style?

We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵

1 6

Reposted by Akari Asai

Stella Li @stellali.bsky.social · Dec 6

31% of US adults use generative AI for healthcare 🤯But most AI systems answer questions assertively—even when they don’t have the necessary context. Introducing #MediQ a framework that enables LLMs to recognize uncertainty🤔and ask the right questions❓when info is missing: 🧵

2 14 68

Akari Asai @akariasai.bsky.social · Dec 8

CopyBench (EMNLP 2024, led by @tomchen0112.bsky.social)
Oral at regulatableml.github.io & Poster at redteaming-gen-ai.github.io

tldr: We benchmarked LLMs' literal/non-literal copying of copyrighted content—risks found even in 8B models.

Detais: www.arxiv.org/abs/2407.07087

The 2nd Workshop on Regulatable ML @NeurIPS2024

Towards Bridging the Gaps between Machine Learning Research and Regulations

regulatableml.github.io

2

Akari Asai @akariasai.bsky.social · Dec 8

MassiveDS (led by @rulinshao.bsky.social) Wednesday Poster at 11-2 pm at West Ballroom#7203

TLDR: We demonstrated scaling retrieval corpora of Retrieval-Augmented LMs to 1.4T helps & achieves more compute-optimal scaling

Details: retrievalscaling.github.io

1 2

Akari Asai @akariasai.bsky.social · Dec 8

Excited to attend #NeurIPS2024 in person! I’ll be presenting MassiveDS and CopyBench. Details below 🧵👇

Let’s catch up and chat about:
- LLMs & Retrieval-Augmented/Augmented LMs
- LLM Applications for science (e.g., OpenScholar) & others
- Ph.D./faculty apps
...and more!

1 1 24

Akari Asai @akariasai.bsky.social · Dec 6

Oh that's a screenshot of my website. Here's link to my CV akariasai.github.io/assets/pdf/a...

akariasai.github.io

1 1

Akari Asai @akariasai.bsky.social · Dec 4

I would love to hear about any opportunities that might be a good fit!! You can find my contact info and CV on my website. akariasai.github.io. I am attending NeurIPS in person so let’s chat!

Akari Asai

A 5th year Ph.D. student at University of Washington, focusing on NLP and ML.

akariasai.github.io

1

Akari Asai @akariasai.bsky.social · Dec 4

🏆 Recognition & Impact: My work has earned EECS Rising Stars 2022, the MIT Tech Review Innovator Award (Japan 2024), paper awards at ACL & NeurIPS, and the IBM Fellowship. My work has been featured in medias like MIT Tech Review, Forbes and VentureBeat.

1 1

Akari Asai @akariasai.bsky.social · Dec 4

🌍 Making Real-World Impacts
Retrieval-Augmented LMs tackle critical challenges like:
1️⃣ Unreliable LMs in expert domains
2️⃣ Information access inequity across languages
I launched OpenScholar for scientific synthesis—20k+ demo requests in week 1! Details: allenai.org/blog/opensch...

Ai2 OpenScholar: Scientific literature synthesis with retrieval-augmented language models | Ai2

Ai2’s & UW’s OpenScholar, a retrieval-augmented LM, helps scientists navigate and synthesize scientific literature.

allenai.org

1

Akari Asai @akariasai.bsky.social · Dec 4

🛠 Building the Foundations:
Retrieval-augmented LMs need more than off-the-shelf models. I developed advanced training/inference algorithms & architectures, including Self-RAG (ICLR 2024 Oral; NeurIPS Workshop Hon. Mention) for adaptive retrieval & self-critique.
Learn more:
selfrag.github.io

Self-RAG: Learning to Retrieve, Generate and Critique through Self-Reflection

Self-RAG: Learning to Retrieve, Generate and Critique through Self-Reflection.

selfrag.github.io

1

Akari Asai @akariasai.bsky.social · Dec 4

🔍 Establishing the Necessity
My work showed that scaling LLMs alone doesn’t solve issues like hallucinations or obsolete knowledge and is compute suboptimal, and Retrieval-Augmented LMs address these challenges. See our ACL 2023 Best Video Award paper:
aclanthology.org/2023.acl-lon...

When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

aclanthology.org

1 1

Akari Asai @akariasai.bsky.social · Dec 4

I’m on the academic job market this year! I’m completing my @uwcse.bsky.social @uwnlp.bsky.social Ph.D. (2025), focusing on overcoming LLM limitations like hallucinations, by building new LMs.
My Ph.D. work focuses on Retrieval-Augmented LMs to create more reliable AI systems 🧵

3 17 70

Reposted by Akari Asai

Kyle Lo @ COLM 2025 🍁 @kylelo.bsky.social · Nov 19

congrats @akariasai.bsky.social:

🔬 retrieval augmented LM for science literature
🧬 open data, weights, index, code, etc
⚗️ new eval suite for science literature tasks
🔭 demo to play w the model

encourage checking out to see what scientific LMs can/cant do today w open research artifacts

Akari Asai @akariasai.bsky.social · Nov 19

1/ Introducing ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge 📚
@uwnlp.bsky.social & Ai2
With open models & 45M-paper datastores, it outperforms proprietary systems & match human experts.
Try out our demo!
openscholar.allen.ai

2 1 18

Reposted by Akari Asai

Luca Soldaini 🎀 @soldaini.net · Nov 19

Super exciting RAG prototype @akariasai.bsky.social build on top of Semantic Scholar!

I love how it returns competent research answers for seemingly out CS domain questions, eg “what’s a bell?” openscholar.allen.ai/query/69cf13...

it’s good in domain too 😉

Akari Asai @akariasai.bsky.social · Nov 19

1/ Introducing ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge 📚
@uwnlp.bsky.social & Ai2
With open models & 45M-paper datastores, it outperforms proprietary systems & match human experts.
Try out our demo!
openscholar.allen.ai

1 20

Reposted by Akari Asai

Maria Antoniak @mariaa.bsky.social · Nov 19

I'm recruiting 1-2 PhD students to work with me at the University of Colorado Boulder! Looking for creative students with interests in #NLP and #CulturalAnalytics.

Boulder is a lovely college town 30 minutes from Denver and 1 hour from Rocky Mountain National Park 😎

Apply by December 15th!

A photo of Boulder, Colorado, shot from above the university campus and looking toward the Flatirons.

10 140 310

Akari Asai @akariasai.bsky.social · Nov 19

8/ ❤️Acknowledgements:
OpenScholar is the result of a collaborative effort UW, Ai2 and many others!
Huge thanks to our incredible team including experts from CS, Bio, and physics, for making this possible!
We’d love your feedback! Reply or email us with questions, ideas, or use cases✨

1

Akari Asai @akariasai.bsky.social · Nov 19

8/ 🧪 Summary
Try it out: openscholar.allen.ai
Read more: allenai.org/blog/opensch... – we discuss more details as well as limitations of OpenScholar, based on our beta testing with CS researchers!
Code & data: github.com/AkariAsai/Op...
Paper: openscholar.allen.ai/paper

Ai2 OpenScholar

openscholar.allen.ai

1 3

Akari Asai @akariasai.bsky.social · Nov 19

7/ 🌐 What’s next?
We're just getting started with OpenScholar! 🚀
Expanding domains: Support for non-CS fields is coming soon. Public API: Full-text search over 45M+ papers will be available shortly.
Try the OpenScholar demo and share your feedback!
openscholar.allen.ai

Ai2 OpenScholar

openscholar.allen.ai

1 3

Akari Asai @akariasai.bsky.social · Nov 19

6/ 💾 Open Access [2]:
📂 OpenScholar Datastore (45M+ papers up to 2024/10): huggingface.co/datasets/Ope...
📊 ScholarQABench: github.com/AkariAsai/Sc...
👩‍🔬 Human evaluation interface: github.com/AkariAsai/Op...

Ai2 OpenScholar

openscholar.allen.ai

1 2

Akari Asai @akariasai.bsky.social · Nov 19

6/ 💾 Open Access [1]:
Prior work in this area has relied on proprietary LMs and/or released only a subset of datastore
We're releasing
Demo: openscholar.allen.ai
🔓 Code & model checkpoints:
github.com/AkariAsai/Op...
huggingface.co/collections/...

Ai2 OpenScholar

openscholar.allen.ai

1 4

Akari Asai @akariasai.bsky.social · Nov 19

5/ 📊 Exert Evaluation Results:
We further conduct expert evaluations with scientists across CS, Bio and Physics, comparing OS against expert answers.
Scientists preferred OpenScholar-8B outputs compared to human-written answers in majority of the times, thanks to its coverage

1 2

Akari Asai @akariasai.bsky.social · Nov 19

5/ 📊 Automatic Results:
So how good OpenScholar?
On ScholarBench, OpenScholar-8B surpassed GPT-4o, concurrent PaperQA2, and other models in factuality & citation accuracy despite being many times cheaper!

1 2

Akari Asai @akariasai.bsky.social · Nov 19

4/ 🧪New dataset: ScholarBench
A benchmark for evaluating scientific language models on real-world, open-ended questions requiring synthesis across multiple papers. 🌟
📚 7 datasets across four scientific disciplines
🧑‍🔬 2,000+ expert-annotated question and 200 answers
📊 Automated metrics

1 4