Lightnews — Scholar-powered news

Willie Neiswanger

@willieneis.bsky.social

7K followers 190 following 240 posts

Assistant Professor in CS + AI at USC. Previously at Stanford, CMU. Machine Learning, Decision Making, AI-for-Science, Generative AI, ML Systems, LLMs. https://willieneis.github.io

willieneis.github.io

Posts Media Videos Starter Packs

Pinned

Willie Neiswanger @willieneis.bsky.social · Nov 10

I'm making a list of AI for Science researchers on bluesky — let me know if I missed you / if you'd like to join!

go.bsky.app/AcP9Lix

160 90 250

Reposted by Willie Neiswanger

Shangshang Wang @shangshang-wang.bsky.social · Apr 23

😃 Want strong LLM reasoning without breaking the bank? We explored just how cost-effectively RL can enhance reasoning using LoRA!

[1/9] Introducing Tina: A family of tiny reasoning models with strong performance at low cost, providing an accessible testbed for RL reasoning. 🧵

1 3 7

Reposted by Willie Neiswanger

Shangshang Wang @shangshang-wang.bsky.social · Feb 19

🔍 Diving deep into LLM reasoning?

From OpenAI's o-series to DeepSeek R1, from post-training to test-time compute — we break it down into structured spreadsheets. 🧵

1 2 4

Willie Neiswanger @willieneis.bsky.social · Feb 12

Added! (bsky.app/profile/will...)

Willie Neiswanger @willieneis.bsky.social · Jan 7

Our paper also contains an in-depth discussion on safety when releasing metagenomic models.

Looking for collaborators to build on this with us — please reach out!

metagene.ai

Willie Neiswanger @willieneis.bsky.social · Jan 7

We leverage the ecosystem of modern LLM tooling—in tokenization, model architecture, training, infra, etc—for performance and extensibility. METAGENE-1 is standardized & easy to use.

Hugging Face: huggingface.co/metagene-ai
Github: github.com/metagene-ai

1 5

Willie Neiswanger @willieneis.bsky.social · Jan 7

METAGENE-1 shows state-of-the-art results on pathogen detection, metagenomic embedding, and other genomic tasks.

We also release new benchmarks for genomic detection and embedding (eg, Gene-MTEB, based on MTEB for LLMs).

See our paper for details: arxiv.org/abs/2501.02045

A subset of results on our Genomic Embedding Benchmark and Pathogen Detection Benchmark.

1 4

Willie Neiswanger @willieneis.bsky.social · Jan 7

Our data pipeline is: human microbiome > wastewater > metagenomic sequences > tokens > training data.

Wastewater provides a rich source of data from tens of thousands of species across the human-adjacent microbiome. In total we pretrain on over 1.5T base pairs of DNA/RNA.

Overview of the metagenomic data collection and sequencing pipeline for model pretraining.

1 1

Willie Neiswanger @willieneis.bsky.social · Jan 7

Metagenomic sequencing of wastewater produces vast amounts of data that can capture public health trends at a societal scale. Our goal is to train a model on this data to help in large-scale wastewater monitoring & detection of novel bio threats.

Overview of METAGENE-1 and applications.

1 1

Willie Neiswanger @willieneis.bsky.social · Jan 7

Excited to release METAGENE-1, a 7B parameter metagenomic foundation model, built to aid in pathogen detection & pandemic monitoring. Pretrained on 1.5 trillion base pairs of DNA/RNA sequenced from wastewater.

A collab w/ USC, PrimeIntellect, & the Nucleic Acid Observatory.

metagene.ai

Metagenomic Foundation Model

Metagenomic Foundation Model for Pandemic Monitoring

metagene.ai

1 20

Reposted by Willie Neiswanger

Keenan Crane @keenancrane.bsky.social · Dec 9

Entropy is one of those formulas that many of us learn, swallow whole, and even use regularly without really understanding.

(E.g., where does that “log” come from? Are there other possible formulas?)

Yet there's an intuitive & almost inevitable way to arrive at this expression.