Lightnews — Scholar-powered news

Reposted by Sweta Karlekar

Nicolas Beltran-Velez @velezbeltran.bsky.social · Dec 12

Hello!

We will be presenting Estimating the Hallucination Rate of Generative AI at NeurIPS. Come if you'd like to chat about epistemic uncertainty for In-Context Learning, or uncertainty more generally. :)

Location: East Exhibit Hall A-C #2703
Time: Friday @ 4:30
Paper: arxiv.org/abs/2406.07457

4 22

Reposted by Sweta Karlekar

Andrew Jesson @anndvision.bsky.social · Dec 13

fun @bleilab.bsky.social x oatml collab

come chat with Nicolas , @swetakar.bsky.social , Quentin , Jannik , and i today

Nicolas Beltran-Velez @velezbeltran.bsky.social · Dec 12

Hello!

We will be presenting Estimating the Hallucination Rate of Generative AI at NeurIPS. Come if you'd like to chat about epistemic uncertainty for In-Context Learning, or uncertainty more generally. :)

Location: East Exhibit Hall A-C #2703
Time: Friday @ 4:30
Paper: arxiv.org/abs/2406.07457

1 1 10

Reposted by Sweta Karlekar

Blei Lab @bleilab.bsky.social · Dec 2

Check out our new paper from the Blei Lab on probabilistic predictions with conditional diffusions and gradient boosted trees! #Neurips2024

Nicolas Beltran-Velez @velezbeltran.bsky.social · Dec 2

I am very excited to share our new Neurips 2024 paper + package, Treeffuser! 🌳 We combine gradient-boosted trees with diffusion models for fast, flexible probabilistic predictions and well-calibrated uncertainty.

paper: arxiv.org/abs/2406.07658
repo: github.com/blei-lab/tre...

🧵(1/8)

Samples y | x from Treeffuser vs. true densities, for multiple values of x under three different scenarios. Treeffuser captures arbitrarily complex conditional distributions that vary with x.

6 33

Reposted by Sweta Karlekar

Blei Lab @bleilab.bsky.social · Dec 10

Check out our new paper about hypothesis testing the circuit hypothesis in LLMs! This work previously won a top paper award at the ICML mechanistic interpretability workshop, and we’re excited to share it at #Neurips2024

claudia shi @claudiashi.bsky.social · Dec 10

The circuit hypothesis proposes that LLM capabilities emerge from small subnetworks within the model. But how can we actually test this? 🤔

joint work with @velezbeltran.bsky.social @maggiemakar.bsky.social @anndvision.bsky.social @bleilab.bsky.social Adria @far.ai Achille and Caro

1 7

Reposted by Sweta Karlekar

Ben Burtenshaw @benburtenshaw.bsky.social · Dec 3

For anyone interested in fine-tuning or aligning LLMs, I’m running this free and open course called smol course. It’s not a big deal, it’s just smol.

🧵>>

9 64 330

Sweta Karlekar @swetakar.bsky.social · Dec 2

Very happy to share some recent work by my colleagues @velezbeltran.bsky.social, @aagrande.bsky.social and @anazaret.bsky.social! Check out their work on tree-based diffusion models (especially the website—it’s quite superb 😊)!

Nicolas Beltran-Velez @velezbeltran.bsky.social · Dec 2

I am very excited to share our new Neurips 2024 paper + package, Treeffuser! 🌳 We combine gradient-boosted trees with diffusion models for fast, flexible probabilistic predictions and well-calibrated uncertainty.

paper: arxiv.org/abs/2406.07658
repo: github.com/blei-lab/tre...

🧵(1/8)

1 1 13

Sweta Karlekar @swetakar.bsky.social · Nov 29

Just learned about @andrewyng.bsky.social's new tool, aisuite (github.com/andrewyng/ai...) and wanted to share! It's a standardized wrapper around chat completions that lets you easily switch between querying different LLM providers, including OpenAI, Anthropic, Mistral, HuggingFace, Ollama, etc.

GitHub - andrewyng/aisuite: Simple, unified interface to multiple Generative AI providers

Simple, unified interface to multiple Generative AI providers - GitHub - andrewyng/aisuite: Simple, unified interface to multiple Generative AI providers

github.com

1 3 22

Reposted by Sweta Karlekar

Jakub M. Tomczak @jmtomczak.bsky.social · Nov 27

Test of Time Paper Awards are out! 2014 was a wonderful year with lots of amazing papers. That's why, we decided to highlight two papers: GANs (@ian-goodfellow.bsky.social et al.) and Seq2Seq (Sutskever et al.). Both papers will be presented in person 😍

Link: blog.neurips.cc/2024/11/27/a...

Announcing the NeurIPS 2024 Test of Time Paper Awards – NeurIPS Blog

blog.neurips.cc

1 14 110

Sweta Karlekar @swetakar.bsky.social · Nov 25

Sorry John, that isn’t my area of expertise!

1

Sweta Karlekar @swetakar.bsky.social · Nov 24

This is very interesting! Do you have any intuition as to whether or not this phenomenon happens only with very simple “reasoning” steps? Does relying on retrieval increase as you progress from simple math to more advanced prompts like GSM8K or adversarially designed prompts (like adding noise)?

1 3

Reposted by Sweta Karlekar

Martin Wattenberg @wattenberg.bsky.social · Nov 23

The Gini coefficient is the standard way to measure inequality, but what does it mean, concretely? I made a little visualization to build intuition:
www.bewitched.com/demo/gini

Many circles of different sizes, representing a visualization of inequality

10 57 200

Reposted by Sweta Karlekar

Christoph Molnar @christophmolnar.bsky.social · Nov 15

Interested in machine learning in science?

Timo and I recently published a book, and even if you are not a scientist, you'll find useful overviews of topics like causality and robustness.

The best part is that you can read it for free: ml-science-book.com

7 30 130

Sweta Karlekar @swetakar.bsky.social · Nov 22

Learning doesn’t have to mean explicit weight changes; ICL can be viewed as temporary implicit finetuning (arxiv.org/abs/2212.10559) or like a “state” change to the model instead of a weight change, akin to how learning happens in fast RL vs slow RL (www.cell.com/trends/cogni...).

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers

Large pretrained language models have shown surprising in-context learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without parameter u...

arxiv.org

8

Reposted by Sweta Karlekar

Jake Handy @jakehandy.com · Nov 22

new paper from Anthropic on LLM evaluation recommendations

www.anthropic.com/research/sta...

A statistical approach to model evaluations

A research paper from Anthropic on how to apply statistics to improve language model evaluations

www.anthropic.com

1 13

Reposted by Sweta Karlekar

Christoph Molnar @christophmolnar.bsky.social · Nov 22

Just realized BlueSky allows sharing valuable stuff cause it doesn't punish links. 🤩

Let's start with "What are embeddings" by @vickiboykis.com

The book is a great summary of embeddings, from history to modern approaches.

The best part: it's free.

Link: vickiboykis.com/what_are_emb...

Over the past decade, embeddings — numerical representations of
machine learning features used as input to deep learning models — have
become a foundational data structure in industrial machine learning
systems. TF-IDF, PCA, and one-hot encoding have always been key tools
in machine learning systems as ways to compress and make sense of
large amounts of textual data. However, traditional approaches were
limited in the amount of context they could reason about with increasing
amounts of data. As the volume, velocity, and variety of data captured
by modern applications has exploded, creating approaches specifically
tailored to scale has become increasingly important.
Google’s Word2Vec paper made an important step in moving from
simple statistical representations to semantic meaning of words. The
subsequent rise of the Transformer architecture and transfer learning, as
well as the latest surge in generative methods has enabled the growth
of embeddings as a foundational machine learning data structure. This
survey paper aims to provide a deep dive into what embeddings are,
their history, and usage patterns in industry.

22 100 650

Sweta Karlekar @swetakar.bsky.social · Nov 20

(Shameless) plug for David Blei's lab at Columbia University! People in the lab currently work on a variety of topics, including probabilistic machine learning, Bayesian stats, mechanistic interpretability, causal inference and NLP.

Please give us a follow! @bleilab.bsky.social

1 3 20

Sweta Karlekar @swetakar.bsky.social · Nov 20

Hi! Our lab does Bayesian stuff :) Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social

1 1

Sweta Karlekar @swetakar.bsky.social · Nov 20

Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social

Sweta Karlekar @swetakar.bsky.social · Nov 20

Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social

1

Sweta Karlekar @swetakar.bsky.social · Nov 20

Could you add Dave blei's lab to this pack as well if it's not already full! @bleilab.bsky.social

1

Reposted by Sweta Karlekar

Nicolas Beltran-Velez @velezbeltran.bsky.social · Nov 20

We created an account for the Blei Lab! Please drop a follow 😊

@bleilab.bsky.social

1 3

Reposted by Sweta Karlekar

Carlos E. Perez (IntuitMachine) @ceperez.bsky.social · Nov 20

Almost 3x faster than FlashAttentiion2 github.com/thu-ml/SageA...

GitHub - thu-ml/SageAttention: Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various mod...

Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models. - thu-ml/SageAttention

github.com

1 6

Sweta Karlekar @swetakar.bsky.social · Nov 20

Oh, I’ve been meaning to check out that YouTube series—thanks! Also sadly, there's no class website, but I can share the "super quick intro to mech interp" presentation I made. It’s somewhat rough, but hopefully, it gets the main points across! sweta.dev/files/intro_...

sweta.dev

1 1

Reposted by Sweta Karlekar

Martyn Plummer @martynplummer.bsky.social · Nov 19

📢 Post-Bayesian online seminar series coming!📢
To stay posted, sign up at
tinyurl.com/postBayes
We'll discuss cutting-edge methods for posteriors that no longer rely on Bayes Theorem.
(e.g., PAC-Bayes, generalised Bayes, Martingale posteriors, ...)
Pls circulate widely!

Mailing list contact information

Information to be added to the post-Bayes mailing list.

tinyurl.com

6 16

Sweta Karlekar @swetakar.bsky.social · Nov 20

Adding Neel Nanda's favorite paper list as well: www.alignmentforum.org/posts/NfFST5...
(6/n)

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum

This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two…

www.alignmentforum.org