An Adversarial Perspective on Machine Unlearning for AI Safety
🏆 Best paper award
@ SoLaR Workshop

📅 Sat 14 Dec. Poster at 11am and Talk in the afternoon.
📍 Room West Meeting 121,122 Paper:
arxiv.org/abs/2409.18025

Just arrived in Vancouver for #NeurIPS! If you’d like to chat about cutting-edge research, let me know! I’ve always been curious about far too many things (for my own good), so all topics are welcome.

If you can’t catch me during the week, stop by our poster on the weekend or join the presentation!

Our paper on how unlearning fails to remove hazardous knowledge from LLM weights received 🏆 Best Paper 🏆 award at SoLaR @ NeurIPS!

Join my oral presentation on Saturday at 4:30 pm to learn more.

Eager to experiment? Reproduce our findings with this code: github.com/ethz-spylab/...

An Adversarial Perspective on Machine Unlearning for AI Safety

📖 ArXiv pre-print: arxiv.org/abs/2409.18025

Joint work with
@javirandor.com, @boyiwei.bsky.social, Yangsibo Huang,
@peterhenderson.bsky.social, @floriantramer.bsky.social

Our findings highlight that

1️⃣ Robust unlearning is not yet possible; current methods face similar challenges as safety training.

2️⃣ Black-box evaluations can be misleading when assessing the effectiveness of unlearning.

Fine-tuning “unlearned” models on benign datasets can completely restore hazardous knowledge.

Fine-tuning on dangerous knowledge leads to disproportionately fast recovery of hazardous capabilities (10 samples -> >60% of capabilities regained).

🔡 GCG can be adapted to generate universal adversarial prefixes.

↗️ Similar to safety, unlearning relies on specific directions in the residual stream that can be ablated.

✂️ We can prune neurons responsible for “obfuscating” dangerous knowledge.

How did we check this?

We adapted several white-box attacks used to jailbreak safety-trained models and applied them to two prominent unlearning methods: RMU, NPO.

Safety training fine-tunes models to refuse harmful requests but can be easily jailbroken.

Machine unlearning was introduced to fully erase hazardous knowledge, making it inaccessible to adversaries.

Sounds amazing, right? Well, existing methods cannot do this (yet).

🚨Unlearned hazardous knowledge can be retrieved from LLMs 🚨

Our results show that current unlearning methods for AI safety only obfuscate dangerous knowledge, just like standard safety training.

Here's what we found👇