Lightnews — Scholar-powered news

Reposted by Vaidehi Patil

Peter Hall @peterha2l.bsky.social · Jul 15

In any case, the work is featuring at an interesting-looking workshop this weekend, put on by @katherinelee.bsky.social, @vaidehipatil.bsky.social, and others. More info here: mugenworkshop.github.io

MUGen @ ICML 2025 - Workshop on Machine Unlearning for Generative AI

mugenworkshop.github.io

1 2

Reposted by Vaidehi Patil

Elias Stengel-Eskin @esteng.bsky.social · May 5

Extremely excited to announce that I will be joining
@utaustin.bsky.social Computer Science in August 2025 as an Assistant Professor! 🎉

5 9 43

Vaidehi Patil @vaidehipatil.bsky.social · May 7

Thanks to my amazing collaborators Yi-Lin Sung , @peterbhase.bsky.social , Jie Peng, Tianlong Chen , @mohitbansal.bsky.social for a wonderful collaboration!

Vaidehi Patil @vaidehipatil.bsky.social · May 7

📎 Check it out here!
📄 Paper: arxiv.org/abs/2505.01456
💻 Code and Dataset: github.com/Vaidehi99/Un...
huggingface.co/datasets/vai...
🤗 HuggingFace: huggingface.co/papers/2505....

Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation

LLMs trained on massive datasets may inadvertently acquire sensitive information such as personal details and potentially harmful content. This risk is further heightened in multimodal LLMs as they in...

arxiv.org

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

Key Findings
🔥 Multimodal attacks are the most effective
🛡️ Our strongest defense is deleting info from hidden states
📉 Larger models are more robust to extraction attacks post-editing compared to smaller ones
🎯 UnLOK-VQA enables targeted evaluations of unlearning defenses

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

⚔️ Benchmarking Multimodal Unlearning Defenses
Multimodal data opens up new attack vectors.
We benchmark 6 unlearning defenses against 7 attack strategies, including:
✅White-box attacks
✅Black-box paraphrased multimodal prompts

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

This enables two key types of evaluation:
✅Generalization Evaluation
✔️Rephrased questions
✔️Rephrased images

✅Specificity Evaluation
✔️Neighboring questions (same image, new question)
✔️Neighboring images (same concept, different image)

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

📦 What Is UnLOK-VQA?
UnLOK-VQA focuses on unlearning pretrained knowledge and builds on OK-VQA, a visual QA dataset. We extend it w/ an automated question-answer generation and image generation pipeline:
✅Forget samples from OK-VQA
✅New samples at varying levels of proximity (easy, medium, hard)

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

This is essential for:
📜 Legal compliance (e.g., GDPR, CCPA, the right to be forgotten)
🔐 Multimodal Privacy (e.g., faces, locations, license plates)
📷 Trust in real-world image-grounded systems

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

🔍 Why Does Multimodal Unlearning Matter?
Existing unlearning benchmarks focus only on text.
But multimodal LLMs are trained on web-scale data—images + captions—making them highly vulnerable to leakage of sensitive or unwanted content.
Unlearning must hold across modalities, not just in language.

1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

We study:
❓ How effectively can we erase multimodal knowledge?
❓ How should we measure forgetting in multimodal settings?
✅We benchmark 6 unlearning defenses against 7 whitebox and blackbox attack strategies

1 1

Vaidehi Patil @vaidehipatil.bsky.social · May 7

🚨 Introducing our @tmlrorg.bsky.social paper “Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation”
We present UnLOK-VQA, a benchmark to evaluate unlearning in vision-and-language models, where both images and text may encode sensitive or private information.

1 8 10

Reposted by Vaidehi Patil

Mohit Bansal @mohitbansal.bsky.social · Apr 21

In Singapore for #ICLR2025 this week to present papers + keynotes 👇, and looking forward to seeing everyone -- happy to chat about research, or faculty+postdoc+phd positions, or simply hanging out (feel free to ping)! 🙂

Also meet our awesome students/postdocs/collaborators presenting their work.

1 4 19

Reposted by Vaidehi Patil

Katherine Lee @katherinelee.bsky.social · Apr 2

Come chat about unlearning with us!!

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

🚨Exciting @icmlconf.bsky.social workshop alert 🚨

We’re thrilled to announce the #ICML2025 Workshop on Machine Unlearning for Generative AI (MUGen)!

⚡Join us in Vancouver this July to dive into cutting-edge research on unlearning in generative AI with top speakers and panelists! ⚡

1 5

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

Call for PC Members!
We’re looking for program committee members!
📝 Submit your Expression of Interest here: forms.gle/ZPEHeymJ4t5N...
#ICML2025

MUGen @ ICML '25 - PC Expression of Interest

We are currently recruiting reviewers for the Program Committee of MUGen (Machine Unlearning for Generative AI) @ ICML '25. If you are interested in participating, please fill out this form. We antici...

forms.gle

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

👩‍💻 Organizers:
Mantas Mazeika, Yang Liu, @katherinelee.bsky.social, @mohitbansal.bsky.social, Bo Li and myself (@vaidehipatil.bsky.social) 🙂

1

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

🔥 Speakers & Panelists:
We're lucky to have an incredible lineup of speakers and panelists covering diverse topics in our workshop:
Nicholas Carlini, Ling Liu, Shagufta Mehnaz, @peterbhase.bsky.social , Eleni Triantafillou, Sijia Liu, @afedercooper.bsky.social, Amy Cyphert

1

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

We invite contributions exploring key challenges and advancements at the intersection of machine unlearning and generative AI!

🔗 Full details & updates: mugenworkshop.github.io

📅 Key Dates:
📝 Submission Deadline: May 19
✅ Acceptance Notifications: June 9
🤝 Workshop Date: July 18 or 19

MUGen @ ICML 2025 - Workshop on Machine Unlearning for Generative AI

mugenworkshop.github.io

1

Vaidehi Patil @vaidehipatil.bsky.social · Apr 2

🚨Exciting @icmlconf.bsky.social workshop alert 🚨

We’re thrilled to announce the #ICML2025 Workshop on Machine Unlearning for Generative AI (MUGen)!

⚡Join us in Vancouver this July to dive into cutting-edge research on unlearning in generative AI with top speakers and panelists! ⚡

1 1 4

Reposted by Vaidehi Patil

Archiki Prasad @archiki.bsky.social · Mar 27

🥳🥳 Honored and grateful to be awarded the 2025 Apple Scholars in AI/ML PhD Fellowship! ✨

Huge shoutout to my advisor @mohitbansal.bsky.social, & many thanks to my lab mates @unccs.bsky.social , past collaborators + internship advisors for their support ☺️🙏

machinelearning.apple.com/updates/appl...

1 3 14

Reposted by Vaidehi Patil

Elias Stengel-Eskin @esteng.bsky.social · Feb 25

🚨UPCORE is our new method for balancing unlearning/forgetting with maintaining model performance.

Best part is it works by selecting a coreset from the data rather than changing the model, so it is compatible with any unlearning method, with consistent gains for 3 methods + 2 tasks!

Vaidehi Patil @vaidehipatil.bsky.social · Feb 25

🚨 Introducing UPCORE, to balance deleting info from LLMs with keeping their other capabilities intact.

UPCORE selects a coreset of forget data, leading to a better trade-off across 2 datasets and 3 unlearning methods.

🧵👇

2 4

Vaidehi Patil @vaidehipatil.bsky.social · Feb 25

Huge thanks to my co-authors
@esteng.bsky.social , and @mohitbansal.bsky.social for a great collaboration!

🚀 Check it out here:
📄 Paper: arxiv.org/abs/2502.15082
💻 Code: github.com/Vaidehi99/UP...
🤗 @huggingface page: huggingface.co/papers/2502....

UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning

User specifications or legal frameworks often require information to be removed from pretrained models, including large language models (LLMs). This requires deleting or "forgetting" a set of data poi...

arxiv.org

1

Vaidehi Patil @vaidehipatil.bsky.social · Feb 25

UPCORE consistently outperforms baselines across all methods:

✔️ Less unintended degradation
✔️ Deletion transferred to pruned points

UPCORE provides a practical, method-agnostic approach that improves the reliability of unlearning techniques.

1 1

Vaidehi Patil @vaidehipatil.bsky.social · Feb 25

Instead of evaluating at a single training checkpoint, we introduce AUC (Area Under the Curve) across deletion effectiveness and utility.

This provides a complete picture of the trade-off between forgetting and knowledge retention over the unlearning trajectory.

1 1

Vaidehi Patil @vaidehipatil.bsky.social · Feb 25

We apply UPCORE across three unlearning methods:
📉 Gradient Ascent
🚫 Refusal
🔄 Negative Preference Optimization (NPO)

We measure:
✔️ Deletion effectiveness – How well the target is removed
✔️ Unintended degradation – Impact on other abilities
✔️ Positive transfer – How well unlearning generalizes

1 1