Lightnews — Scholar-powered news

Lucas Alegre

@lnalegre.bsky.social

2K followers 170 following 30 posts

Professor at INF - @ufrgs.br | Ph.D. in Computer Science. I am interested in multi-policy reinforcement learning (RL) algorithms. Personal page: https://lucasalegre.github.io

lucasalegre.github.io

Posts Media Videos Starter Packs

Pinned

Lucas Alegre @lnalegre.bsky.social · Feb 16

By exploiting these connections, we introduced SFOLS, a method that is capable of constructing a set of policies and combining them via GPI with the guarantee of obtaining the optimal policy for any novel linearly-expressible tasks!

ICML'22 paper: proceedings.mlr.press/v162/alegre2...

Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfer

In many real-world applications, reinforcement learning (RL) agents might have to solve multiple tasks, each one typically modeled via a reward function. If reward functions are expressed linearly,...

proceedings.mlr.press

1 1

Lucas Alegre @lnalegre.bsky.social · Sep 3

It is really cool to see our work on multi-step GPI being cited in this amazing survey! :)

proceedings.neurips.cc/paper_files/...

Lucas Alegre @lnalegre.bsky.social · Aug 1

On average I have a good score, but it has happened to me before to have 3/4 reviewers accepting the paper, and 1 negative reviewer convincing the AC to reject.

Lucas Alegre @lnalegre.bsky.social · Aug 1

And now I got the classic rebuttal response:

"I have no concerns with the paper, all the theory is great, but since you did not run experiments in expensive domains with image-based environments, I will not increase my score".

The goal of experiments is to validate the claims! Not to beat Atari!

1 2

Lucas Alegre @lnalegre.bsky.social · Jul 24

I got the classic NeurIPS reviews "why did you not compare with [completely unrelated method whose comparison would not help support any of the paper's claim]?"

Questioning myself whether I should spend my weekend running this useless experiment or if I should argue with the reviewer.

1 3

Lucas Alegre @lnalegre.bsky.social · Jun 20

Finally, reporting only IQM may compromise scientific transparency and fairness, as it can mask poor or unstable performance. Agarwal et al. (2021), who introduced IQM in this context, recommend using it in conjunction with other statistics rather than as a standalone measure.

Lucas Alegre @lnalegre.bsky.social · Jun 20

Yes, Interquartile Mean (IQM) is a robust statistic that reduces the influence of outliers. But it does not by itself provide a clear and fair analysis of performance. In particular, IQM does not capture the full distribution of returns and may hide important information about variability and risk.

1 1

Lucas Alegre @lnalegre.bsky.social · Jun 20

While I really like the paper "Deep Reinforcement Learning at the Edge of the Statistical Precipice" (openreview.net/forum?id=uqv...), I have seen papers evaluating performance using only the IQM metric and claiming that it is a fairer metric than the mean based on this paper, which is simply wrong.

Deep Reinforcement Learning at the Edge of the Statistical Precipice

Our findings call for a change in how we report performance on benchmarks when using only a few runs, for which we present more reliable protocols accompanied with an open-source library.

openreview.net

1 1 6

Lucas Alegre @lnalegre.bsky.social · Jun 2

This work was done during my time as an intern at Disney Research Zürich. It was amazing and really fun to develop this idea with the Robotics Team!

Lucas Alegre @lnalegre.bsky.social · Jun 2

Check out AMOR now on arXiv:

Paper: arxiv.org/abs/2505.23708
Full Video: youtube.com/watch?v=gQid...

#SIGGRAPH2025 #RL #robotics

1 2

Lucas Alegre @lnalegre.bsky.social · Jun 2

A base policy with uniform weights might fail on challenging motions, but with a few weight tweaks, it nails them. Like this double spin. 🌀😵‍💫

Curious how tuning weights mid-motion can help improve the sim-to-real gap and unlock dynamic, expressive behaviors?

1 1

Lucas Alegre @lnalegre.bsky.social · Jun 2

AMOR trains a single policy conditioned on reward weights and motion context, letting you fine-tune the reward after training.
Want smoother motions? Better accuracy? Just adjust the weights — no retraining needed!

1 1

Lucas Alegre @lnalegre.bsky.social · Jun 2

We are excited to share our #SIGGRAPH2025 paper,

“AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning”!
Lucas Alegre*, Agon Serifi*, Ruben Grandia, David Müller, Espen Knoop, Moritz Baecher

1 3

Lucas Alegre @lnalegre.bsky.social · Jun 2

Annoyed by having to retrain your entire policy just because your reward weights did not quite work on the real robot? 🤖

www.youtube.com/watch?v=gQid...

AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning

YouTube video by DisneyResearchHub

www.youtube.com

1 1 6

Lucas Alegre @lnalegre.bsky.social · May 30

Thank you, Peter! :)

Lucas Alegre @lnalegre.bsky.social · May 29

I'm really glad to have been selected as one of the ICML 2025 Top Reviewers!

Too bad I won't be able to go since my last submission was not accepted, even with scores Accept, Accept, Weak Accept, and Weak Reject 🫠

Lucas Alegre @lnalegre.bsky.social · Mar 17

Last week, I was at @khipu-ai.bsky.social in Santiago, Chile. It was really amazing to see so many great speakers and researchers from Latin America together!

1 4

Reposted by Lucas Alegre

Pablo Samuel Castro @pcastr.bsky.social · Mar 5

RL is so back!

(well, for some of us, it never really left)

awards.acm.org/about/2024-t...

Andrew Barto and Richard Sutton are the recipients of the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning.

Andrew Barto and Richard Sutton as the recipients of the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. In a series of papers beginning...

awards.acm.org

1 12 72

Lucas Alegre @lnalegre.bsky.social · Feb 16

Thank you!
link.springer.com/article/10.1...
This paper is a great start point!

A practical guide to multi-objective reinforcement learning and planning - Autonomous Agents and Multi-Agent Systems

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learnin...

link.springer.com

Lucas Alegre @lnalegre.bsky.social · Feb 16

Thank you! 😊

Lucas Alegre @lnalegre.bsky.social · Feb 16

Finally, I would like to thank my advisors, Prof. Ana Bazzan and Prof. Bruno C. da Silva; Prof. Ann Nowé who received me at VUB for my PhD stay; and Disney Research Zürich, where I interned.

I am very grateful to everyone with that I had the chance to collaborate in all such amazing projects! 💙

Lucas Alegre @lnalegre.bsky.social · Feb 16

I believe all these contributions open room for many interesting ideas for multi-policy RL methods. Especially in transfer learning (SFs&GPI) and multi-objective RL settings! 🚀

1 1

Lucas Alegre @lnalegre.bsky.social · Feb 16

* MO-Gymnasium (github.com/Farama-Found...) is a library of MORL environments; and

* MORL Baselines (github.com/LucasAlegre/...) is a library of MORL algorithms.

Both have become standards in MORL research and have over 100k downloads in the past year!

GitHub - Farama-Foundation/MO-Gymnasium: Multi-objective Gymnasium environments for reinforcement learning

Multi-objective Gymnasium environments for reinforcement learning - Farama-Foundation/MO-Gymnasium

github.com

1 1

Lucas Alegre @lnalegre.bsky.social · Feb 16

Besides the theoretical and algorithmic contributions, we also introduced an open-source toolkit for MORL research!

NeurIPS D&B 2023 Paper - openreview.net/pdf?id=jfwRL...

openreview.net

1 1

Lucas Alegre @lnalegre.bsky.social · Feb 16

Next, we further explored how to leverage approximate models of the environment to improve zero-shot policy transfer. Our method, ℎ-GPI, interpolates between model-free GPI and fully model-based planning as a function of the planning horizon ℎ.

NeurIPS 2023 Paper - openreview.net/pdf?id=KFj0Q...

openreview.net

1 1

Lucas Alegre @lnalegre.bsky.social · Feb 16

Next, we further explored these ideas and introduced two novel MORL algorithms that exploit GPI to increase sample efficiency in MORL: GPI-LS and GPI-PD.

AAMAS'23 paper: tinyurl.com/aamas23

Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritization | Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems

You will be notified whenever a record that you have chosen has been cited.

tinyurl.com

1 1