Lightnews — Scholar-powered news

Levi Lelis

@programsynthesis.bsky.social

310 followers 480 following 81 posts

Associate Professor - University of Alberta Canada CIFAR AI Chair with Amii Machine Learning and Program Synthesis he/him; ele/dele 🇨🇦 🇧🇷 https://www.cs.ualberta.ca/~santanad

www.cs.ualberta.ca

Posts Media Videos Starter Packs

Pinned

Levi Lelis @programsynthesis.bsky.social · Dec 18

I recently spoke at IPAM's Naturalistic Approaches to Artificial Intelligence Workshop, sharing some of the programmatic perspectives we're exploring in reinforcement learning research.

youtu.be/UNpg05yxc3o?...

Levi Lelis - Learning Libraries of Programmatic Policies - IPAM at UCLA

YouTube video by Institute for Pure & Applied Mathematics (IPAM)

youtu.be

1 2

Reposted by Levi Lelis

Gaia Vince @wanderinggaia.bsky.social · 27d

Brazil shows how it’s done in a democracy

Brazil’s supreme court finds Bolsonaro guilty of plotting military coup

Former president faces decades-long jail sentence for seeking to forcibly cling to power after losing 2022 election

www.theguardian.com

5 83 320

Reposted by Levi Lelis

Matthew Guzdial @matthewguz.bsky.social · Aug 25

Excited to announce that our work on Reinforcement Learning for Arachnophobia treatment has been accepted at ACM Transactions on Interactive Intelligent Systems! We found that an RL agent could more effectively adapt VR spiders to achieve specified anxiety levels in users compared to current SOTA.

A graph showing that a rules-based approach consistently underperformed at achieving desired anxiety levels measured in normalized SCL compared to an RL approach.

A brownish red virtual spider a medium distance away

5 7 55

Reposted by Levi Lelis

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Jul 27

Was talking to a student who wasn't sure about why one would get a PhD. So I wrote up a list of reasons!
www.eugenevinitsky.com/posts/reason...

Eugene Vinitsky

www.eugenevinitsky.com

7 11 51

Reposted by Levi Lelis

Levi Lelis @programsynthesis.bsky.social · Jul 2

Previous work has shown that programmatic policies—computer programs written in a domain-specific language—generalize to out-of-distribution problems more easily than neural policies.

Is this really the case? 🧵

2 4 7

Levi Lelis @programsynthesis.bsky.social · Jul 2

Sometimes, neural networks (with little tweaks) are enough. Other times, solving the task requires a programmatic representation to capture algorithmic structure.

Preprint: arxiv.org/abs/2506.14162

Common Benchmarks Undervalue the Generalization Power of Programmatic Policies

Algorithms for learning programmatic representations for sequential decision-making problems are often evaluated on out-of-distribution (OOD) problems, with the common conclusion that programmatic pol...

arxiv.org

Levi Lelis @programsynthesis.bsky.social · Jul 2

1. Is the representation expressive enough to find solutions that generalize?
2. Can our search procedure find a policy that generalizes?

1 2

Levi Lelis @programsynthesis.bsky.social · Jul 2

So, when should we use neural vs. programmatic policies for OOD generalization?

Rather than treating programmatic policies as the default, we should ask:

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

As an illustrative example, we changed the grid-world task so that a solution policy must use a queue or stack to solve a navigation task. FunSearch found a Python program that provably generalizes. As one would expect, neural nets couldn’t solve the problem.

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

Are neural and programmatic policies similar in terms OOD generalization? We don't think so. We think that benchmark problems used in previous work actually undervalue what programmatic representations can do.

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

Programmatic policies appeared to generalize better in previous work because they never learned to go fast in the easy training tracks. Neural nets optimized speed well, which made it difficult to generalize to tracks with sharp curves.

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

In a car-racing task, we adjusted the reward to encourage cautious driving. Neural nets generalized just as well as programmatic policies.

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

We had to perform simple changes to the neural policies' training pipeline to attain similar OOD generalization to that exhibited by programmatic ones.

In a grid-world problem, we used the same sparse observation space as used with the programmatic policies augmented with the agent's last action.

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

In a preprint, led by my Master's student Amirhossein Rajabpour, we revisit some of these OOD generalization claims and show that neural policies generalize just as well as programmatic ones on benchmark problems used in previous work.

Preprint: arxiv.org/abs/2506.14162

arXiv.org e-Print archive

arxiv.org

1 1

Levi Lelis @programsynthesis.bsky.social · Jul 2

2 4 7

Reposted by Levi Lelis

Marc Lanctot @sharky6000.bsky.social · Jun 29

If like me your Discover feed has been even worse lately and you are here for ML/AI news and discussion, check out these two feeds:

- Paper Skygest
- ML Feed: Trending

Links below 👇

3 4 32

Reposted by Levi Lelis

Martin Klissarov @martinklissarov.bsky.social · Jun 27

As AI agents face increasingly long and complex tasks, decomposing them into subtasks becomes increasingly appealing.

But how do we discover such temporal structure?

Hierarchical RL provides a natural formalism-yet many questions remain open.

Here's our overview of the field🧵

1 10 34

Reposted by Levi Lelis

Mark Gongloff @markgongloff.bsky.social · Jun 24

As hot as this summer is, it’s also one of the coolest we’ll ever enjoy again.

Just how much hotter and deadlier summers will get is still up to us. Right now we’re working hard to make them worse

🎁 link to my @opinion.bloomberg.com column:

www.bloomberg.com/opinion/arti...

The Heat Dome Wants a Word With Climate-Change Deniers

The temperatures gripping the US this week were made up to five times more likely by the fact that the atmosphere is simply hotter.

www.bloomberg.com

3 41 86

Reposted by Levi Lelis

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · Jun 21

Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please!

Full posting to come in a bit.

Robust Autonomy Emerges from Self-Play

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic drivi...

arxiv.org

4 24 60

Levi Lelis @programsynthesis.bsky.social · Jun 17

In addition to Sat's pointers, I would also take a look at the following recent paper by @swarat.bsky.social:

www.cs.utexas.edu/~swarat/pubs...

Also, the following paper covers most of the recent works on neuro-guided bottom-up synthesis algorithms:

webdocs.cs.ualberta.ca/~santanad/pa...

www.cs.utexas.edu

Reposted by Levi Lelis

Matthew Guzdial @matthewguz.bsky.social · Jun 16

We’re extending the AIIDE deadline! Partially due to author requests, partially due to a significant increase in submissions meaning I need to increase the PC!

AIIDE @aiide.bsky.social · Jun 16

🚨#AIIDE25 full paper deadline extended!

Due to requests from authors, we have extended the deadline for full paper submission to June 27, 2025!

🔗 More information: sites.google.com/ualberta.ca/...

AIIDE 2025 - Call for Papers

AIIDE 2025 welcomes submissions across the vast field of Artificial Intelligence and Interactive Digital Entertainment. We are particularly interested in novel contributions and applications, as well ...

sites.google.com

1 4 5

Levi Lelis @programsynthesis.bsky.social · Jun 13

I wanted to thank the folks who reviewed our paper. Your feedback helped us improve our work, especially by asking us to include experiments on more difficult instances and the TSP. Thank you!

Levi Lelis @programsynthesis.bsky.social · Jun 13

Still, many important problems with real-world applications, such as the TSP and program synthesis, share some of the properties we assume in this work.

Levi Lelis @programsynthesis.bsky.social · Jun 13

The work has a few limitations. The policy learning scheme was evaluated only on needle-in-the-haystack deterministic problems. Also, since we are using tree search algorithms, we assume the agent has access to an efficient forward model.

Levi Lelis @programsynthesis.bsky.social · Jun 13

In other cases, where clustering seems unable to find relevant structure, the subgoal-based policies do not seem to harm the search, as in Sokoban problems.

Levi Lelis @programsynthesis.bsky.social · Jun 13

The empirical results are strong when clustering effectively detects the problem's underlying structure.