Author | Lightnews

CSML IIT Lab @pontilgroup.bsky.social · May 2

Excited to share our group’s latest work at #AISTATS2025! 🎓
Tackling concentration in dependent data settings with empirical Bernstein bounds for Hilbert space-valued processes.
📍Catch the poster tomorrow!

🔁 See the original tweet for details!

Erfan Mirzaei @erfunmirzaei.bsky.social · May 2

🚨 Poster at #AISTATS2025 tomorrow!
📍Poster Session 1 #125

We present a new empirical Bernstein inequality for Hilbert space-valued random processes—relevant for dependent, even non-stationary data.

w/ Andreas Maurer, @vladimir-slk.bsky.social & M. Pontil

📄 Paper: openreview.net/forum?id=a0E...

3

CSML IIT Lab @pontilgroup.bsky.social · Apr 9

DeltaProduct is here! Achieve better state tracing through highly parallel execution. Explore more!🚀

julien-siems.bsky.social @julien-siems.bsky.social · Mar 28

1/9 There is a fundamental tradeoff between parallelizability and expressivity of Large Language Models. We propose a new linear RNN architecture, DeltaProduct, that can effectively navigate this tradeoff. Here's how!

1 5

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

P11] (submitted to The Journal of Chemical Physics)
chemrxiv.org/engage/chemr...

Kooplearn library:
kooplearn.readthedocs.io/latest/

For the longer version of the thread, you can take a look at this blog post:
vladi-iit.github.io/posts/2024-1...

Slow dynamical modes from static averages

In recent times, efforts are being made at describing the evolution of a complex system not through long trajectories, but via the study of probability distribution evolution. This more collective app...

chemrxiv.org

2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

Publications:
[P1] NeurIPS 2022
arxiv.org/abs/2205.14027

[P2] NeurIPS2023
arxiv.org/abs/2302.02004

[P3] ICML2024
arxiv.org/abs/2312.13426

[P4] NeurIPS2023
arxiv.org/abs/2306.04520

[P5] ICLR 2024
arxiv.org/abs/2307.09912

[P6] NeurIPS2024
arxiv.org/abs/2405.12940

Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces

We study a class of dynamical systems modelled as Markov chains that admit an invariant distribution via the corresponding transfer, or Koopman, operator. While data-driven algorithms to reconstruct s...

arxiv.org

1 2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

14/ Looking ahead, we’re excited to tackle new challenges:
• Learning from partial observations
• Modeling non-time-homogeneous dynamics
• Expanding applications in neuroscience, genetics, and climate modeling

Stay tuned for groundbreaking updates from our team! 🌍

1 2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

🙏 Collaborations with the Dynamic Legged Systems group led by Claudio Semini and the Atomistic Simulations group led by Michele Parrinello enriched our research, resulting in impactful works like [P9, P10] and [P7, P11].

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

12/ This journey wouldn’t have been possible without the inspiring collaborations that shaped our work.

🌟 Special thanks to Karim Lounici from École Polytechnique, whose insights were a major driving force behind many projects.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

11/ One of our most exciting results:
[P8] NeurIPS 2024 proposed Neural Conditional Probability (NCP) to efficiently learn conditional distributions. It simplifies uncertainty quantification and guarantees accuracy for nonlinear, high-dimensional data.

Predicting the quantiles for opening/closing of the Chignolin protein in the next simulation step

1 2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

10/ [P7] NeurIPS 2024 developed methods to discover slow dynamical modes in systems like molecular simulations. This is transformative for studying rare events and costly data acquisition scenarios in atomistic systems.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

9/ Addressing continuous dynamics:
[P6] NeurIPS 2024 introduced a physics-informed framework for learning Infinitesimal Generators (IG) of stochastic systems, ensuring robust spectral estimation.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

8/ 🌟 Representation learning takes center stage in:
[P5] ICLR 2024
We combined neural networks with operator theory via Deep Projection Networks (DPNets). This approach enhances robustness, scalability, and interpretability for dynamical systems.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

7/ 📈 Scaling up:
[P4] NeurIPS 2023 introduced a Nyström sketching-based method to reduce computational costs from cubic to almost linear without sacrificing accuracy. Validated on massive datasets like molecular dynamics, see figure.

Free energy surface of Chignolin protein folding

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

6/ [P3] ICML 2024 addressed a critical issue in TO-based modeling: reliable long-term predictions.
Our Deflate-Learn-Inflate (DLI) paradigm ensures uniform error bounds, even for infinite time horizons. This method stabilized predictions in real-world tasks; see the figure.

Effects of metric distortion in learning eigenvalues (left) and stabilization of forecasting (right) for Ornstein-Uhlenbeck process

1 2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

5/ [P2] NeurIPS 2023 advanced TOs with theoretical guarantees for spectral decomposition—previously lacking finite sample guarantees. We developed sharp learning rates, enabling accurate, reliable models for long-term system behavior.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

4/ 🔑 The journey began with:
[P1] NeurIPS 2022
We introduced the first ML formulation for learning TO, which led to the development of the open-source Kooplearn library. This step laid the groundwork for exploring the theoretical limits of operator learning from finite data.

1 2

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

3/TOs describe system evolution over finite time intervals, while IGs capture instantaneous rates of change. Their spectral decomposition is key for identifying dominant modes and understanding long-term behavior in complex or stochastic systems.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

2/ 🌐 Our work revolves around Markov/Transfer Operators (TO) and their Infinitesimal Generators (IG)—tools that allow us to model complex dynamical systems by understanding their evolution in higher-dimensional spaces. Here’s why this matters.

1 1

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

1/ 🚀 Over the past two years, our team, CSML, at IIT, has made significant strides in the data-driven modeling of dynamical systems. Curious about how we use advanced operator-based techniques to tackle real-world challenges? Let’s dive in! 🧵👇

1 3 5

CSML IIT Lab @pontilgroup.bsky.social · Jan 15

An inspiring dive into understanding dynamical processes through 'The Operator Way.' A fascinating approach made accessible for everyone—check it out! 👇👀

Pietro Novelli @pienovelli.bsky.social · Jan 9

For the past four years, I’ve been working on a topic that’s both fascinating and challenging to explain. In this post, I’ve tried to present The Operator Way — a paradigm for understanding dynamical processes — in plain, approachable terms.

pietronvll.github.io/the-operator...

1 4

Reposted by CSML IIT Lab

Riccardo Grazzi @riccardograzzi.bsky.social · Dec 10

Excited to present
"Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues"
at the M3L workshop at #NeurIPS
https://buff.ly/3BlcD4y

If interested, you can attend the presentation the 14th at 15:00, pass at the afternoon poster session, or DM me to discuss :)

Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues

Linear Recurrent Neural Networks (LRNNs) such as Mamba, RWKV, GLA, mLSTM, and DeltaNet have emerged as efficient alternatives to Transformers in large language modeling, offering linear scaling with…

buff.ly

3 9

Reposted by CSML IIT Lab

Pietro Novelli @pienovelli.bsky.social · Dec 12

In his book “The Nature of Statistical Learning” V. Vapnik wrote:
“When solving a given problem, try to avoid a more general problem as an intermediate step”

1 3 8

CSML IIT Lab @pontilgroup.bsky.social · Dec 10

Join us at our posters and talks to connect, share ideas, and explore collaborations. 🚀✨

3

CSML IIT Lab @pontilgroup.bsky.social · Dec 10

🔬 Fine-tuning Foundation Models for Molecular Dynamics: A Data-Efficient Approach with Random Features
✍️ @pienovelli.bsky.social, L. Bonati, P. Buigues, G. Meanti, L. Rosasco, M. Pontil | 📅ML4PS Workshop, Dec 15.

1 3

CSML IIT Lab @pontilgroup.bsky.social · Dec 10

🔗 Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
✍️ R. Grazzi, J. Siems, J. Franke, A. Zela, F. Hutter, M. Pontil
📃https://arxiv.org/abs/2411.12537 | 📅 Oral @ M3L workshop, Dec 14, 15:00 - 15:15.

1 3

CSML IIT Lab @pontilgroup.bsky.social · Dec 10

🌊 Learning the Infinitesimal Generator of Stochastic Diffusion Processes
✍️V. Kostic, H. Halconruy, @tdevergne.bsky.social, K. Lounici, M. Pontil
📃https://arxiv.org/abs/2405.12940 | 📅 Poster #5410 Dec 13, 16:30 - 19:30.

1 3