We (w/ Moritz Hardt, Olawale Salaudeen and
@joavanschoren.bsky.social) are organizing the Workshop on the Science of Benchmarking & Evaluating AI @euripsconf.bsky.social 2025 in Copenhagen!

📢 Call for Posters: rb.gy/kyid4f
📅 Deadline: Oct 10, 2025 (AoE)
🔗 More info: rebrand.ly/bg931sf

So excited and honored to receive an ERC Starting Grant for the project BrainAlign!! BrainAlign will bring LLMs closer to human understanding by directly aligning them with the human brain.

Stay tuned for our findings, and multiple postdoc and PhD openings in the coming years!

While I’m sad to leave the MPI for Software Systems and its people, it's time to move on. Starting October, I will be a postdoctoral researcher at @hpi.bsky.social, working with @swachter.bsky.social. Super excited about this next chapter!

todo:
* thesis defense ✅

Grateful to the committee and reviewers Marius Kloft, @arkrause.bsky.social, @rupakmajumdar.bsky.social, and @tobigerstenberg.bsky.social for their time and support. No words are enough to thank my advisor @autreche.bsky.social for everything I’ve learned from him so far 🙏

Grateful to the tutorial chairs @urish.bsky.social and @sineadwilliamson.bsky.social for the opportunity, and big thanks to the PC chairs @smaglia.bsky.social and @csilviavr.bsky.social for being on the ground and making UAI 2025 such a unique experience!

Last week I had the pleasure of presenting a 2.5-hour tutorial on "Counterfactuals in Minds and Machines" at UAI 2025 in Rio 🇧🇷, prepared together with @autreche.bsky.social and @tobigerstenberg.bsky.social. We've made all materials and references available here: learning.mpi-sws.org/counterfactu...

Heading to Rio de Janeiro 🇧🇷 for UAI 2025 (@auai.org) to present our tutorial with @tobigerstenberg.bsky.social and @autreche.bsky.social on "Counterfactuals in Minds and Machines" on Monday. Looking forward to this! If you are in Rio, let's meet!

In Athens 🇬🇷 for the Greeks in AI symposium. Super excited to present our work on "Counterfactual Token Generation in LLMs" (bit.ly/4nMibs2) and see all the amazing work Greek people all over the world are doing on AI! If you are in Athens, let's meet! Next, heading to👇

did you check our amazing list of tutorials in Rio?
spanning

- hyperparameter optimization
- counterfactual reasoning
- bayesian nonparametrics for causality
- causal inference with deep generative models
- modern variational inference

👉 www.auai.org/uai2025/tuto...

Awesome work led by Ander, with Nastaran Okati and @autreche.bsky.social.

The LLM API you use returns (and charges you for) 5 tokens. Did the LLM actually generate 5 tokens? Or is the provider overcharging you? 🤔 In arxiv.org/abs/2505.21627, led by Ander Artola Velasco, we argue (game-theoretically) for a change from pay-per-token to pay-per-character.

Presenting this today at 17:00 in Hall 4 #6

This work is a collaborative effort with a fantastic team: Nina Corvelo Benz, Eleni Straitouri, Ivi Chatzi, Ander Artola Velasco, Suhas Thejaswi, and @autreche.bsky.social

In Singapore for #ICLR2025! I'll be presenting our work on a causal methodology for evaluating LLMs (arxiv.org/abs/2502.01754) at the "Building Trust in LLMs" workshop on Monday. If you are working on causality, game theory and/or LLMs, let's grab a ☕️ during the conference!

LLMs rely on randomization to respond to a prompt: they may respond differently to the same prompt if asked multiple times. In “Evaluation of LLMs via Coupled Token Generation” (arxiv.org/abs/2502.01754), we argue that the eval of LLMs should control for this randomization 1/

Let's talk causality and LLMs! Come find us at the posters in East Hall C. 11:30-12:00 & 14:30-15:00. #neurips2024

What would an LLM have said, counterfactually? Here is a short video illustrating our method for counterfactual token generation. We will present this work at the CaLM workshop at #neurips2024. See you in Vancouver!
📜 arxiv.org/abs/2409.17027
💻 made with manim in python

Hey there 🦋
Let's start with an intro. I'm a final-year PhD student at the Max Planck Institute for Software Systems, working on machine learning, decision making and social aspects of AI. Currently on the academic job market, looking for tenure-track positions👇
💻 stsirtsis.github.io