Vincent Herrmann
@vincentherrmann.bsky.social
Working on creativity, curiosity and interestingness. PhD @ IDSIA with Jürgen Schmidhuber in Lugano, Switzerland. Classical pianist.
https://vincentherrmann.github.io
https://vincentherrmann.github.io
This could be a key for building more curious, autonomous AI.
📄 Paper: arxiv.org/abs/2503.13431
💻 Code: github.com/vincentherrm...
🍁 ICML: icml.cc/virtual/2025...
📄 Paper: arxiv.org/abs/2503.13431
💻 Code: github.com/vincentherrm...
🍁 ICML: icml.cc/virtual/2025...
Measuring In-Context Computation Complexity via Hidden State Prediction
Detecting when a neural sequence model does "interesting" computation is an open problem. The next token prediction loss is a poor indicator: Low loss can stem from trivially predictable sequences tha...
arxiv.org
July 17, 2025 at 4:34 PM
This could be a key for building more curious, autonomous AI.
📄 Paper: arxiv.org/abs/2503.13431
💻 Code: github.com/vincentherrm...
🍁 ICML: icml.cc/virtual/2025...
📄 Paper: arxiv.org/abs/2503.13431
💻 Code: github.com/vincentherrm...
🍁 ICML: icml.cc/virtual/2025...
One of the most exciting results: For math problems, reasoning chains with a higher PHi loss are significantly more likely to be correct. The model essentially signals when it's "working hard" to find the right answer.
July 17, 2025 at 4:33 PM
One of the most exciting results: For math problems, reasoning chains with a higher PHi loss are significantly more likely to be correct. The model essentially signals when it's "working hard" to find the right answer.
And it works! PHi Loss cleanly separates "interesting" tasks (like in-context learning, modeling new code/literature) from "boring" ones (memorization, random data), while next-token loss doesn't. We can use pre-trained LLMs or models trained from scratch.
July 17, 2025 at 4:33 PM
And it works! PHi Loss cleanly separates "interesting" tasks (like in-context learning, modeling new code/literature) from "boring" ones (memorization, random data), while next-token loss doesn't. We can use pre-trained LLMs or models trained from scratch.
Our solution: Instead of next token loss, we measure the predictability of the model's hidden state.
We introduce the PHi (Prediction of Hidden states) layer and PHi Loss. High PHi loss means the model's hidden state is complex and unpredictable—a sign of interesting computation.
We introduce the PHi (Prediction of Hidden states) layer and PHi Loss. High PHi loss means the model's hidden state is complex and unpredictable—a sign of interesting computation.
July 17, 2025 at 4:32 PM
Our solution: Instead of next token loss, we measure the predictability of the model's hidden state.
We introduce the PHi (Prediction of Hidden states) layer and PHi Loss. High PHi loss means the model's hidden state is complex and unpredictable—a sign of interesting computation.
We introduce the PHi (Prediction of Hidden states) layer and PHi Loss. High PHi loss means the model's hidden state is complex and unpredictable—a sign of interesting computation.
Standard next token loss is clearly not the right tool:
A model predicting random static (the noisy TV problem 📺) has high loss but isn't doing any interesting work.
Other kinds of data might be accurately predictable, but only after difficult computation🤔
A model predicting random static (the noisy TV problem 📺) has high loss but isn't doing any interesting work.
Other kinds of data might be accurately predictable, but only after difficult computation🤔
July 17, 2025 at 4:31 PM
Standard next token loss is clearly not the right tool:
A model predicting random static (the noisy TV problem 📺) has high loss but isn't doing any interesting work.
Other kinds of data might be accurately predictable, but only after difficult computation🤔
A model predicting random static (the noisy TV problem 📺) has high loss but isn't doing any interesting work.
Other kinds of data might be accurately predictable, but only after difficult computation🤔