Lightnews — Scholar-powered news

Matthias Niessner @niessner.bsky.social · 11h

On the bright side, tooling for training has dramatically improved since then. Deep learning frameworks (PyTorch et. al) and scheduling systems such as SLURM or Kubernetes have become the backbone of modern AI.

1 1

Matthias Niessner @niessner.bsky.social · 11h

Given the humongous compute demands of recent generative frontier AI models -- LLMs, image, and video models, etc. --, where compute is measured in Gigawatts, these challenges seem quite amusing.

1

Matthias Niessner @niessner.bsky.social · 11h

The required compute was typically a couple of GPUs on a single desktop machine, trained over several days; e.g., AlexNet was trained on two GTX 580 3GB GPUs for 5-6 days.

1

Matthias Niessner @niessner.bsky.social · 11h

In the 'early days' of modern deep learning (2012-2015) when ConvNets such as AlexNet or VGG came out, it was considered almost impractical to train an ImageNet classifier from scratch.

2 1

Matthias Niessner @niessner.bsky.social · 14d

Fantastic retreat this weekend by our research groups!

Internal reviews, ideas brainstorming, paper reading, and much more! Of course also many social activities -- the highlight being our kayaking trip - lots of fun :)

10

Matthias Niessner @niessner.bsky.social · 24d

All six of our submissions were accepted to #NeurIPS2025 🎉🥳

Awesome works about Gaussian Splatting Primitives, Lighting Estimation, Texturing, and much more GenAI :)

Great work by Peter Kocsis, Yujin Chen, Zhening Huang, Jiapeng Tang, Nicolas von Lützow, Jonathan Schmidt 🔥🔥🔥

1 10

Matthias Niessner @niessner.bsky.social · 25d

We generate multiple videos along short, pre-defined trajectories that explore the scene in depth. Our scene memory conditions each video on the most relevant prior views while avoiding collisions.

Great work by Manuel Schneider & @LukasHollein

1 3

Matthias Niessner @niessner.bsky.social · 25d

Can we use video diffusion to generate 3D scenes?

𝐖𝐨𝐫𝐥𝐝𝐄𝐱𝐩𝐥𝐨𝐫𝐞𝐫 (#SIGGRAPHAsia25) creates fully-navigable scenes via autoregressive video generation.

Text input -> 3DGS scene output & interactive rendering!

🌍http://mschneider456.github.io/world-explorer/
📽️https://youtu.be/N6NJsNyiv6I

2 2 16

Matthias Niessner @niessner.bsky.social · Aug 5

We further propose a color-based densification and progressive training scheme for improved quality and faster convergence.

shivangi-aneja.github.io/projects/sca...
youtu.be/VyWkgsGdbkk

Great work by Shivangi Aneja, Sebastian Weiss, Irene Baeza Rojo, Prashanth Chandran, Gaspard Zoss, Derek Bradley

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions

shivangi-aneja.github.io

Matthias Niessner @niessner.bsky.social · Aug 5

We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions.

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions

shivangi-aneja.github.io

1

Matthias Niessner @niessner.bsky.social · Aug 5

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH)

We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details.

shivangi-aneja.github.io/projects/sca...

1 6

Matthias Niessner @niessner.bsky.social · Jul 4

TL;DR RGB-D scan as input -> compact, CAD scene representation that also features materials in order to create a digital copy that features the looks of a real environment.

Great work by Zhening (Jack) Huang in collaboration with Xiaoyang Wu, Fangcheng Zhong, Hengshuang Zhao, Joan Lasenby

1

Matthias Niessner @niessner.bsky.social · Jul 4

📢 LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans🏠✨

-> converts RGB-D scans into compact, realistic, and interactive 3D scenes — featuring high-quality meshes, PBR materials, and articulated objects.

📷https://youtu.be/ecK9m3LXg2c
🌍https://litereality.github.io

1 11

Matthias Niessner @niessner.bsky.social · Jun 27

Seven papers accepted at #ICCV2025!

Exciting topics: lots of generative AI using transformers, diffusion, 3DGS, etc. focusing on image synthesis, geometry generation, avatars, and much more - check it out!

So proud of everyone involved - let's go🚀🚀🚀

niessnerlab.org/publications...

1 2 6

Matthias Niessner @niessner.bsky.social · Jun 21

Want to work on cutting-edge #AI?

We have several fully-funded 𝐏𝐡𝐃 & 𝐏𝐨𝐬𝐭𝐃𝐨𝐜 𝐨𝐩𝐞𝐧𝐢𝐧𝐠𝐬 in our Visual Computing & AI Lab in Munich!

Apply here: application.vc.in.tum.de

Topics have a strong focus on Generative AI, 3DGs, NeRFs, Diffusion, LLMs, etc.

3 10

Matthias Niessner @niessner.bsky.social · Jun 10

#CVPR submissions per year have significantly increased.

Now over 11k / year with an expectation to grow even further. This comes with a lot of implications, how to handle the reviews, presentations, etc. Kudos to the organizers for all the efforts that went into it.

1 5

Matthias Niessner @niessner.bsky.social · Jun 10

Super excited to be in Nashville for #CVPR2025!

Looking forward to catching up with everyone -- feel free to reach out if you want to chat!

Everything is a Honky Tonk :)

12

Matthias Niessner @niessner.bsky.social · Jun 9

In addition, we introduce a new OLAT dataset of human heads that features high-resolution and high frame rate multi-view recordings of diverse subjects in a calibrated light stage setting.

Great work by Jonathan Schmidt and Simon Giebenhain.

4

Matthias Niessner @niessner.bsky.social · Jun 9

📢BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading📢

We propose a hybrid neural shading scheme for creating intrinsically decomposed 3DGS head avatars, that allow real-time relighting and animation.

🌍https://lnkd.in/evNt8bV2
📷https://lnkd.in/ekB5QeEK

1 1 12

Matthias Niessner @niessner.bsky.social · Jun 5

📢Code Release of Pixel3DMM 📢
Looking for a robust and accurate face tracker?

We handle challenging in-the-wild settings, such as extreme lighting conditions, fast movements, and occlusions.

👨‍💻https://lnkd.in/e3dX23WV
🌍https://lnkd.in/eQ3Zpn3J

Pixel3DMM can be run on videos and single images.

1 2 22

Matthias Niessner @niessner.bsky.social · Jun 4

📢PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors📢

We propose a new optimization to up-sample textures of 3D assets (albedo, roughness, metallic, and normal maps) by leveraging 2D super-resolution models.

📝http://arxiv.org/abs/2506.02846
📽️https://youtu.be/eaM5S3Mt1RM

2 13

Reposted by Matthias Niessner

TechCrunch @techcrunch.com · May 27

One of Europe’s top AI researchers raised a $13M seed to crack the ‘holy grail’ of models

One of Europe’s top AI researchers raised a $13M seed to crack the ‘holy grail’ of models | TechCrunch

One of Europe’s most prominent AI researchers, Matthias Niessner, is now the CEO of SpAItial, a startup working on spatial foundation models.

techcrunch.com

3 4 26

Matthias Niessner @niessner.bsky.social · May 27

🚀🚀🚀Announcing our $13M funding round to build the next generation of AI: 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 that can generate entire 3D environments anchored in space & time. 🚀🚀🚀

Interested? Join our world-class team:
🌍 spaitial.ai

youtu.be/FiGX82RUz8U

SpAItial AI: Building Spatial Foundation Models

YouTube video by SpAItial AI

youtu.be

4 9 53

Matthias Niessner @niessner.bsky.social · May 9

of 3D hair strand reconstructions from real-world scans of 400 different people, featuring complicated hairstyles, such as ponytails and buns.

🌍 seva100.github.io/GeomHair
📷 youtu.be/h9vqTiFo9As

Great work by Rachmadio L., Artem Sevastopolsky Egor Zakharov, Vanessa Sklyarova

GeomHair

GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans

seva100.github.io

1

Matthias Niessner @niessner.bsky.social · May 9

We enhance the reconstruction with a diffusion prior trained on synthetic hair data and adapted to each scan using a tailored text prompt, allowing us to recover both simple and complex hairstyles without relying on color input.

We also introduce Strands400, the largest publicly available dataset

GeomHair

GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans

seva100.github.io

1 1