banner
jonbarron.bsky.social
Jon Barron
@jonbarron.bsky.social
3.6K followers 150 following 250 posts
AI researcher at Google DeepMind. Synthesized views are my own. 📍SF Bay Area 🔗 http://jonbarron.info This feed is a partial mirror of https://twitter.com/jon_barron
jonbarron.info
Posts Media Videos Starter Packs
Pinned
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 28
Here's a recording of my 3DV keynote from a couple weeks ago. If you're already familiar with my research, I recommend skipping to ~22 minutes in where I get to the fun stuff (whether or not 3D has been bitter-lesson'ed by video generation models)

www.youtube.com/watch?v=hFlF...
Radiance Fields and the Future of Generative Media
YouTube video by Jon Barron
www.youtube.com
2 12 60
Reposted by Jon Barron
paulgavrikov.bsky.social
Paul Gavrikov @paulgavrikov.bsky.social · Sep 8
Is basic image understanding solved in today’s SOTA VLMs? Not quite.

We present VisualOverload, a VQA benchmark testing simple vision skills (like counting & OCR) in dense scenes. Even the best model (o3) only scores 19.8% on our hardest split.
2 5 16
Reposted by Jon Barron
hpdailyrant.bsky.social
Hà Phan @hpdailyrant.bsky.social · Jul 16
Here’s what I’ve been working on for the past year. This is SkyTour, a 3D exterior tour utilizing Gaussian Splat. The UX is in the modeling of the “flight path.” I led the prototyping team that built the first POC. I was the sole designer and researcher on the project, one of the 1st inventors.
5 5 67
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Jul 3
Ah cool, then why is that last bit true?
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Jul 3
I don't see how the last sentence follows logically from the two prior sentences.
1
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Jul 3
Be sure to do a dedication where you thank a ton of people, it's kind plus it feels good.

Besides that I'd just do a staple job of your papers. Doing new stuff in a thesis is usually a mistake, unless you later submit it as a paper or post it online somewhere. Nobody reads past the dedication.
1 3
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Jun 29
This thread rules
3
Reposted by Jon Barron
niessner.bsky.social
Matthias Niessner @niessner.bsky.social · May 27
🚀🚀🚀Announcing our $13M funding round to build the next generation of AI: 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 that can generate entire 3D environments anchored in space & time. 🚀🚀🚀

Interested? Join our world-class team:
🌍 spaitial.ai

youtu.be/FiGX82RUz8U
SpAItial AI: Building Spatial Foundation Models
YouTube video by SpAItial AI
youtu.be
4 9 53
Reposted by Jon Barron
uoftcompsci.bsky.social
U of T Department of Computer Science @uoftcompsci.bsky.social · Apr 30
📺 Now available: Watch the recording of Aaron Hertzmann's talk, "Can Computers Create Art?" www.youtube.com/watch?v=40CB...
@uoftartsci.bsky.social
4 10
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 28
Here's a recording of my 3DV keynote from a couple weeks ago. If you're already familiar with my research, I recommend skipping to ~22 minutes in where I get to the fun stuff (whether or not 3D has been bitter-lesson'ed by video generation models)

www.youtube.com/watch?v=hFlF...
Radiance Fields and the Future of Generative Media
YouTube video by Jon Barron
www.youtube.com
2 12 60
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 24
www.instagram.com/mrtoledano/ for anyone else who wanted to see more of this artist's work, really cool stuff!
Login • Instagram
Welcome back to Instagram. Sign in to check out what your friends, family & interests have been capturing & sharing around the world.
www.instagram.com
2
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 9
yeah those fisher kernel models were surprisingly gnarly towards the end of their run.
1
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
yep absolutely. Super hard to do, but absolutely the best approach if it works.
4
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
If you want you can see the models that AlexNet beat in the 2012 imagenet competition, they were quite huge, here's one: www.image-net.org/static_files.... But I think the better though experiment is to imagine how large a shallow model would have to be to match AlexNet's capacity (very very huge)
www.image-net.org
1 2
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
One pattern I like (used in DreamFusion and CAT3D) is to "go slow to go fast" --- generate something small and slow to harness all that AI goodness, and then bake that 3D generation into something that renders fast. Moving along this speed/size continuum is a powerful tool.
1 7
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
It makes sense that radiance fields trended towards speed --- real-time performance is paramount in 3D graphics. But what we've seen in AI suggests that magical things can happen if you forgo speed and embrace compression. What else is in that lower left corner of this graph?
2 4
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
And this gets a bit hand-wavy, but NLP also started with shallow+fast+big n-gram models, then moved to parse trees etc, and then on to transformers. And yes, I know, transformers aren't actually small, but they are insanely compressed! "Compression is intelligence", as they say.
1 3
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
In fact, it's the *opposite* of what we saw in object recognition. There we started with shallow+fast+big models like mixtures of Gaussians on color, then moved to more compact and hierarchical models using trees and features, and finally to highly compressed CNNs and VITs.
2 4
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
Let's plot the trajectory of these three generations, with speed on the x-axis and model size on the y-axis. Over time, we've been steadily moving to bigger and faster models, up and to the right. This is sensible, but it's not the trend that other AI fields have been on...
1 5
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
Generation three swapped out those voxel grids for a bag of particles, with 3DGS getting the most adoption (shout out to 2021's pulsar though). These models are larger than grids, and can be tricky to optimize, but the upside for rendering speed is so huge that it's worth it.
1 7
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
The second generation was all about swapping out MLPs for a giant voxel grid of some kind, usually with some hierarchy/aliasing (NGP) or low-rank (TensoRF) trick for dealing with OOMs. These grids are much bigger than MLPs, but they're easy to train and fast to render.
1 4
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Apr 8
A thread of thoughts on radiance fields, from my keynote at 3DV:

Radiance fields have had 3 distinct generations. First was NeRF: just posenc and a tiny MLP. This was slow to train but worked really well, and it was unusually compressed --- The NeRF was smaller than the images.
2 21 92
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Mar 19
Here's Bolt3D: fast feed-forward 3D generation from one or many input images. Diffusion means that generated scenes contain lots of interesting structure in unobserved regions. ~6 seconds to generate, renders in real time.

Project page: szymanowiczs.github.io/bolt3d
Arxiv: arxiv.org/abs/2503.14445
1 6 34
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Mar 19
I made this handy cheat sheet for the jargon that 6DOF math maps to for cameras and vehicles. Worth learning if you, like me, are worried about embarrassing yourself in front of a cinematographer or naval admiral.
5 34
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Mar 11
It's certainly a shocking result, but I think concluding that "Sora learned 3D consistency" isn't a totally well-founded claim. We have no real idea what any models learn under the hood, and it should be possible for models to produce plausible videos without actually doing anything "in 3D".
4
jonbarron.bsky.social
Jon Barron @jonbarron.bsky.social · Mar 3
But if there's an actual physical conference with talks and posters, there would need to be an actual cutoff date for submissions to be presented at this year's conference. Wouldn't that become the de facto deadline, even in a theoretically rolling system?
1 2