I’m so ready for the robot X Games

Likewise, voice mode is a qualitatively different interaction

Oh I love that! Having an always-on voice channel for meta-interactions is a clear part of what I’m envisioning

If you're around NeurIPS and want to chat about this stuff, hit me up.

Under the hood, the model will interpret every click and update the UI. In the limiting case, your whole computer is just a large model: it generates pixels, reads your clicks and taps, sends some API calls, then generates the next frame. Generative UI is a whole different computing paradigm.

Models that generate UI will build tools that help us communicate with them. DALL-E generates sliders for your image that control its outputs. A coding model generates a WYSIWYG editor for the web page it built. Instant feedback and rich interaction.

Right now we're a little stuck thinking about AI as a person, which comes with the baggage of how we interact with people. But large models don't have the same limitations as humans.

Seems like a close relative of the Janus problem, but I was always suspicious that guidance was the root cause of that. I don’t expect guidance to have a preference for left/right orientation in general… I bet big-ish models do fine on this