How it works: I prompt Gemini with the first frame of video to give me (x, y) coordinates with descriptions. Then I use optical flow to track those positions over time. Of course optical flow has limitations, but it was a fast lightweight way to prototype the idea quickly.

I made it with Gemini spatial intelligence + opencv.js in Google AI Studio. Link here: ai.studio/apps/drive/1... It's been fun testing on old videos from my photo library as Gemini is able to uncover new details.

Visualizing Gemini intelligence with things in my life 🔊🧵

Gemini spatial understanding + opencv.js 🔊

Testing streaming vector shapes with Gemini 3 Flash. Really fast. Realtime screencap⚡🔊

I always ask Gemini to discuss ideas with me before coding, so I can craft the details I really care about - the sine wave, wobbly-ness, the amplitude ... 🥁

Video has sound 🔊

Sketch ➡️ animation ✏️ Multimodal prompting has been really powerful for this drum machine prototype ... 🧵

Porting features with language ... 🧵

Vibe coding an interface inspired by this fun 1960s drum machine with Gemini.

Polyrhythms are especially easy and fun to make. Song link: alexanderchen.github.io/typeloop/?so...

Every song is represented as a string of text, so you can share it by just copy-pasting or making a link. Here are all the sounds you can try. 🥁

Type Loop 🎵⌨️ Create and share music by typing! Play here: alexanderchen.github.io/typeloop/ 🔊 Open-source. Built w/ Gemini. 🧵 #genuary #genuary10

Dot Motion 🔴🎵 A simple way to explore motion, time, and sound. Code is open-source. Video has 🔊 Link in 🧵 #genuary #genuary2

You can click "+" to add more agents. (Fun to watch the parallel conversations all unfold)

Each agent remembers past conversations and uses memory to decide on goal.

Information is passed through back-and-forth conversations like this one.

TinyTown 🙂🍄 Lightweight multi-agent social simulation built w/ Gemini 3 Flash. Agents look for 🍄. When one finds it, it tells coordinates to others. Not sure where I'm taking this prototype next (open to ideas!) Code is open-source ➡️ ai.studio/apps/drive/1...