✅ End to end generation of expressive performance *audio* from score *images*!

An important step towards seamless interaction with computer music systems and a fun collaboration between Dasaem’s group at Sogang University and my group at CMU

At #CHI2025 in Yokohama this week 🌸. My first CHI, excited to finally get to attend! Happy to chat with anyone about human AI interaction for music or programming

Congrats Kaitlyn and Cornell!!

Also “relative inefficiency of input-space models starts to be economically preferable over the increased engineering complexity of latent-space models”

I wonder about this! If latents shift the scaling laws for generative modeling by an order of magnitude or more, hard to imagine this going away

Incredible post. I still don’t have a clear mental model for the need for *both* perceptual and adversarial losses. Seems like they both encourage preservation of certain higher frequency material. Is using both just a hack that works or is there some more fundamental explanation?

Remarkably thorough and crisp as usual. Probably the single best resource for understanding the latents behind generative modeling that power modern gen AI

Sander shh 🤫 you’re giving away all of the good research ideas!!

I have acquired a Disklavier and Piano Genie has been resurrected :)

@pcastr.bsky.social Disklavier jam session over the internet soon?

Thrilled to share that my *incoming* PhD student Yewon Kim’s work on multimodal inspiration in music AI has been recognized with a Best Paper Award at #CHI2025 🎉

Yewon really knocked it out of the park here. Can't wait to see what she does for her PhD!

arxiv.org/abs/2412.18940

Inaugurating new acct to share work from my PhD student!

Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint

Excited to be evaluating human-AI *workflows* holistically!