Lightnews — Scholar-powered news

Alexandre Défossez @honualx.bsky.social · May 23

Learn more about it with Vaclav, lead engineer for Unmute, getting interviewed by the AI:

Alexandre Défossez @honualx.bsky.social · May 23

We just released unmute.sh 🔇🔊
It is a text LLM wrapper, based on in-house streaming ASR, TTS, semantic VAD to reduce latency. ⏱️
Unlike Moshi 🟢, Unmute 🔊 is turn base, but allows customization in two clicks🖱️: voice and prompt!
Paper and open source coming soon.

1 1 9

Alexandre Défossez @honualx.bsky.social · Apr 1

We just open sourced a fine tuning codebase for Moshi!

Kyutai @kyutai-labs.bsky.social · Apr 1

Have you enjoyed talking to 🟢Moshi and dreamt of making your own speech to speech chat experience🧑‍🔬🤖? It's now possible with the moshi-finetune codebase! Plug your own dataset and change the voice/tone/personality of Moshi 💚🔌💿. An example after finetuning w/ only 20 hours of the DailyTalk dataset. 🧵

4

Alexandre Défossez @honualx.bsky.social · Mar 31

Just back from holidays, so a bit late, to announce MoshiVis, extending Moshi's multimodal capabilities to take in images 📷.
Only 200M weights were added to plug a ViT through cross attention with gating 🖼️🔀🎤
Training relies on a mix of text only and text+audio synthetic data (~20k hours) 💽

Kyutai @kyutai-labs.bsky.social · Mar 21

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images!

It sees, understands, and talks about images — naturally, and out loud.

This opens up new applications, from audio description for the visual impaired to visual access to information.

2 3

Alexandre Défossez @honualx.bsky.social · Mar 13

I'll start my presentation in 10 minutes, you can join in Zoom: concordia-ca.zoom.us/j/81541793947
See you there!

Alexandre Défossez @honualx.bsky.social · Mar 10

I'll present a dive into Moshi 🟢 and our translation model Hibiki 🇫🇷♻️🇬🇧 as part of the next @convai-rg.bsky.social reading group 👨‍🏫📗.

📅 13th of March 🕰️ 11am ET, 4pm in Paris.

I'll discuss Mimi 🗜️ and multi-stream audio modeling 🔊.
Join on Zoom, replay on YT.

⬛ ⬛ 🟧 🟧 🟨 🟨 🟩 🟩 🟩 ⬛
⬛ 🟧 🟧 🟨 🟨 🟩 🟩 🟩 ⬛ ⬛

convai-rg.bsky.social @convai-rg.bsky.social · Mar 10

📢 Join our Conversational AI Reading Group!
📅 Thursday, March 13 | 11 AM - 12 PM EST
🎙Speaker: Alexandre Defossez
📖 Topic: "Moshi: a speech-text foundation model for real-time dialogue"
🔗 Details: (poonehmousavi.github.io/rg)
▶️ Missed a session? Watch on YouTube: (www.youtube.com/@CONVAI_RG) 🚀

Alexandre Défossez @honualx.bsky.social · Mar 10

I'll present a dive into Moshi 🟢 and our translation model Hibiki 🇫🇷♻️🇬🇧 as part of the next @convai-rg.bsky.social reading group 👨‍🏫📗.

📅 13th of March 🕰️ 11am ET, 4pm in Paris.

I'll discuss Mimi 🗜️ and multi-stream audio modeling 🔊.
Join on Zoom, replay on YT.

⬛ ⬛ 🟧 🟧 🟨 🟨 🟩 🟩 🟩 ⬛
⬛ 🟧 🟧 🟨 🟨 🟩 🟩 🟩 ⬛ ⬛

convai-rg.bsky.social @convai-rg.bsky.social · Mar 10

📢 Join our Conversational AI Reading Group!
📅 Thursday, March 13 | 11 AM - 12 PM EST
🎙Speaker: Alexandre Defossez
📖 Topic: "Moshi: a speech-text foundation model for real-time dialogue"
🔗 Details: (poonehmousavi.github.io/rg)
▶️ Missed a session? Watch on YouTube: (www.youtube.com/@CONVAI_RG) 🚀

Pooneh Mousavi

Homepage of Pooneh Mousavi

poonehmousavi.github.io

1 6

Reposted by Alexandre Défossez

Kyutai @kyutai-labs.bsky.social · Feb 11

Even Kavinsky 🎧🪩 can't break Hibiki! Just like Moshi, Hibiki is robust to extreme background conditions 💥🔊.

4 8

Reposted by Alexandre Défossez

Jean-Rémi King @jeanremiking.bsky.social · Feb 11

Very happy to have participated in this *beautiful* documentary from Florent Muller, on the frontiers between humans and machines,
following next @yann-lecun.bsky.social and so many humbling figures of AI:
www.france.tv/documentaire...

France TV - Replay et Direct tv des chaînes France Télévisions (ex Pluzz)

Retrouvez toutes les vidéos, articles et podcasts des programmes des chaînes de France Télévisions.

www.france.tv

2 7

Reposted by Alexandre Défossez

Jean-Rémi King @jeanremiking.bsky.social · Feb 10

Our latest studies on the decoding text from brain activity, reviewed by MIT Tech Review @technologyreview.com

www.technologyreview.com/2025/02/07/1...

6 18

Reposted by Alexandre Défossez

Quentin Berthet @qberthet.bsky.social · Feb 10

Check out our paper, with Lawrence Stewart and @bachfrancis.bsky.social

Link: arxiv.org/abs/2502.02996

1/8

Building Bridges between Regression, Clustering, and Classification

Regression, the task of predicting a continuous scalar target y based on some features x is one of the most fundamental tasks in machine learning and statistics. It has been observed and...

arxiv.org

1 2 9

Alexandre Défossez @honualx.bsky.social · Feb 10

Excited to meet and exchange with a number of actors from all around the world at the AI Summit 🌍

2

Alexandre Défossez @honualx.bsky.social · Feb 7

We just released Hibiki, a 🎙️-to-🔊 simultaneous translation model 🇫🇷🇬🇧
We leverage a large synthetic corpus synthesized from the text translation model MADLAD, and our own TTS + simple lag rule.
Model is decoder only, runs at scale, even on device 📲
github.com/kyutai-labs/hibiki

Kyutai @kyutai-labs.bsky.social · Feb 7

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧.
Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech. 🧵

1

Reposted by Alexandre Défossez

Jean-Rémi King @jeanremiking.bsky.social · Jan 15

🚨Job alert (Please RT)

What: masters internship and/or PhD positions
Where: Rothschild Foundation Hospital (Paris, France)
Topic: AI and Neuroscience
Supervised by: Pierre Bourdillon and myself
Apply here: forms.gle/KKnea2QAjhAe...
Deadline: Feb 5th

forms.gle

11 15

Alexandre Défossez @honualx.bsky.social · Jan 13

We just released the Helium-1 model , a 2B multi-lingual LLM which @exgrv.bsky.social and @lmazare.bsky.social have been crafting for us! Best model so far under 2.17B params on multi-lingual benchmarks 🇬🇧🇮🇹🇪🇸🇵🇹🇫🇷🇩🇪
On HF, under CC-BY licence: huggingface.co/kyutai/heliu...

8 25