AI Engineer Europe 2026
Reachy Mini: the $300 open source robot you can actually hack — Andres Marafioti, Hugging Face
Read the talk
Reachy Mini: building an affordable robot people can make their own
A small, expressive robot opens up a larger engineering problem: connecting voice, vision, motion, and fast inference so people can invent their own ways to interact with machines.
From a talk by Andres Marafioti
Before you start: Familiarity with language models and basic CPU/GPU inference will help with the serving discussion; no robotics background is required.
Who gets to experiment with robots?
At a robotics club in Zurich, Andres Marafioti watched humanoid robots box. The experience was impressive and a little disturbing: machines were becoming capable enough to fight, while companies were proposing home robots that could water plants or scroll TikTok. Marafioti, who introduces himself as Andy and leads multimodal research at Hugging Face, starts with the gap between that progress and who can afford to experiment with it.
Marafioti puts the robots he is discussing at roughly $50,000 and upward. He estimates Waymo vehicles in the mid-six figures; his aside about their presence in London is something he has heard, not seen. These are expensive platforms on which to discover what an everyday interaction with a robot should feel like.
Their shapes impose another constraint. A humanoid looks familiar, which helps someone form an expectation of its capabilities. But hardware does not have to follow biology. Marafioti proposes a spider-shaped machine as a potentially faster, more agile, more stable alternative. Copying a human body narrows the design space before anyone has explored what other forms might offer.
The practical test is a classroom: a high school is unlikely to buy ten $50,000 humanoids for students to play with. Enterprise-oriented machines are also difficult to adapt, and their appearance does not necessarily invite connection. Robotics capability is advancing faster than access to robotics experimentation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Voice is ready; the interaction is still open
Voice software offers a different starting point. Commercial options include Gradium, GPT Realtime, and the ubiquitous, if imperfect, Siri. Open models include Voxtral and Kokoro, which Marafioti highlights as an 80-million-parameter model that sounds good. Speech-understanding models and Hugging Face’s speech-to-speech pipeline provide the pieces for assembling a voice agent yourself.
Having those pieces does not settle how people should talk with robots. Marafioti wants that experience to develop through broad experimentation, much as computers developed through a hacker culture, rather than being determined by a single company or group. Reachy Mini is designed for hackers, researchers, students, and anyone with a computer who wants to try something.
Its expressive, nonhuman form is deliberate. It can be something you talk to without presenting itself as a replacement person. That changes the starting question from how to imitate a human to what kinds of relationships and behaviors this new object might support.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A kit that invites modification
Reachy Mini ships unassembled. Building it is meant to teach owners how its parts fit together, so replacing a broken component becomes a familiar operation: order the part and swap it. Marafioti reports positive feedback on assembly and sees that knowledge as central to repairability. Ease of use, he acknowledges, is still improving; the hardware comes with software tools intended to make development approachable.
The talk presents two hardware options at the following historical prices; the official overview has since updated its pricing.
| Version described in the talk | Price | Onboard Raspberry Pi | Battery |
|---|---|---|---|
| Lower-cost version | $300 | No | No |
| Self-contained version | $450 | Yes | Yes |
The cheaper version suits people who do not need onboard compute or a battery. Marafioti says it is also sold in bulk to high schools and universities for experimentation.
Community modifications make the idea concrete. One owner turns the robot into a Halloween pumpkin using 3D-printed parts, replacement antennas, a covering over the body, and lights. The underlying robot needs no major redesign: changing its exterior is enough to give it a different character.
Another owner discovers that petting the robot and having it react can be enjoyable in its own right. An audience member asks whether it purrs. Marafioti’s answer is that you can make it purr—not that purring is already a built-in behavior. These experiments let owners develop their own intuition for what the robot should be.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A conversation that reaches into the physical world
Taking out a keyboard to address a humanoid would be an awkward default interaction. Voice is a more natural candidate, so the Reachy Mini Conversation App supplies tools for building that experience. Marafioti introduces a prerecorded demonstration, filmed on his phone that morning.
The robot begins with a lobster-themed personality: it is functioning within acceptable parameters, but a good lobster joke would improve its day. Asked to take a photo and describe what is in front of it, it reports a person taking a mirror selfie in what looks like a hotel room, wearing a white T-shirt with a yellow emoji and orange pants. It then turns the visual description into another lobster joke, this time at the pants’ expense.
The interlocutor promises to change before the talk, then asks the robot to show happiness. Its spoken response agrees, while warning against expecting a lobster-themed dance of joy. The sequence brings together personality, a camera request, a visual description, and an emotion request within one conversation. After the recording, Marafioti offers attendees the chance to talk to the physical robot themselves.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separating the robot, speech pipeline, and inference service
Marafioti describes the models and agents as open source, but running them locally still requires compute that many owners do not have. Hosted inference makes the conversation app accessible without a local GPU. Marafioti reports 7,500 robots shipped and says conversation is their most-used app. That creates a serving problem as well as a robot-control problem.
The system has three layers. At its center is the speech-to-speech project Marafioti has maintained at Hugging Face for about two years. Its turn-processing path is:
- Detect speech. Voice activity detection identifies when someone is talking.
- Transcribe incrementally. Parakeet converts speech to text. Marafioti says the deployment transcribes every 150 milliseconds and returns partial transcriptions so the robot can react before the utterance is complete.
- Generate a response and actions. The completed transcription goes to an LLM, which can reply and call tools for movement or camera use.
- Synthesize speech. Qwen3-TTS turns the response into audio.
Above that pipeline, the robot application owns the physical interaction: microphone input, speaker output, echo cancellation, tool dispatch, emotion behaviors, camera access, and face tracking. Echo cancellation matters because the robot must hear the person without treating its own speech as fresh input. Tool dispatch connects the language model’s requested actions to the robot’s capabilities. Owners can modify or replace the speech pipeline; the default service runs on Hugging Face Inference Endpoints.
The deployment layer scales conversation compute according to the number of connected robots, while LLM inference runs on separate endpoints. Marafioti says the team is using Qwen 3.5 27B, having made it fast enough for this application. Conversation workers have a practical concurrency limit before latency rises too far, but their connection count alone does not describe the LLM load.
Consider his example: one worker has eight users talking frequently, while another has eight users who are silent. Both workers have the same number of connections, yet their demand for language-model inference is very different. Separating LLM capacity from conversation workers allows it to scale with actual usage, reducing resources tied up behind quiet connections.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Making Qwen3-TTS fast enough to converse
Qwen3-TTS offered the speech quality Marafioti had been waiting for, but the released implementation did not deliver the latency he expected from the technical report. He spent roughly two weeks working with Codex to improve it. The distinction is between the released implementation and the paper’s serving system: the latter used an internal vLLM engine with compilation and CUDA graph optimizations. Its latency results were not measurements of the default implementation.
The first obstacle was output delivery. The implementation generated the whole response before returning audio. For a ten-second utterance, a listener had to wait until all ten seconds had been generated before hearing the beginning. Streaming separates the start of playback from completion of generation: the first available audio can be delivered while later audio is still being produced.
The next obstacle was repeated CPU–GPU coordination during autoregressive generation. Marafioti describes this as 500 steps per audio packet, with coordination at each step. The optimization target is the repeated host involvement in that loop. Capturing and replaying GPU work can reduce the overhead, but the model’s dynamic key-value cache was changing shape as the input grew, obstructing the execution strategy he wanted.
The team replaced the dynamic cache with a static KV cache, allocating more memory up front to keep its shape stable. That enabled CUDA graph capture and faster generation.
| Change | What it addresses | Tradeoff or requirement |
|---|---|---|
| Stream audio | Waiting for the entire utterance | Deliver usable chunks incrementally |
| Use a static KV cache | Changing cache shapes | Allocate more memory up front |
| Capture CUDA graphs | Repeated CPU–GPU coordination | Keep captured execution stable |
These changes address different parts of the delay: streaming advances delivery, while stable GPU execution accelerates the work producing the audio.
Marafioti reports a generation-speed factor rising from 0.8 to 5.8 times real time. Interpreting those numbers as audio seconds produced per wall-clock second makes their reciprocal the time needed to generate one second of audio:
python
for speed in (0.8, 5.8):
milliseconds_per_audio_second = 1000 / speed
print(f"{speed:.1f}x real time: {milliseconds_per_audio_second:.0f} ms")
That gives approximately 1,250 milliseconds and 172 milliseconds, respectively, consistent with his corrected verbal approximation of about 1.2 seconds versus 200 milliseconds. Marafioti also reports that time to first audio fell from several seconds, depending on the output, to a few milliseconds. The talk does not specify hardware, workload, or warmup conditions for these figures.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The first audio packet is not the whole latency budget
A second recorded demonstration shows a Hugging Face Space cloning selected voices and synthesizing pasted text in real time. One sample gives the recurring lobster a new voice; another describes reconstructing a voice from a short fragment of sound. The demonstration makes the intended interaction tangible: choose a voice, supply text, and begin hearing the result promptly.
The synthesized demo narration claims first-token latency under 200 milliseconds and generation at four times real time. These are separate claims from the preceding spoken figures, with different terminology and no shared measurement conditions supplied. Marafioti names the open-source implementation FasterQwen3TTS, available independently or through Reachy Mini’s voice agent. The current repository’s benchmark table also distinguishes throughput from relative speedup: a 5.8-fold speedup is not the same quantity as generating at 5.8 times real time.
Marafioti says the online demo has seen consistent daily use since its release roughly two months earlier. But the model’s time to first audio is only part of what a listener experiences. Requests and audio must also pass through the serving infrastructure. In this deployment, Marafioti says infrastructure overhead takes about as long as model processing. Once synthesis becomes fast, that surrounding overhead becomes a substantial part of the user’s wait. A responsive voice agent therefore needs an end-to-end latency budget, not just a fast model measurement.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build beyond the supplied voice agent
The supplied conversation experience is a starting point. Marafioti wants people to develop their own interactions, with that work open to a community rather than gated behind expensive robots. Providing hardware, models, and application tools serves the same goal: letting more people participate in deciding how robots behave around us.
For the movement application running on the onstage robot, Marafioti says he gave Codex the robot repository and a description of the desired behavior, and it produced the app in one shot. He uses that example to invite even noncoders to begin experimenting. The result he points to is a specific movement app, demonstrating how an accessible repository can shorten the path from an idea to a behavior on the robot.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Where applications run, and what extending them requires
The Q&A first clarifies where applications execute. Many apps that do not require a GPU can run directly on the Raspberry Pi-equipped robot, within that hardware’s limits. A laptop can supply compute instead. Marafioti names Java and Python as possibilities and tentatively suggests HTML; this is a claim about flexibility in building applications, not a list of established SDKs. The official hardware overview explicitly documents Python support.
Hardware extensibility requires more hands-on work. Asked about plugins, extra servos, or making the robot move around a room, Marafioti says some hacking is necessary. The team tries to make its open-source robots fit together: he cites the SO-100 and SO-101 arms, and a three-wheel base called Kiwi designed to accept Reachy Mini. He has not seen that mobile combination implemented, though he says the parts were designed to fit. Physical composability is an invitation to build, not a demonstrated turnkey mobility system.
The final question returns to Qwen3-TTS: beyond streaming and speed, what was missing? Marafioti describes the original implementation as functionally quite complete but generally slow, with the feel of a research implementation. He says Qwen’s hosted API is fast and speculates that the gap might encourage API use; that commercial explanation is his interpretation.
His own optimized implementation has a different maintenance burden. Users report cases that do not work because the model supports more behaviors than his initial optimization covered. Those reports require additional implementation. The closing distinction is useful for anyone adapting research software: preserving a model’s range of capabilities while making its common path fast is ongoing engineering work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Marafioti's streaming TTS implementation, with CUDA graph acceleration, voice-cloning examples and hardware-specific benchmarks.
Architecture and evaluation of Qwen3-TTS, including the serving conditions behind its first-packet latency measurements.
A configurable voice-agent pipeline connecting speech recognition, language models and speech synthesis.
Conversation software connecting Reachy Mini to voice backends, camera tools, expressive motion and customizable personalities.
Further reading
Official overview of the kit, hardware variants, programming support and community features; pricing has been updated since the talk.
Read the complete timestamped transcript
- 0:00
[upbeat music] Um, can I, are you hearing me well?
- 0:18
No. Okay.
- 0:19
Well, I think we hear you.
- 0:20
Yeah, you-- yeah, now, now with the speakers. Good. Hey, how are you, everyone? Uh, my name is Andy Marafioti. Um, I'm going to talk today about this little robot called Reachy Mini.
- 0:31
Um, I'm going to try to explain to you why we developed it, why we think it's important, and what we are trying to do with it. Um, and yeah, about me, I lead multimodal research at Hugging Face, which I don't have time to explain very much, but I hope you know Hugging Face.
- 0:46
Good. So first, very quickly, the state of robotics. Um, we are actually making strides in robotics. I think for the day-to-day life, it's not very clear, but robots are just coming and, uh, they're coming at neck-breaking speed.
- 1:00
We have really, really good humanoids nowadays. Um, a couple of weekends ago, I was at the robotics club in Zurich, Switzerland, and we had a little boxing match between humanoid robots.
- 1:12
Uh, it was incre-incredible and a little bit disturbing. Um, there are also several companies today trying to sell you robots to put in your homes to do things like water your plants and scroll TikTok doomlessly.
- 1:26
Um, and of course, we have self-driving cars. I think Waymo is now in London, I heard. I haven't seen them, but pretty cool. Um, now, something here that I always see with the strides that we're making with robotics is, A, these things are really, really expensive.
- 1:43
They are all at least in mid-five figure range, so fifty K up. Um, and they don't seem to really be coming down. They seem to be targeting that price range for now.
- 1:55
Uh, not to say the humanoids. The Waymo, I think they are, like, six figures mid. [laughs]
- 2:01
Uh, and they also, they kind of look like something that you know. So it's not very much about being creative. It's very much about trying to imitate reality. And this is not biology.
- 2:12
This is really hardware, and we are really constraining the robots to do this. Uh, if you take a humanoid robot, it could look like a spider, uh, and just move around way faster, be more agile, be more stable.
- 2:28
But we're actually making it be a humanoid such that we look at it and we think like, "Oh, yeah, robot, human. It's the same, sort of. Like, I understand what it can do."
- 2:39
Uh, that to me is a mistake. They also don't look very friendly in general. Uh, so I see a few problems with the current state of robotics. All of these robots, which are making strides, and they're becoming really, really good, they are still too expensive to prototype.
- 2:56
I don't see any high school anywhere in the world ordering ten of these fifty K humanoid robots to let their students play with. Um, they're very complex to adapt.
- 3:06
They're really, like, targeting companies. And they don't-- you cannot really connect with them. Now, as you saw today already, you-- we can actually build pretty good voice agents. The state of voice AI is quite advanced.
- 3:21
There are really good commercial solutions. There's, um, Gradio in Paris developing voice agents. There's GPT Realtime, which I'm sure most of you have played with. Uh, you can chat with it, and it re- will reply, and it sounds fairly, uh, natural.
- 3:36
There's Siri, which has its problems, but it's in all iPhones. Hey, kudos. Uh, and in the open source, we also have a lot of really good models. Uh, you heard the talk from Mistral, maybe if you were here before.
- 3:48
Voxral is a really good open source model. We also have tiny models like Kokoro at just eighty million parameters that sound good. We have good models to understand speech, and we have good pipelines like speech-to-speech from Hugging Face to put all together and make your voice agents yourself.
- 4:06
So what I see here is that voice AI is quite mature. We have the tools. We have the tools commercially. We have the tools in the open source. But still, no one is really working on how we're going to talk with these robots.
- 4:20
And we are thinking, okay, given that robots are coming, how can we manage to put this in the hands of everyone such that it's-- the experience of the future is not dominated by one company or one group of people, but really it's made, um, like computers were made, in a bit more of a hacker fashion?
- 4:41
And that's how we came up with Reachy Mini. It's a little bit our response to that. We had-- We made it targeting really, uh, hackers, researchers, students, dreamers. Uh, if you have a computer, you should be able to play with it and make things.
- 4:58
That's a little bit our target. Um, a little bit of our vision. The idea is that it's an expressive robot, that you can talk to it, but it doesn't look like a human.
- 5:10
And that, for us, is very important because it already puts your mind in a different place creative-wise, that you are going to start developing new ways of interacting with this.
- 5:20
And it's not going to be just, okay, this is a human replacement. It's not. It's a new thing. Uh, and we want people to be able to explore that.
- 5:28
Uh, we made it affordable. We made it easy to use. Well, we are making it easier to use. [laughs] We made it repairable. Actually, we ship these robots unassembled, so the first thing you do when you get the robot is you need to assemble it yourself.
- 5:43
And once you're done with it, which we get super positive feedback from, you can basically repair anything that's gonna break in the robot ever. You have all of the knowledge.
- 5:52
You have all of the tools. Um, it really just is ordering the p- the parts and changing it. It's very hackable. It's very cute. That's maybe a personal opinion, but I think it's cute.
- 6:03
And we are trying to give you a set of software tools, uh, for you to develop with. Uh, now very quickly, we sell two of these robots. Uh, they are both, uh, four hundred and fifty and three hundred USD.
- 6:17
And the difference between both of them is the four hundred and fifty, which is this one that I have here, has a Raspberry Pi inside, and it has a battery.
- 6:26
And that's it. Basically, if you don't need a battery, you don't need a Raspberry Pi, you can order the cheaper one. Uh, we are selling those cheaper ones in bulk for high schools or universities to play with, to experimen- to experiment, to hack with.
- 6:40
Um, and here I wanted to show you a few, um, cool developments that we are seeing. For example, in the middle, what I mean with hackable, people can 3D print new parts for the robot, and in this case, they didn't need to really change anything.
- 6:54
They're just taking the antennas out, putting a different type of antenna. They are putting something on top of the body. They are putting some lights, and they have suddenly a little Halloween pumpkin for, to play with.
- 7:07
Um, on the right, something that we hadn't thought about, someone figured, "Oh, you can just pet the robot, and it can be fun." [laughs] And it reacts.
- 7:16
Does it purr?
- 7:17
Um [audience laughing] you can make it purr. [laughs]
- 7:21
Okay.
- 7:21
Uh, so that's, that's a little bit like, for us, it's more about people exploring and people developing their own intuition for how this should be and, and making it their own.
- 7:34
But I was talking to you about voice AI. We think one of the main ways to talking with robots is gonna be through voice. I don't see anyone going to a humanoid robot and taking out the keyboard and typing something.
- 7:48
That, that's not going to be what happens. Eh, so we need to give people the tools to develop voice, uh, experiences. Uh, and we are doing that. We have a piece of software to, for you to co- to converse with the robot.
- 8:02
I want to show you a little demo. I'm not as brave as Sam from Mistral who did the live demo. I'm gonna show you a video. Um, please for the audio people, if you can maybe put it up a little bit.
- 8:14
I did this this morning with my phone.
- 8:19
Hey, Reachy. How are you doing?
- 8:19
Functioning within acceptable-
- 8:21
Okay. Maybe a little bit higher. [laughs] Yeah.
- 8:29
Hey, Reachy. How are you doing?
- 8:33
Functioning within acceptable parameters.
- 8:36
That demo said, "Stop." You, France. [audience laughing] You're supposed to be my friend. [laughs]
- 8:43
I'm good. Um, could you take a photo and tell me what you-
- 8:46
Uh, is it now a little bit higher? I, I don't hear it.
- 8:49
Hey, Reachy. How are you doing?
- 8:50
Okay.
- 8:52
Functioning within acceptable parameters. I'm still waiting for someone to tell me a good lobster joke to really get my circuits buzzing. How about you?
- 9:02
I'm good. Um, could you take a photo and tell me what you see in front of you?
- 9:09
Snap a picture and see what's in front of me. Though I suspect it's just a very boring wall, which is basically the lobster equivalent of a rock. I see a guy taking a mirror selfie in what looks like a [REDACTED:location], wearing a white T-shirt with a yellow emoji on it and orange pants.
- 9:26
Honestly, the only thing more crusted than those pants is a lobster that's been sitting on the beach too long.
- 9:34
Okay, okay, I'll change my pants before the talk. Um, but could you first show me an emotion? Let's do happiness.
- 9:43
I'll show you happiness, [camera clicking] though I hope you're not expecting me to burst into a lobster-themed dance of joy.
- 9:50
Okay, so that quickly for, for the conversation. If you guys want to do live demos, I have the robot here. I leave it. You can talk to it. [audience laughing]
- 9:58
Um, and what we're trying to do is we're trying to give people the tools to actually, uh, do the conversation. So everything that we are doing with the robot is open source.
- 10:08
All of these models are open source. The agents are open source. Eh, but we understand also that people are a little bit GPU poor maybe, and they cannot run out of those models locally.
- 10:17
We are also a little bit GPU poor, and we have now shipped seven thousand five hundred of these robots, uh, which means we have a pretty sizable fleet of people talking to the robot.
- 10:27
It's our most used app by far. People just like putting it there and talking to it. Uh, and because you guys are a very technical audience, I wanted to show you a little bit how we are actually serving this.
- 10:39
Uh, there are three levels to this system. In the middle, there's the speech-to-speech pipeline. That's a project that I've been maintaining with Hugging Face for the last two years.
- 10:49
Uh, the Mistral, Sam from Mistral told you a little bit how it works. You have a voice activity detection system that knows if you are talking or, or not.
- 10:57
That sends it to a speech-to-text system. In our case, we are using Parakeet because it's super fast. So we transcribe every hundred and fifty milliseconds, and we send back the partial transcriptions for the robot to react if it hears something interesting.
- 11:11
And when the transcription is, is done, we send it to an LLM. The LLM replies, can also do tool calling for movement or to use the camera, and then we send it to the text-to-speech system, which we are using Qwen3 TTS.
- 11:24
Now, on the higher level, we have the, our actual conversation app running the robot. That is taking the input and the output from, um, input from microphone and output through the speakers.
- 11:35
It's doing the echo cancellation to not hear itself when it's talking and to tr- to hear you. It's doing the tool dispatching to move, do emotions, and it's using the camera.
- 11:44
It can do face tracking to follow you around, um, that sort of thing. And then the speech-to-speech pipeline, if you don't want to tweak it and use your own, we have it deployed in Hugging Face inference endpoints.
- 11:57
We have a load balancer that determines how many compute nodes we have at, at each time, uh, depending on how many robots are connected. And we have separated the LLM inference endpoints.
- 12:07
We are actually using now Qwen three point five twenty-seven B 'cause we managed to make it fast enough. [laughs] Um, but we scale that differently because each of the conversation nodes can have an amount of concurrent users without the latency spiking up too much.
- 12:22
And it changes a lot if in one node you have eight users that are talking a lot and using a lot the LLM, and in another node you have eight users that are not talking at all.
- 12:31
Uh, so it really helps to save on resources if you have the LLM separated. Um, and to show you a little bit of how far we are going into trying to make this work and also, uh, benefiting from the fact that you are a technical audience, I'm going to talk a little bit about, uh, Qwen3-TTS.
- 12:49
So this is a model from Qwen that came out two or three months ago. For us, it was really, uh, a really great moment because TTS models hadn't been as good in the open source and as fast as Qwen3-TTS when it came out, and we were really expecting for something like that.
- 13:07
We knew it was coming, but we didn't know when it was gonna come. Uh, the issue is that the model that they released actually ended up being of the quality that they showed you but not really of the speed that they showed you.
- 13:20
Uh, the paper claimed very low latencies, but the model didn't really achieve them. So I spent maybe two weeks with Codex trying to get the model trained by them to actually be fast enough for voice agents.
- 13:34
Um, and I wanted to tell you a little bit, a little bit how I did that and what the main issues were. A first issue that I found is that the model would generate the whole output before giving it to you, so y- it wasn't streaming, uh, which meant if you wanted ten seconds of audio, it needed
- 13:50
to generate the ten seconds. That can be solved with streaming. Uh, it's harder in practice [laughs] than in theory as things usually are, but when it works, it works and it's really cool.
- 14:01
The next thing that I noticed was the model being an autoregressive model was doing five hundred steps for each packet of audio that it was generating and for each of those five hundred steps it needs to coordinate the CPU with the GPU.
- 14:15
So it needs to send data back and forth between the CPU and the GPU. That is pretty bad. The way to solve that is to compile your model so that all of those interactions happen directly in the GPU.
- 14:27
That couldn't happen by default because it was using a dynamic KV cache. So the KV cache was evolving depending with the size of the inputs that it was processing.
- 14:36
We changed that for a static KV cache. We used more RAM from the get go, but that makes it faster. And then we could use CUDA graph captures to capture the whole model and to be able to accelerate generation significantly.
- 14:50
Uh, we went on a real-time factor from being below real time, so at zero point eight, which means you generate one second you take one point two seconds, uh, to being five point eight, which means for one second you take twenty seconds, I guess.
- 15:04
Uh, no, for one second you take two hundred milliseconds. Yes. [laughs] Thank you. And we also reduced obviously the time to first audio significantly from several seconds, depending on what you're generating, to a few milliseconds.
- 15:19
Uh, and I wanted to show you now a quick demo, not live, recorded of this model on a Space on Hugging Face. You can go and test it yourself.
- 15:28
I cloned a few voices and I make the model say things. Uh, this happens all in real time. It does the cloning of the voices that I chose, and it generates the text that I'm copy-pasting.
- 15:39
Against the odds, the wild lobster has found a new vessel for its voice and with it the possibility to realize its full potential.
- 15:53
Give it a moment of sound, just a fragment, and it will give you back a voice that feels almost human.
- 16:04
The original system works, but only offline and at sub-real time speeds. FasterQwenTTS brings first-token latency under two hundred milliseconds and runs at four X real time. Once you cross that threshold, entirely new applications open up.
- 16:18
Great. That, that is also open source. Now you can go look for FasterQwen3TTS and use it or you can use it with our voice agent from, uh, Reachy Mini.
- 16:27
You can also test this demo online. Uh, I released it like maybe two months ago and I think I- it has been like used every day consistently. Um, and something that I wanted to highlight quickly here, there's a difference between the time to first audio of the model and what the client perceives, 'cause on top of the
- 16:47
basic thing that people are telling you of the model there's all of the infrastructure times that actually add up a lot. In this case the infrastructure times are just as much as just the model because the model is quite fast, right?
- 17:01
But still when you're thinking of voice agents you need to consider all these things and how they add up. Uh, and going back, what we want you to do is get this robot and make it your own and create your own interactions.
- 17:16
We don't want you to just stop at using our voice agents. We are trying to give people the tools 'cause we want... We know that robots are coming. We know that they are gonna be everywhere and we want how we interact with those robots to be communal, to be dev- uh, developed by everyone that wants to develop
- 17:34
it, and we don't want it to be guarded behind fifty K robots. We want people to actually be working on this. So we try to put as many resources as we can towards that.
- 17:45
Uh, and the last thing, you can basically vibe code things with the robot. The application that I had today now of the robot just running here doing some movements, I once showed it with Codex.
- 17:57
I told it like, "Hey, here's the repo for the robot. Here's what I want it to do. Make it." And it just did it. So it's not like you need a lot of knowledge to do these things.
- 18:07
You can really start today even if you're not a coder and you can make cool things. Uh, yeah. So thank you. [audience applauding]
- 18:20
Yeah. Um, I'm gonna be around. I really like the concept of this being a conversation starter, so if anyone wants to talk to me about this, go ahead. We have one minute fifty-seven seconds for questions if anyone has them.
- 18:35
Uh, Prince.
- 18:36
Are you guys considering, uh, apps outside of, uh-
- 18:43
Like apps that run directly on, on, on the robot
- 18:45
Yeah, yeah. So a lot of the apps run directly on the robot, anything that doesn't use a GPU. Uh, of course, this one has a Raspberry Pi, so you need to work with that sort of hardware.
- 18:54
But y- you can also use your own laptop as the hardware. Um, and we are not constrained by that. Actually, we also are not constrained by languages. You can make your apps in Java or in Python or in, I don't know, HTML, I think.
- 19:10
Okay.
- 19:10
The sky's the limit.
- 19:11
Okay, awesome.
- 19:14
Yes.
- 19:16
Is there, like, a system of plugins for extensibility? Like, if you want to add, like, a servo, making it, like, move around the room, something like that. Do you have something that's somewhat supported, or do you need to hack around it?
- 19:27
So you sort of need to hack around it, but this is maybe our fourth open source robot, and we try to make them all stack with each other. So the SO10-100 and the SO10-101 are arms that you can, uh, plug together, and we have something called the Kiwi, which is like a base with three wheels that Reachy
- 19:47
Mini just comes into, so it can sort of move around. Uh, I still haven't seen it done, but we know it fits 'cause we designed it that way. [laughs]
- 19:56
Yeah.
- 19:58
Uh, I'm really interested in your Qwen, uh, TTS work.
- 20:01
Yeah.
- 20:01
Uh, what else, what other limitations did you find with the original implementation? Like, I know the streaming stuff, like the speed. Any other things that you see Qwen TTS is lacking?
- 20:11
Mm. So the implementation was quite complete, but just generally slow. It felt a little bit like it's a research implementation. If you use their API, their API is actually quite fast.
- 20:21
So I think that was a little bit their strategy of trying to get people towards the API because the open source model isn't, uh, up there. Um, and for me, I, I've been having a lot of issues come to the repo from people that are like, "Oh, this thing doesn't work," and then I need to implement it
- 20:39
just because the model can do so many different things, and I didn't think of all the corner cases. But no, the original implementation is quite complete.
- 20:47
Uh, I'm at time, so I thank you a lot. Uh, as I said, I'm gonna be around, so if you want to talk to me, please do. Uh, and I'm going to mute this, but please mute the laptop so I can take out the cable. [applause] [outro jingle]