Reachy Mini’s conversation stack could move onto local hardware

The local Reachy Mini guide described a speech-to-speech stack composed of voice detection, speech recognition, a language model, and speech synthesis. It exposed a realtime-compatible interface and showed how the robot could connect to a locally running backend.

A cascade has a useful property: its components can be replaced independently. That allows experiments with different voices, recognition systems, and models without rebuilding the entire interaction. It also means total conversational latency is the sum of several stages rather than one isolated inference time.

Use the source’s setup as a baseline and measure each stage before swapping components. Confirm that the complete configured stack runs locally; changing one backend can alter where audio or text is processed. Test the robot’s interaction, not just the individual models.

Local conversation is a pipeline, not one model

A spoken interaction combines several tasks: capturing audio, deciding when someone has finished speaking, transcribing the request, generating a response and producing speech. Moving these stages onto local hardware changes where the work happens, but it does not remove the need to coordinate them.

A robot can have a capable language model and still feel difficult to talk to if it interrupts too early or waits too long after a question. The quality of the interaction depends on timing and turn-taking as much as on the wording of the answer.

Measure the pauses people hear

Break the total response delay into stages. Record the time between the user finishing a sentence and the system detecting that boundary, then the time needed for transcription, response generation and speech startup. This helps identify whether the bottleneck is the model or the surrounding audio workflow.

Test short questions, long requests and sentences with natural pauses. A person thinking halfway through a sentence should not automatically be treated as having finished. Background noise and the robot’s own speaker output are also important cases, because a quiet-room demonstration can hide echo and interruption problems.

Fit the whole workload on the device

Memory estimates should include every active component, not only the language-model weights. Audio buffers, speech models, runtime overhead and concurrent applications all compete for the same resources. A configuration that loads successfully may still become unstable during a long conversation.

Try a sustained session rather than a single request. Observe whether latency increases, whether memory is released between turns and whether the device becomes thermally constrained. Reducing one model’s size can sometimes improve the experience more than chasing a marginal improvement in answer quality.

Make local privacy concrete

Running a model locally does not automatically mean that every part of an application stays local. Check where models are downloaded from, whether telemetry is enabled and whether any fallback path sends requests to a remote service. Describe the actual data flow instead of relying on the word “local” as a complete privacy explanation.

If recordings or transcripts are retained, give them an explicit purpose and retention policy. A system can process speech locally while still accumulating more personal information than its user expects.

Preserve control during failures

A failed transcription or an unavailable model should lead to a clear recoverable state. Keep robot motion limits and stop controls independent of the conversational model. The goal is a system that can continue behaving safely even when it cannot produce a useful answer.

A good local conversation setup is one that remains understandable under ordinary interruptions, noise and resource limits—not only one that answers impressively when every stage works perfectly.