On an RTX 5090, it takes roughly 850 milliseconds from submitting a question to the avatar speaking. We judge that local voice avatars have crossed the threshold for “real-time conversation,” but not the threshold for “running on an ordinary computer.”

The stack integrates three models: SoulX-FlashHead generates the avatar’s moving image while speaking, OWSM v4 converts speech into text, and LFM3-Audio generates conversational replies and synthesizes speech. Speech recognition and generation run in parallel, but LFM3-Audio is invoked twice, and the interface itself adds further overhead.

What this is

audio.cpp 0.9.5 adds a web-based voice-chat application with an avatar. Rather than calling a ready-made model in the cloud, it runs the entire chain locally: listening to the user, understanding and answering, generating speech, and driving the avatar.

In one path, a single LFM3-Audio call returns speech directly. In the other, it first produces a text response, then uses its speech-synthesis function to generate another voice. SoulX-FlashHead generates the avatar in real time, so users see not only a voice assistant but also a face that changes as the answer is delivered.

Industry view

We believe the 850-millisecond latency shows that this combination of models is moving from a demonstration into a usable interactive product. Processing transcription and answering in parallel reduces serial waiting, while the web interface lets developers validate the full experience quickly.

The risks are equally clear. Real-time avatar generation still depends heavily on the GPU, and developers recommend an RTX 4090 or a more powerful configuration. Without cuDNN acceleration components, inference runs at half the speed and consumes twice the VRAM. Repeated LFM3-Audio calls and prototype-interface overhead also mean current performance figures are not the ceiling.

This stack is better suited to validating a product format than proving that consumer-scale adoption has arrived. Without lighter models and less expensive hardware, the privacy advantage of “local execution” may still be offset by the cost of the device.

Impact on regular people

For enterprise IT: Businesses may use local avatars for customer-service training, pre-sales presentations and internal assistants, reducing the need to upload sensitive voice data to the cloud. Before deployment, however, they must assess GPU procurement, model licensing and maintenance costs together.

For individual professionals: The stack makes it easier to build meeting role-play exercises, language practice tools and voice assistants with a human-like avatar. Yet an ordinary office PC may not run it smoothly for now, so any time savings may come with additional hardware spending.

For the consumer market: Once similar features appear in mature applications, users will become accustomed to “speak while watching” interactions. For now, however, this is more of an early-stage product for developers to try than a standard feature of mass-market consumer electronics.