hoidhxd

/

NVIDIA-NemotronLabs-VoiceChat-11B-GGUF

Like 8

That is expected — --serve is idle until you talk to it. It is not a network server and it does not open a microphone; it is a subprocess protocol. After the three ggufs load it writes exactly one line to stdout:

{"kind":"ready","tts":true,"function_head":true,"frame_cap":2250,"frame_rate":12.5}

and then blocks on stdin, waiting for one JSON object per line. Nothing happens until you send one. Simplest possible check — one turn, then EOF closes it:

echo '{"cmd":"turn","audio":"question.wav","out":"answer.wav"}' | VC_NO_BARGE=1 VC_FORCE_BOS=1 llama-voicechat -m llamacpp/nemotron_voicechat_11b-stt-llm-Q4_0.gguf --mmproj llamacpp/mmproj-voicechat-perception-Q4_0.gguf --tts llamacpp/voicechat-tts-Q4_0.gguf --serve

Or leave it running and paste lines into the terminal as you go. Commands in:

{"cmd":"system","text":"You are ..."} before the first turn only {"cmd":"turn","audio":"in.wav","out":"out.wav"} {"cmd":"tool_response","text":"{...}"} only while a call is pending {"cmd":"reset"} {"cmd":"ping"} {"cmd":"quit"}

Events out, one per line: ready, turn_start, assistant_text_delta, function_delta, tool_call, audio, turn_end, progress, bye, error. In this mode stdout is JSON only — every log line goes to stderr, so if you piped both together that is worth separating.

VC_NO_BARGE=1 VC_FORCE_BOS=1 really do matter here. Without them the model opens its own turn about a second into your clip, answers the first second of the question, then repeats itself — and on --serve that damage persists, because the state it leaves behind is the state the next turn starts from.

Now, the duplex part. The model genuinely is duplex: one 12.5 Hz timeline, and the perception embedding is summed into the previous frame's token embedding rather than appended as extra positions. That is also why --serve is worth having — the conversation is that timeline continuing, so chat history costs nothing: no prompt is rebuilt, no messages are replayed, the next turn's audio is just appended. A system prompt runs once at session open, not per turn.

But what --serve gives you is multi-turn push-to-talk, not live full duplex. A turn is still a path to a finished wav. There is no chunked audio input yet, so you cannot interrupt it mid-answer. Nothing in the model blocks that — it needs a driver that pushes 80 ms chunks and decodes each one in under 80 ms. On an RTX 5060 Ti with -ngl 99 a frame costs ~0.05 s, so there is about 1.6x of headroom; on CPU it is ~0.28 s per frame, ~3.5x slower than real time, and a full turn measured at 843 frames takes ~205 s versus 36-47 s on that GPU.

Which brings me back to: which build and which GPU are you on? If you are on the CPU zip, or on the CUDA one without -ngl 99 --device CUDA0, the first turn after ready will also look like a hang for several minutes. VC_DUMP=1 puts one line per frame on stderr if you want to watch it move, and progress events do the same in JSON.

I have a small Python web front-end here that drives one --serve process per session (browser mic in, text streaming back, wav played on arrival). It is not in the repo yet — say the word and I will push it as an example driver.

oh thanks for taking your time to explain that makes a whole lot of sense, and yes please do so if you can.
we might save a tree if not everyone has to give the same instruction to their favorite LLM to build it.

Thanks again!

Pushed: https://github.com/sansamour/llama-voicechat-demo

It is the driver behind --serve — one FastAPI file, one static page, no build step. Clone it, drop the binary in bin/ and the four ggufs in models/, pip install -r requirements.txt, then ./run.sh (or .\run.ps1) and open http://localhost:31333/. No certificate needed: localhost is a secure context, so the microphone works over plain http. The README has separate step-by-step installs for Windows and Ubuntu, since there is no prebuilt Linux binary and you have to build the fork there.

You get push-to-talk multi-turn conversation, the text channel streaming in as the model writes it, the answer wav played back per turn, an editable system prompt, and two example tools wired through the function channel end to end. The barge-in guard is on by default.

It will not do the live full-duplex thing we talked about — it uploads a finished recording per turn. That is still the missing piece, and it is a driver problem, not a model one.

Thanks for asking for it; you were right that it should not be something everyone re-derives.