jiangzhuo9357/omnivoice-0-6b-gguf

OmniVoice 0.6B: synthesize.cpp GGUF

GGUF conversions of the official OmniVoice checkpoint for synthesize.cpp.

Ported from k2-fsa/OmniVoice revision 468e927ba3716cd8dd86421148dfb3046e9f9d7b and validated on 2026-08-07 against the pinned upstream PyTorch implementation.

A self-contained OmniVoice inference package converted from the flagship k2-fsa/OmniVoice checkpoint. It is a non-autoregressive mask-predict diffusion language model: a bidirectional Qwen3-0.6B backbone refines a fixed-length canvas of 8 acoustic codebooks x 1025-entry vocabulary over 32 parallel denoising steps with classifier-free guidance, and the Higgs Audio V2 codec's DAC-style decoder turns the finished token grid into 24 kHz audio. Voice arrives through Reference Audio cloning, Description Text voice design, or an unnamed auto-voice default with no speaker conditioning, whose speaker is emergent rather than pinned by the seed alone. The package carries its own byte-level BPE text frontend, so raw UTF-8 text goes in and audio comes out.

One hazard to know before your first request, and nothing in the runtime warns about it. The canvas length is estimated from the text and then divided by the speaking rate. Below roughly 37-40 frames (about 1.5 seconds) there are too few positions to place the text, and the output degenerates into a near-DC sub-50 Hz rumble instead of speech. Two ordinary requests reach that: a very short text (--text "Hi." resolves to a 22-frame canvas), and a high --rate on a short one -- the declared speaking_rate_range goes to 2.0 and is accepted silently, but upstream's own demo UI caps speed at 1.5, and the pinned upstream model is already degenerate at rate 1.50 on a 32-character sentence. This is upstream's behavior, faithfully reproduced, not a defect in this port: on the affected validation case this package matches the reference implementation's waveform at Pearson r = 0.999670, the tightest agreement of any case measured, and the reference's own audio for it is equally unintelligible. Rate 2.0 on a long text is fine; the variable is the resolved canvas length, not the rate.

Downloads

Profile	Download	Size	Tensor storage	SHA-256	What to know before choosing it
F32	omnivoice-0-6b-F32.gguf	3190.0 MB (3,189,953,504 bytes)	798 F32	f6d504ffaddcbf32f80f1f6c847f075bbd5d2c7b50fe95a194ceb635772f9fa3	The reference conversion, bit-for-bit from the upstream checkpoint. Take it to reproduce a measurement; otherwise F16 matches it on every measured case and is 1.2 GB smaller. 3506 MiB accelerator resident, 2.6 s to load.
F16	omnivoice-0-6b-F16.gguf	1964.9 MB (1,964,929,440 bytes)	599 F32 + 199 F16	65c8cca59b350ccfdc6ad96c5a683b276f8c0fd5c8e96675d6dc110da3f52f70	Recommended default. Same voice as F32 on every measured case, all three voice modes, 38.4% smaller. 2353 MiB accelerator resident, 1.6 s to load. Not faster than F32.
Q8	omnivoice-0-6b-Q8.gguf	1390.7 MB (1,390,699,680 bytes)	599 F32 + 199 Q8_0	61aec0de7cfa9246487e309c43508de9c95cf52fd225ce3fada3b4bb4982374e	Smaller, and it renders a different voice than F32 does for most auto-voice cases and for Description Text -- 2.87 to 12.22 dB of long-term average spectrum distance, and a listener heard five of six sampled auto-voice pairs as different people while judging the quality of both indistinguishable. Not every case moves: the slowest-rate case measures 2.87 dB, below the same ~3 dB line this table uses to call F16 unchanged, so "different voice" is the common outcome rather than a universal one. Reference Audio cloning is not affected (2.02 / 2.70 dB): it reproduces F32's voice. Nothing is disabled, the API is identical, and a Description Text prompt is still honored -- "female, young adult, high pitch" gives 343.5 Hz against F32's 333.3 Hz, so you get a different voice within the description, not an ignored one. 56.4% smaller, 1811 MiB accelerator resident, 1.1-1.4 s to load. Not faster than F32.

All profiles use the same OmniVoice architecture and public synthesize.cpp API. The profile name describes a versioned storage policy, not the language or Execution Backend.

A profile name here says which half of the model it quantizes. This family has two: a Qwen3-based generator that paints the token canvas, and the Higgs Audio V2 codec that turns the canvas into audio. F16 and Q8 quantize the generator half and leave all 486 codec tensors bit-identical to F32, which is why all three reproduce the Reference Audio cloning path's token grid exactly — and why a quantized generator re-draws the other grid, the greedy decode canvas, on purpose. A profile that quantized the codec half instead would carry a _CODEC qualifier in its name; none is published, because every one produced so far breaks the cloning grid and fails that exact-token gate.

No profile here is faster than F32, and none claims to be. Measured against F32 on this project's CUDA reference host: Q8 -0.4%, F16 +2.7%; on CPU, Q8 +1.5% and F16 -1.2%, where a positive number is the slower one. The generator is compute-bound at roughly 205 multiply-accumulates per weight byte, so a narrower weight does not relieve the bottleneck. Two of those four figures land marginally under F32, by about one part in a hundred; neither is a reason to pick a profile. What a smaller profile buys is disk, accelerator memory and load time.

Sidecar files

These files are not GGUF tensor storage; they are declared resources that must travel with the package. Verify each digest before relying on its contents.

Role	File	Size	SHA-256
Boson Higgs Audio 2 Community License (codec weights; dual attribution required on redistribution)	LICENSE-higgs-audio-2.txt	9.0 KB (9,171 bytes)	ac933dc084d119bd20401956b90d11ae87c248b2da62622cd580d82cdf2fa049
Meta Llama 3 Community License (required by the Boson agreement above, which is derived from it and defines itself to include it)	LICENSE-meta-llama-3.txt	7.8 KB (7,801 bytes)	475211637354ce4c14b9c3dacccbefbaba735fe1b0db97d6c4fd11cd1356819f

Validation status

validation_level: port_validated

8 graph stages were replayed for 20 cases on DGX Spark CPU, NVIDIA GB10 CUDA 13.3. Duration structure was exact in every case. Both halves now run on CUDA, and only one of them did through Plan 4. The codec's decode graph (the RVQ dequantizer, the acoustic decoder, and the final projection -- 152 tensors) moved first: swept over all twenty golden cases, every one of its 8,440 nodes left the CPU while every one of the generator's 880,032 nodes stayed on it, held there by docs/backends.md's discrete-outputs rule. On 2026-08-08 the generator earned that rule's one narrow exception -- its per-step token choice picks the content of a canvas whose shape deterministic host arithmetic fixed before the first forward runs, and grid size was then measured to match the CPU oracle on all seventeen greedy cases -- so it received its own accelerator-resident weight twin and now moves too. 2,421.04 of the model's 3,042.2 MiB are mirrored, and the suite's longest case falls from 122.61 s to 4.84 s, 0.168x real time. (Corrected 2026-08-10: this read 5.481 s / 0.1906x, which was measured before the two host-side optimizations that followed it and so understated the shipped package. Re-measured on the published F32 GGUF, five runs, 4.844-4.866 s.) Committed token content consequently diverges from the CPU baseline on most cases: a different valid realization inside an unchanged canvas, audited by ear before the move was accepted, not a regression. The cloning path's own RVQ token grids stay byte-exact across the move.

Profile	CPU waveform cosine	DGX Spark CUDA waveform cosine	Worst LTAS distance from F32/CPU (17 greedy cases)
F32	0.99999986	0.99999635	— (reference)
F16	not measured	not measured	2.87 dB
Q8	not measured	not measured	12.22 dB

Waveform cosine is measured for completeness; it is not this family's real guarantee and is not enforced as a tolerance the way it is for Kokoro or VITS. What the Port Validation Suite reports instead is exact token identity, and it is two claims with two different scopes. First: the greedy decode loop's full 8 x T unmasking grid matches the oracle's byte-for-byte on 17 of 17 golden cases for F32 on CPU -- the reference configuration, and the only one that certifies it. A quantized generator re-draws that grid by design (Q8 commits a different token at 95.83% of positions), and so does the CUDA backend, whose TF32 arithmetic moves the per-step argmax; both change what the canvas contains and never how large it is, both were accepted by listening rather than by a token gate, and their flip counts are recorded as data, not gated. Second, and this is the claim that is gated unconditionally: the Reference Audio cloning path's own RVQ encode grid matches the oracle on both clone cases -- 8 codebooks x 351 frames, 2,808 of 2,808 tokens exact -- and it holds on every profile listed above, because F16 and Q8 quantize the generator half only and leave every one of the 486 codec.* tensors bit-identical to the F32 package. A profile that flips even one of those 2,808 tokens is not shipped, and that rule is what decides which profiles are listed above. Every codec-half profile produced so far fails it and none is listed: F16_CODEC mismatched 103 of 2,808 tokens (3.7%), and Q8_CODEC_MIXED mismatched 1,023 (36.4%) while it still block-quantized convolution kernels, 98 (3.49%) since the conv-exempt codec policy of 2026-08-09 stopped it doing so. None is a knife-edge margin call, so none ships and no perceptual claim substitutes for the gate. The two waveform-cosine columns are measured against the PyTorch oracle and are reported for F32 alone. They are not withheld for the quantized profiles; they do not apply to them. Because a quantized generator re-draws the token grid rather than perturbing F32's, its output is a different valid realization of the same request, and a waveform distance to the oracle would score that difference as error. The last column is the measurement that does separate them: long-term average spectrum distance from F32's own CPU render, the speaker-identity proxy this family uses, worst case over the seventeen greedy validation cases. Roughly 3 dB is the line a listener has consistently called "same person", calibrated on fourteen answers across two audits with no exceptions. Every deep generator probe gates on cosine rather than a sample-wise bound because pre-norm outlier channels reach a max-abs of 0.08 while cosine still holds at eight nines; a sample-wise threshold there would reject a provably correct port.

Quality evaluation has not been run. These results establish that the port, the unconditioned default-Voice request path, deterministic request path, and CPU/CUDA execution work. They do not claim perceptual equivalence, naturalness, intelligibility, or speaker similarity.

A listening audit found no obvious regression. One maintainer compared a small set against the reference and reported nothing audible. That is release evidence, not a measurement: no rated comparison, no panel, no score, and it does not change the validation level. It says a defect large enough to hear was not found in what was heard.

Voices and input

This package exposes no preset speaker catalog. Its package default has no fixed identity and no speaker conditioning: with no Voice Profile supplied, which speaker a request produces is emergent, and the synthesis seed alone does not pin it. Reproducing a speaker needs the same seed and the same Execution Backend, package, and decode settings; switching between the CPU and CUDA backends can produce a different speaker for an otherwise identical request. Measured on this package's own validation cases, one case in seventeen changes speaker between the CPU and CUDA backends, and a listener judged that pair to be two different speakers of equal audio quality. Callers who want a stable identity should instead build a Voice Profile from Reference Audio (voice cloning) or Description Text (voice design) and pass it to the request. It produces 24000 Hz mono F32 audio.

This package accepts raw UTF-8 text through the built-in synthesize.qwen_bpe frontend. The frontend tokenizes text directly: no grapheme-to-phoneme conversion happens or is needed, and the runtime does not silently invoke eSpeak or download a frontend.

Usage

Build synthesize.cpp and synthesize a deterministic request:

git clone https://github.com/handy-computer/synthesize.cpp.git
cd synthesize.cpp
cmake -S . -B build -DSYNTH_BUILD_CLI=ON
cmake --build build -j

hf download jiangzhuo9357/omnivoice-0-6b-gguf omnivoice-0-6b-F16.gguf LICENSE-higgs-audio-2.txt LICENSE-meta-llama-3.txt \
  --local-dir models/omnivoice-0-6b

build/bin/synthesize-cli \
  --model models/omnivoice-0-6b/omnivoice-0-6b-F16.gguf \
  --output output.wav \
  --text "The engine reads a whole paragraph without pausing for breath. It holds one voice from the first word to the last. Nothing in the timing depends on the machine that runs it. It sounds the same everywhere." \
  --language en \
  --seed 0

The same local GGUF can be loaded through the public C ABI and wrapped by C++, Rust, or Python. Model loading never contacts Hugging Face.

License and checkpoint provenance

This package carries three separate upstream grants, and only two of them touch anything shipped here. The generator (LM) weights are CC-BY-NC, with no version stated by upstream -- the upstream model card carries no license: frontmatter key at all; the only statement is this prose, quoted verbatim from the pinned upstream model page:

Our code is released under the Apache 2.0 License. The pre-trained model is licensed under the CC-BY-NC due to constraints from its training data (e.g., Emilia).

Upstream names no CC-BY-NC version anywhere in that statement or elsewhere on the card, so this card says exactly that rather than inventing one (never cc-by-nc-4.0: no version is asserted upstream and none is asserted here). These weights are not licensed for commercial use. The restriction traces to the training data the statement itself names -- Emilia, a large-scale multilingual speech corpus distributed under its own non-commercial terms -- so it is a property of what the generator was trained on, not a choice this project made.

The codec (Higgs Audio V2) weights carry a second, separate license: the Boson Higgs Audio 2 Community License. The weights repository bundles it as audio_tokenizer/LICENSE, and this package republishes it beside the weights as LICENSE-higgs-audio-2.txt (listed below).

Two licence files travel with these weights, not one. The Boson agreement defines its own name to include Meta's -- "Agreement" means the terms and conditions ... set forth herein and the Meta License Agreement -- and section 1.b.i(A) then requires "a copy of this Agreement and the ... Meta License's Llama 3 agreement" to accompany any redistribution of the Higgs Materials. So LICENSE-meta-llama-3.txt ships here too. Corrected 2026-08-10: this package originally shipped only the Boson text while this card said it did so "per that agreement's own redistribution terms", which was a claim of compliance it did not meet.

You must download both together with the GGUF -- the usage command below does; the runtime does not enforce the pairing, and nothing in the GGUF's metadata declares either file, so a download of the weights alone leaves you holding the Higgs Materials with none of the agreement that governs them. The Boson agreement is itself derived from the Meta Llama 3 Community License and requires dual attribution in any redistribution, which this card carries forward verbatim from the sidecar text:

"Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved." "Boson Higgs Audio 2 is licensed under the Boson Community License, Copyright © Boson AI USA, Inc. All Rights Reserved."

That agreement also requires a redistributor to display the following notice, which it specifies verbatim and which this card carries for that purpose:

"Built with Higgs Materials licensed from Boson AI USA, Inc., Copyright Boson AI USA, Inc., All Rights Reserved and Meta Llama 3 licensed under the Meta Llama 3 Community License, Copyright Meta Platforms, Inc., All Right Reserved"

Two further obligations, stated in the agreement's own terms rather than paraphrased. Corrected 2026-08-08: this card previously said "caps commercial use at 100,000 monthly active users", which was wrong twice over and wrong in the reader's favour -- the agreement says annual, and it does not cap anything.

Above 100,000 annual active users in the preceding calendar year, section 2 does not impose a ceiling: it withdraws authorisation. You "must request an expanded license from Boson AI, which Boson AI may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Boson AI otherwise expressly grants you such rights."

Section 1.b.i(v) forbids using the Higgs Materials "or any output or results of the Higgs Materials to improve any other large language model (excluding Boson Higgs Audio 2 or derivative works thereof)". Note the scope: other large language models, with Boson's own carved out -- not, as this card previously said, all models.

Both are independent of, and in addition to, the generator's own CC-BY-NC restriction above.

Apache-2.0 covers only the upstream GitHub source code (k2-fsa/OmniVoice) and nothing produced by this project: no weight file, converted artifact, or GGUF in this package may be described as apache-2.0.

Finally, the upstream card carries a use disclaimer this package repeats verbatim rather than paraphrases:

Users are strictly prohibited from using this model for unauthorized voice cloning, voice impersonation, fraud, scams, or any other illegal or unethical activities. All users shall ensure full compliance with applicable local laws, regulations, and ethical standards. The developers assume no liability for any misuse of this model and advocate for responsible AI development and use, encouraging the community to uphold safety and ethical principles in AI research and applications.

This is a Restricted Model Package (ADR 0018), not a Published Model Package, and must never be described as one: it carries the full upstream terms above rather than an unrestricted embeddable-in-other-programs grant. Both upstream licenses permit non-commercial redistribution with attribution, and upstream itself distributes the weights openly on Hugging Face -- the basis for republishing this converted GGUF under exactly those terms, not a broadening of them. Publication itself remains a separate act requiring its own per-act confirmation.

Only en, zh, and ja are declared as this family's Language Capability Catalog. The checkpoint claims 600+ languages through its training data and prompt format, and an untested language tag may well produce recognizable speech, but no language beyond these three has its own validation case, so none beyond these three is advertised.

---

Original upstream project card

Reproduced from the pinned upstream repository card for offline provenance. The upstream repository remains authoritative.

OmniVoice 🌍

         

OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech with superior inference speed, supporting voice cloning and voice design.

• Paper: OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
• Repository: GitHub
• Demo: Hugging Face Space
• Colab: Google Colab Notebook

Key Features

• 600+ Languages Supported: The broadest language coverage among zero-shot TTS models.
• Voice Cloning: State-of-the-art voice cloning quality from a short reference audio.
• Voice Design: Control voices via assigned speaker attributes (gender, age, pitch, dialect/accent, whisper, etc.).
• Fine-grained Control: Non-verbal symbols (e.g., [laughter]) and pronunciation correction via pinyin or phonemes.
• Fast Inference: RTF as low as 0.025 (40x faster than real-time).
• Diffusion Language Model-style Architecture: A clean, streamlined, and scalable design that delivers both quality and speed.

Usage

To get started, install the omnivoice library:

We recommend using a fresh virtual environment (e.g., conda, venv, etc.) to avoid conflicts.

Step 1: Install PyTorch

NVIDIA GPU

# Install pytorch with your CUDA version, e.g.
pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128

See PyTorch official site for other versions installation.

Apple Silicon

pip install torch==2.8.0 torchaudio==2.8.0

Step 2: Install OmniVoice

pip install omnivoice

Python API

You can use OmniVoice for zero-shot voice cloning as follows:

from omnivoice import OmniVoice
import soundfile as sf
import torch

# Load the model
model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)

# Generate audio
audio = model.generate(
    text="Hello, this is a test of zero-shot voice cloning.",
    ref_audio="ref.wav",
    ref_text="Transcription of the reference audio.",
) # audio is a list of `np.ndarray` with shape (T,) at 24 kHz.

sf.write("out.wav", audio[0], 24000)

For more generation modes (e.g., voice design), functions (e.g., non-verbal symbols, pronunciation correction) and comprehensive usage instructions, see our GitHub Repository.

Discussion & Communication

You can directly discuss on GitHub Issues.

You can also scan the QR code to join our wechat group or follow our wechat official account.

Wechat Group	Wechat Official Account

Citation

@article{zhu2026omnivoice,
      title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
      author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
      journal={arXiv preprint arXiv:2604.00688},
      year={2026}
}

License

Our code is released under the Apache 2.0 License. The pre-trained model is licensed under the CC-BY-NC due to constraints from its training data (e.g., Emilia).

Disclaimer

Users are strictly prohibited from using this model for unauthorized voice cloning, voice impersonation, fraud, scams, or any other illegal or unethical activities. All users shall ensure full compliance with applicable local laws, regulations, and ethical standards. The developers assume no liability for any misuse of this model and advocate for responsible AI development and use, encouraging the community to uphold safety and ethical principles in AI research and applications.