jiangzhuo9357/vits-ljspeech-gguf

VITS LJSpeech: synthesize.cpp GGUF

GGUF conversions of the official VITS LJSpeech checkpoint for synthesize.cpp.

Ported from jaywalnut310/vits revision 2e561ba58618d021b5b8323d3765880f7e0ecfdb and validated on 2026-07-27 against the pinned upstream PyTorch implementation.

A self-contained single-speaker VITS inference package converted from the official LJSpeech checkpoint. It supports CPU and CUDA execution through the same public library interface and uses the checkpoint's fixed package-default Voice without requiring Voice selection.

Downloads

Profile	Download	Size	Tensor storage	SHA-256
F32	vits-ljspeech-F32.gguf	113.2 MB (113,245,056 bytes)	460 F32	bd17e44c7c2d761d33c1527059bd3921f9d3fa9d46bd73b3e020d746a8b7db7b
F16	vits-ljspeech-F16.gguf	75.8 MB (75,778,368 bytes)	346 F32 + 114 F16	8683788b4dec7b81bf86f1cca024ed790724e491e2df40f77f1fc0bc614f860f
Q8_MIXED	vits-ljspeech-Q8_MIXED.gguf	58.2 MB (58,215,040 bytes)	346 F32 + 114 Q8_0	f751607f7514fa2aa1dca3ba46738b081c6b6c6b9579bfd22b1b80016551a3eb

All profiles use the same VITS architecture and public synthesize.cpp API. The profile name describes a versioned storage policy, not the language or Execution Backend.

Validation status

validation_level: port_validated

7 graph stages were replayed for 12 cases on DGX Spark CPU, NVIDIA GB10 CUDA 13.3. Duration structure was exact in every case. Two of this family's stages run on CPU on every Execution Backend, and the other five run on CUDA when CUDA is the selected Execution Backend. Model::Impl::compute_duration_stage uses a CPU-only scheduler over CPU-resident mirrors of the text encoder, the duration predictor and the Voice tensors, because the duration path ceils its logits into an integer frame count and every later stage's shape follows from that integer. Removing the strict-FP32 gate made it backend-dependent: on the 315-token ljs-long case CUDA ceiled one token differently and moved the frame count by 256 samples. The text encoder is held too, not just the rounding stage, because the duration stage re-runs it and feeds its output straight into the predictor. 27.5 of the model's 113.2 MB are consequently resident in both buffers, and synthesis wall time on ljs-long rises from 1.09 s to 1.39 s, still 7.6 times faster than real time. In exchange the frame counts match CPU exactly and both variants report structural_exact on all twelve CUDA cases. This is a deliberate placement, not a fallback: no node falls back to CPU inside a CUDA graph.

Profile	CPU max PCM drift	DGX Spark CUDA max PCM drift
F32	0.00029484	0.01902072
F16	0.01533963	0.02761611
Q8_MIXED	0.36558404	0.37650996

Drift is the largest absolute difference between one PCM sample and the reference's.

Quality evaluation has not been run. These results establish that the port, the fixed-Voice request path, deterministic request path, and CPU/CUDA execution work. They do not claim perceptual equivalence, naturalness, intelligibility, or speaker similarity.

Voices and input

This package has one fixed package-default Voice and does not expose a preset speaker catalog. It produces 22050 Hz mono F32 audio. Callers omit Voice selection; the package always uses its trained Voice.

This package accepts UTF-8 phoneme strings through the built-in synthesize.symbol_map frontend, and also accepts exact token IDs. The frontend validates UTF-8, maps each Unicode scalar through the symbol table embedded in the GGUF, and applies the model's blank-insertion rule.

The built-in frontend does not perform grapheme-to-phoneme conversion or text normalization. Callers starting from raw text must currently run a compatible G2P frontend externally. The runtime does not silently invoke eSpeak or download a frontend.

Usage

Build synthesize.cpp and synthesize a deterministic request:

git clone https://github.com/handy-computer/synthesize.cpp.git
cd synthesize.cpp
cmake -S . -B build -DSYNTH_BUILD_CLI=ON
cmake --build build -j

hf download jiangzhuo9357/vits-ljspeech-gguf vits-ljspeech-F16.gguf \
  --local-dir models/vits-ljspeech

build/bin/synthesize-cli \
  --model models/vits-ljspeech/vits-ljspeech-F16.gguf \
  --output output.wav \
  --phonemes "ˈeɪ." \
  --language en \
  --seed 0

The same local GGUF can be loaded through the public C ABI and wrapped by C++, Rust, or Python. Model loading never contacts Hugging Face.

License and checkpoint provenance

The pinned VITS source repository is MIT licensed: license at the ported revision. The official checkpoint is published by the VITS author in the repository's linked Google Drive folder, but does not carry a separate checkpoint-license statement.

The synthesize.cpp maintainer has chosen to publish these converted artifacts on the explicit project-policy assumption that the absence of separate checkpoint terms imposes no additional redistribution restriction. Users who require a different legal interpretation should review the upstream source and checkpoint provenance before redistribution.

---

Original upstream project card

Reproduced from the pinned upstream repository card for offline provenance. The upstream repository remains authoritative.

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

Jaehyeon Kim, Jungil Kong, and Juhee Son

In our recent paper, we propose VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech.

Several recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.

Visit our demo for audio samples.

We also provide the pretrained models.

** Update note: Thanks to Rishikesh (ऋषिकेश), our interactive TTS demo is now available on Colab Notebook.

VITS at training	VITS at inference

Pre-requisites

0. Python >= 3.6
1. Clone this repository
2. Install python requirements. Please refer requirements.txt
   1. You may need to install espeak first: apt-get install espeak
3. Download datasets
   1. Download and extract the LJ Speech dataset, then rename or create a link to the dataset folder: ln -s /path/to/LJSpeech-1.1/wavs DUMMY1
   2. For mult-speaker setting, download and extract the VCTK dataset, and downsample wav files to 22050 Hz. Then rename or create a link to the dataset folder: ln -s /path/to/VCTK-Corpus/downsampled_wavs DUMMY2
4. Build Monotonic Alignment Search and run preprocessing if you use your own datasets.

# Cython-version Monotonoic Alignment Search
cd monotonic_align
python setup.py build_ext --inplace

# Preprocessing (g2p) for your own datasets. Preprocessed phonemes for LJ Speech and VCTK have been already provided.
# python preprocess.py --text_index 1 --filelists filelists/ljs_audio_text_train_filelist.txt filelists/ljs_audio_text_val_filelist.txt filelists/ljs_audio_text_test_filelist.txt 
# python preprocess.py --text_index 2 --filelists filelists/vctk_audio_sid_text_train_filelist.txt filelists/vctk_audio_sid_text_val_filelist.txt filelists/vctk_audio_sid_text_test_filelist.txt

Training Exmaple

# LJ Speech
python train.py -c configs/ljs_base.json -m ljs_base

# VCTK
python train_ms.py -c configs/vctk_base.json -m vctk_base

Inference Example

See inference.ipynb