gpillon/Qwen3-ASR-0.6B-GGUF-ignis

Qwen3-ASR-0.6B GGUF for ignis

Run it with ignis

These files are the small speech model of ignis, a local, OpenAI-compatible inference server. ignis runs this model on the CPU, beside the language model on the GPU, with our own Rust executor: it takes no VRAM, and your audio never leaves the machine. You can dictate prompts in the ignis Playground, or point any OpenAI client at /v1/audio/transcriptions. The default is the larger Qwen3-ASR-1.7B; this one is faster on small CPUs.

ignis-server model download qwen3-asr-0.6b

Then pick it: add --speech-model qwen3-asr-0.6b to your server command, or set model: qwen3-asr-0.6b under speech: in the config file.

With the server running, transcribe a clip; an OpenAI client that sends model: whisper-1 works too:

curl http://127.0.0.1:8000/v1/audio/transcriptions -F file=@dictation.wav -F model=whisper-1

Speech support shipped in ignis v0.9.0: take the latest release (tracking issue #350). Documentation: gpillon.github.io/ignis.

Files

File	What	Used by ignis as
Qwen3-ASR-0.6B-Q8_0.gguf	decoder, Q8_0	qwen3-asr-0.6b
Qwen3-ASR-0.6B-Q4_K_M.gguf	decoder, Q4_K_M	not in the catalog
Qwen3-ASR-0.6B-bf16.gguf	decoder, bf16	reference
mmproj-Qwen3-ASR-0.6B-Q8_0.gguf	audio encoder, Q8_0	the encoder of qwen3-asr-0.6b
mmproj-Qwen3-ASR-0.6B-bf16.gguf	audio encoder, bf16	reference

SHA-256 of every file: SHA256SUMS. The ignis catalog pins this repo's revision and these hashes.

Measured with ignis

ignis's CPU executor, warm, whole clip, the official prompt, qwen3-asr-0.6b (Q8_0):

	i9-10900K, 9 threads	4-core container, 3 threads
time for a 60.8 s clip (times real time)	9.25 s (6.6x)	16.31 s (3.7x)

Word error rate, pooled:

	40 FLEURS test clips (Italian, English)	11 Italian dictations
qwen3-asr-0.6b (Q8_0)	5.91%	7.74%
Transformers, FP32 (the official checkpoint)	6.64%	8.23%

How they were made

Converted by us from the official Qwen/Qwen3-ASR-0.6B at revision 5eb144179a02acc5e5ba31e748d22b0cf3e303b0, with llama.cpp b11840's convert_hf_to_gguf.py (decoder and --mmproj encoder, bf16 and Q8_0) and llama-quantize (Q4_K_M from bf16). Our Q8_0 files were checked against ggml-org's conversion.

License and attribution

Qwen3-ASR is by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (LICENSE). These files are a modified form of it: converted to GGUF and quantized, by gpillon for ignis.