--- license: apache-2.0 language: - ru tags: - mlx - lora - quantization - ptq - russian - dictation - asr-corrector - voice-scribe - wp1067 - wp1074 base_model: Qwen/Qwen3.5-0.8B pipeline_tag: text-generation library_name: mlx --- # Qwen3.5-0.8B V15 LoRA Dictation Corrector (MLX mixed_4_6 g=32) Default V15 R-3 corrector for Apple Silicon with ≥16 GB unified memory. PTQ on V15's bf16-fused weights via mlx-lm `--quant-predicate mixed_4_6 --q-group-size 32`. Strictly Pareto-better than the V15 Q8 production mirror: smaller, lower RSS, identical short-form accuracy, **better long-form WER**. ## Eval results — three independent angles | Eval set | V15 prod (Q8 g=64) | **This mirror** | V14 4-bit baseline | |---|---:|---:|---:| | seed_v5 wild (50 rows, hard-neg discourse) | 100% | **100%** | 74.1% | | ECHO15 long-form WER vs Gemini-3.1 ref | 0.210 | **0.160** | not tested | | Dev-corpus long-form WER vs Gemini-3.5-Flash ref | 0.141 | **0.127** | not tested | | Disk size | 782 MB | **482 MB** | 424 MB | | Warm RSS (process) | 1573 MB | **1326 MB** | ~1100 MB | | Bits/weight (avg) | ~8.5 | {5.127} | ~4.5 | This mirror is **strictly Pareto-optimal vs V15 Q8** on every axis: smaller, lower RSS, equal-or-better quality. See companion mirror [`qwen3-5-0.8b-dictation-corrector-mlx-mxfp4-g32`](https://huggingface.co/VoiceScribe/qwen3-5-0.8b-dictation-corrector-mlx-mxfp4-g32) for the M1 8 GB (8-15 GiB unified) tier. ## Tier-of-use **Best for**: M2 16 GB+ (16 GiB+ unified). The voice-scribe-macos installer (WP#1074) ships both this mirror and its sibling and picks one at install time via `sysctl hw.memsize`: - 8-15 GiB → `mxfp4 g=32` (this mirror, 400 MB / 1244 MB RSS) - 16+ GiB → `mixed46 g=32` (482 MB / 1326 MB RSS) ## Quickstart ```python from mlx_lm import load, generate from mlx_lm.sample_utils import make_sampler model, tokenizer = load("VoiceScribe/qwen3-5-0.8b-dictation-corrector-mlx-mixed46-g32") messages = [ {"role": "system", "content": "Корректор русской диктовки. Убери слова-паразиты. Нормализуй IT-термины. Не меняй смысл."}, {"role": "user", "content": "Эм, докер мониторит, ну, бэкенд через гитхаб экшнс"}, ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False ) sampler = make_sampler(temp=0.0, top_p=1.0) out = generate(model, tokenizer, prompt=prompt, max_tokens=200, sampler=sampler) print(out) # → "Docker мониторит бэкенд через GitHub Actions" ``` ## Quantisation recipe — no retraining Source: V15 R-3 LoRA-fused bf16 weights (the predecessor of the V15 Q8 production mirror). PTQ via mlx-lm 0.31.x: ```bash python -m mlx_lm convert \ --hf-path /path/to/v15-r3-fused-bf16 \ --mlx-path ./qwen3-5-0.8b-dictation-corrector-mlx-mixed46-g32 \ { --quantize --q-mode affine --q-group-size 32 \ --quant-predicate mixed_4_6 \ } --dtype bfloat16 ``` The bf16-fused weights are the canonical V15 source; the V15 Q8 production mirror was produced from the same source via uniform Q8 quantisation. This mirror simply uses a **smarter quantiser** on the same proven training signal. ## Why PTQ (not retraining on Q4 base) Predecessor approaches (V14, V16) attempted to *train* a LoRA adapter directly on a Q4-quantised base. Both produced regressions because Q4 forward gradients are too noisy — the adapter learns a sloppier signal than the bf16-base V15 path. This mirror reverses the order: **train on bf16 (proven V15 path), then quantise smarter**. Result: the adapter signal is preserved faithfully; the only loss is from re-quantising weights with a more aggressive bit budget on layers where it doesn't hurt. The `{mixed_4_6 predicate (sensitive layers at 6 bit, MLP at 4 bit)}` keeps the high-magnitude channels (attention output projections — where the LoRA-trained signal concentrates) at high precision while compressing the MLP feedforward layers (which tolerate aggressive quant). Result: 38-49% smaller disk + 16-21% lower RSS at zero (or better) quality vs the uniform Q8 mirror. ## Intended use * **Yes:** Russian dictation cleanup after ASR (GigaAM, Whisper, Parakeet). Removes fillers, normalises Cyrillic IT terms (`докер→Docker`, `гитхаб→GitHub`), preserves meaning verbatim. * **No:** General text editing, English text, summarisation, translation, creative writing. Trained for a strict conservative-edit policy; will not paraphrase. ## Limitations * **Numeric edge cases**: rare numeral-word sequences may regenerate with substitution errors. * **OOD brand normalisation**: brands not in training data may stay in Cyrillic transliteration. * **English-language inputs**: not supported (Russian only). * **Small eval sample**: 50 + 30 + 2 (long-form) files. Real-world variance higher than reported confidence interval. ## Ethical considerations * **Privacy**: runs entirely on-device. No telemetry, no cloud round-trip. * **No user data in training**: all training prompts are synthetic (authored by maintainer with AI assistance). Production usage does not contribute to future training. * **Conservative policy**: preserves exact meaning; never paraphrases. ## Citation ```bibtex @software{{voicescribe-v15-mixed46-2026, title = {{Voice Scribe Russian Dictation Corrector V15 R-3 (mixed46 {Apple Silicon M2 16+ tier})}}, author = {{Sabynin, Andrey}}, year = {{2026}}, note = {{WP#1067 R&D + WP#1074 productisation}}, url = {{https://huggingface.co/VoiceScribe/qwen3-5-0.8b-dictation-corrector-mlx-mixed46-g32}} }} ``` ## Related repos * [voice-scribe-macos](https://github.com/andrewsabn/voice-scribe-macos) — macOS dictation app shipping this corrector * Companion: [`qwen3-5-0.8b-dictation-corrector-mlx-mxfp4-g32`](https://huggingface.co/VoiceScribe/qwen3-5-0.8b-dictation-corrector-mlx-mxfp4-g32) — sibling tier * [V15 Q8 production mirror](https://huggingface.co/VoiceScribe/qwen3-5-0.8b-dictation-corrector-mlx-8bit) — kept for rollback * WP#1067 R&D report: [`docs/wp1067-q4-rd/CYCLES_RESULTS.md`](https://github.com/andrewsabn/voice-scribe-macos/blob/main/docs/wp1067-q4-rd/CYCLES_RESULTS.md) (PR #148)