Qwen3.8-Flash-Next (non-abliterated) NVFP4 QAD + LMHead NVFP4 โ GDN-MSE drafter variant
The non-abliterated twin of claidler/Qwen3.8-Flash-Next-Ablit-NVFP4-QAD-LMHead-NVFP4-GDNMSE: same surgery, refusal-capable (non-abliterated) body. Built from the PLAIN-scale local-inference-lab/Qwen3.8-Flash-Next-NVFP4 checkpoint (already QAD-distilled from Qwen/Qwen3.8-Flash-Next, mixed-precision NVFP4/MXFP8, no CSF scales), plus:
- lm_head repacked to NVFP4 (W4A16, modelopt naming) โ identical recipe to the ablit twin
- The 29 dense BF16 MTP drafter tensors (shards 19-20) replaced by the retrained GDN-MSE drafter from ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE @ revision 16c9bd54 (hardlink farm)
Measured on 2x DGX Spark TP=2 with the ablit twin of the same drafter (vLLM 0.30, cjjlaidler/qwen38-flash-next-cluster-vllm v1, 10 Oct 2026):
- Spec-decode acceptance: 39.11% total vs 38.11% baseline (pos0 +3.4, pos1 +5.1, pos2 +3.2 pp)
- Decode throughput: 81.7 tok/s median vs 75.2 baseline (+8.6%), TTFT 117 ms vs 121 ms
- Gates: ab-gates trio 3/3, 120k needle 3/3, image smoke PASS
Serves on the same image/flags as the ablit twin (MTP speculative decoding, K=5, marlin MoE backend). Only ever change model: when swapping ablit/non-ablit; keep served-model-name: qwen-3.8-flash-next so clients never reconfigure.
Supersedes claidler/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-LMHead-NVFP4, which was accidentally built from the CSF sibling base and has no CSF load path in vLLM 0.30.
- Downloads last month
- 73
Model tree for claidler/Qwen3.8-Flash-Next-NVFP4-MXFP8-QAD-LMHead-NVFP4-GDNMSE
Base model
Qwen/Qwen3.8-Flash-Next