Qwen3.8-Flash-Next (non-abliterated) NVFP4 QAD + LMHead NVFP4 โ€” GDN-MSE drafter variant

The non-abliterated twin of claidler/Qwen3.8-Flash-Next-Ablit-NVFP4-QAD-LMHead-NVFP4-GDNMSE: same surgery, refusal-capable (non-abliterated) body. Built from the PLAIN-scale local-inference-lab/Qwen3.8-Flash-Next-NVFP4 checkpoint (already QAD-distilled from Qwen/Qwen3.8-Flash-Next, mixed-precision NVFP4/MXFP8, no CSF scales), plus:

  1. lm_head repacked to NVFP4 (W4A16, modelopt naming) โ€” identical recipe to the ablit twin
  2. The 29 dense BF16 MTP drafter tensors (shards 19-20) replaced by the retrained GDN-MSE drafter from ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE @ revision 16c9bd54 (hardlink farm)

Measured on 2x DGX Spark TP=2 with the ablit twin of the same drafter (vLLM 0.30, cjjlaidler/qwen38-flash-next-cluster-vllm v1, 10 Oct 2026):

  • Spec-decode acceptance: 39.11% total vs 38.11% baseline (pos0 +3.4, pos1 +5.1, pos2 +3.2 pp)
  • Decode throughput: 81.7 tok/s median vs 75.2 baseline (+8.6%), TTFT 117 ms vs 121 ms
  • Gates: ab-gates trio 3/3, 120k needle 3/3, image smoke PASS

Serves on the same image/flags as the ablit twin (MTP speculative decoding, K=5, marlin MoE backend). Only ever change model: when swapping ablit/non-ablit; keep served-model-name: qwen-3.8-flash-next so clients never reconfigure.

Supersedes claidler/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-LMHead-NVFP4, which was accidentally built from the CSF sibling base and has no CSF load path in vLLM 0.30.

Downloads last month
73
Safetensors
Model size
92B params
Tensor type
I64
ยท
BF16
ยท
F32
ยท
F8_E4M3
ยท
U8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for claidler/Qwen3.8-Flash-Next-NVFP4-MXFP8-QAD-LMHead-NVFP4-GDNMSE

Quantized
(5)
this model