Qwen3.8-27B-DFlash2-NVFP4

Native NVFP4 (W4A16, group-16) requantization of the Qwen3.8-27B DFlash2 block drafter (5 Qwen3-style layers, 1.92B params). Built for the all-NVFP4 vLLM stack: NVFP4 target weights + NVFP4 draft weights + NVFP4 KV cache on SM120 (RTX 5090), verified at 262,144-token context with DFlash2 K=3.

Format

compressed-tensors nvfp4-pack-quantized:

• weights: float4, strategy: tensor_group, group_size: 16, symmetric, dynamic: false, observer: memoryless_minmax, scale_dtype: torch.float8_e4m3fn
• packed tensors: weight_packed (uint8, [out, in/2]), weight_scale (fp8_e4m3fn, [out, in/16]), weight_global_scale (float32, divisor convention)
• quantized: self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj, fc (36 matrices total)
• kept BF16: kernel_projection (grouped convs), candidate_selector, hidden_projection, norms

Quantization

Round-to-nearest weight-only quantization (no calibration) from the BF16 checkpoint, using the release image's compressed_tensors 0.17.0 (compressors/nvfp4/helpers.pack_fp4_to_uint8). Reproducible with scripts/build_nvfp4_draft.py (source: vLLM-sm12x DFlash2 release repo).

Checksums

• model.safetensors (1361734592 bytes): db19f8496af66a5b8cc26c682a7c80a754af0fa62929e6b2f14aa14fe97d7126
• config.json: 106a0e81a35acfbdfcd8d40786216f260a51e12e690ea820e9c05bc5d7ab7045

Usage (vLLM, SM120, all-NVFP4)

vllm serve <Qwen3.8-27B NVFP4 target>   --speculative-config '{"method":"dflash","model":"<this repo>",'
  '  "num_speculative_tokens":3,"kv_cache_dtype":"nvfp4"}'   --kv-cache-dtype nvfp4   --max-model-len 262144 --max-num-seqs 8   --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE",'
  '  "cudagraph_capture_sizes":[4,8,12,16,20,24,28,32]}'

Measured on the all-NVFP4 262k profile: 341,765-token NVFP4 KV pool (1.30x of 262k), greedy determinism byte-identical, acceptance mean 2.35 / 44.9% avg, and a 254,658-token request completing end-to-end.

The drafter shares the target's embeddings and lm_head (it ships neither), matching the upstream DFlash2 design. Apache-2.0, like the original drafter by Inco.