Koopah/Qwen3.6-35B-A3B-NVFP4

Qwen3.6-35B-A3B — NVFP4 (validated frozen snapshot)

A frozen, validated snapshot of the original Unsloth NVFP4 export of Qwen/Qwen3.6-35B-A3B: single-file nvfp4-pack-quantized (compressed-tensors), calibrated on UltraChat (~2M tokens), exported with llm-compressor 0.15.1. Identical to unsloth/Qwen3.6-35B-A3B-NVFP4 at revision 612d523c58 (pre-2026-07-10).

Why this repo exists

On 2026-07-10 the upstream repo silently replaced its weights with a different quantization: a mixed-precision re-export (FP8-dynamic layer groups, a static FP8 KV-cache scheme, llm-compressor 0.17.2-alpha). In our testing that re-export produces degenerate looping output — verbatim phrase repetition beginning within a few hundred tokens — reproduced on two SM120 GPUs (RTX 5090, RTX PRO 6000 Blackwell), with and without fp8 KV cache, on sglang 0.5.17:

target artifact	output
this snapshot (pre-2026-07-10 export)	clean, coherent, correct
upstream main (2026-07-10 re-export)	verbatim loops (top 8-gram repeated 200–650×)

Every published benchmark and quality validation of the DSPARK draft models — v2 · v1 · GGUF — was run against this artifact. Use it as the serving target for those drafters.

Serving

sglang (≥ 0.5.17, fastest measured: ~619 tok/s single-user on SM120 with the v2 drafter):

python -m sglang.launch_server \
  --model-path Koopah/Qwen3.6-35B-A3B-NVFP4 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path Koopah/Qwen3.6-35B-A3B-NVFP4-DSPARK-v2 \
  --reasoning-parser qwen3 --trust-remote-code

vLLM (0.26.x; ~580 tok/s single-user):

vllm serve Koopah/Qwen3.6-35B-A3B-NVFP4 --trust-remote-code \
  --mamba-ssm-cache-dtype bfloat16 \
  --speculative-config '{"method": "dspark", "model": "Koopah/Qwen3.6-35B-A3B-NVFP4-DSPARK-v2", "num_speculative_tokens": 8}'

llama.cpp: use the GGUF conversion of this same artifact in the GGUF repo.

Validation carried by this snapshot

• Temperature-0 outputs coherent and task-correct (math / code / factual probes), speculative and plain decoding.
• DSPARK v2 acceptance profile matches the drafter's certified rollout profile; single-user 619 tok/s (sglang 0.5.17) / 580 tok/s (vLLM 0.26) / ~506 tok/s (llama.cpp performance branch) on RTX PRO 6000 Blackwell.
• Credit for the quantization itself belongs to Unsloth; this repo only freezes the working revision with its provenance recorded.