Fix: AWQ MTP 0% acceptance by adding mtp.* to modules_to_not_convert

Previous revisions advertised `mtp_num_hidden_layers: 1` and shipped BF16 MTP
tensors in `model_extra_tensors.safetensors`, but the AWQ skip list did not
name them. vLLM's `Qwen3_5MultiTokenPredictor` therefore constructed its `fc`
and decoder linears as AWQ-packed (qweight/qzeros/scales), so the BF16 `.weight`
tensors were silently dropped by AutoWeightsLoader and the MTP head ran on
uninitialised parameters. Symptom: drafts generated but `Avg Draft acceptance
rate: 0.0%`.

This commit adds `mtp`, `mtp.fc`, and `mtp.layers.0.{self_attn,mlp}.<proj>`
to `modules_to_not_convert` in both `config.json` and
`quantization_config.json`. The vLLM-internal `mtp.* -> model.*` checkpoint
rename happens only inside `Qwen3_5MTP.load_weights`; the skip list is matched
against constructor prefixes (`mtp.*`), so `model.*` aliases are not added
(they would substring-match real backbone layers).

Measured on a single NVIDIA B300 SXM6, vLLM 0.21.0, this checkpoint, GPU 2,
8 mixed reasoning/code prompts, greedy decoding, max_tokens=512:
baseline : 115 tok/s
MTP num_speculative_tokens=3 : 251 tok/s (2.19x)
Avg Draft acceptance rate : ~77%
Per-position acceptance : 0.92 / 0.79 / 0.65
Mean acceptance length : ~3.3

README throughput table updated to mirror the NVFP4 sibling repo (baseline,
MTP n=3, batched columns plus a speedup column).

Reported by user BuiDoan on the HF discussion of this repo; thanks for the
Drafted/Accepted counts that made the diagnosis unambiguous.
Only the three config/README files change; safetensors are byte-identical.

README.md

@@ -58,10 +58,13 @@ Quantization preserves accuracy within standard error of the BF16 source on ever
 
 Measured on a single NVIDIA B300 SXM6 with vLLM 0.21.0 and `torch.compile` enabled:
 
-| Setup | Throughput |
-|---|---:|
-| Batch = 1, no MTP | 107 tok/s |
-| Batch = 8 continuous batching, no MTP | 880 tok/s |
+| Setup | Throughput | Speedup |
+|---|---:|---:|
+| Batch = 1, no MTP | 115 tok/s | 1.00× |
+| Batch = 1, MTP `num_speculative_tokens = 3` | **251 tok/s** | **2.19×** |
+| Batch = 8 continuous batching, no MTP | 880 tok/s | — |
+
+MTP speculative decoding hits an **Avg Draft acceptance rate of ~77 %** (per-position: 0.92 / 0.79 / 0.65) with a mean acceptance length of ~3.3, measured on a mixed reasoning + code prompt set at greedy decoding.
 
 Self-test of tool calling with `--tool-call-parser qwen3_coder`: passes (model emits well-formed `<tool_call>...</tool_call>` syntax).
 
@@ -72,6 +75,8 @@ Self-test of tool calling with `--tool-call-parser qwen3_coder`: passes (model e
 | **INT4 asymmetric, group_size = 128** | `q_proj`, `k_proj`, `v_proj`, MLP `gate_proj`, MLP `up_proj`, DeltaNet `in_proj_qkv`, `in_proj_z` |
 | **BF16** | `o_proj`, MLP `down_proj`, `lm_head`, `embed_tokens`, norms, DeltaNet small projections (`in_proj_a`, `in_proj_b`), DeltaNet `out_proj`, vision tower, multimodal projector, 1-layer MTP head, **first 5 and last 5 transformer layers (APEX edge protection)** |
 
+The AWQ skip list also names every `mtp.*` linear module explicitly so the MTP draft head stays unquantized — previous revisions of this checkpoint omitted those entries, which caused vLLM to build the MTP head with AWQ-packed parameters and produced 0 % draft acceptance.
+
 Tuned with [AutoRound](https://github.com/intel/auto-round) on 1024 self-generated reasoning traces (200 iterations per block, batch_size=1).
 
 Calibration data: 1024 self-generated traces from the BF16 source model (256 prompts × 4 generations) covering math, code, logic, analysis, creative writing, general knowledge, tool calling, and Korean.
@@ -82,7 +87,7 @@ Calibration data: 1024 self-generated traces from the BF16 source model (256 pro
 |---|---:|---|
 | `model-*.safetensors` (13 shards) | ~25 GB | Main quantized weights |
 | `model_extra_tensors.safetensors` | ~1 GB | MTP head + edge-protected layers (BF16) |
-| `quantization_config.json` | <1 KB | AWQ config (`quant_method=awq`, `bits=4`, `group_size=128`, `zero_point=true`) |
+| `quantization_config.json` | <1 KB | AWQ config (`quant_method=awq`, `bits=4`, `group_size=128`, `zero_point=true`) with BF16 MTP skip entries |
 | `config.json` + tokenizer + processor configs | <100 MB | Standard metadata |
 
 Total checkpoint size: **~26 GB** (down from ~54 GB BF16 source).

config.json

@@ -232,7 +232,16 @@
       "model.language_model.layers.26.linear_attn.in_proj_a",
       "model.language_model.layers.32.linear_attn.in_proj_a",
       "model.language_model.layers.56.linear_attn.in_proj_b",
-      "model.visual.merger.linear_fc1"
+      "model.visual.merger.linear_fc1",
+      "mtp",
+      "mtp.fc",
+      "mtp.layers.0.self_attn.q_proj",
+      "mtp.layers.0.self_attn.k_proj",
+      "mtp.layers.0.self_attn.v_proj",
+      "mtp.layers.0.self_attn.o_proj",
+      "mtp.layers.0.mlp.gate_proj",
+      "mtp.layers.0.mlp.up_proj",
+      "mtp.layers.0.mlp.down_proj"
     ],
     "nsamples": 1024,
     "provider": "auto-round",
@@ -379,4 +388,4 @@
   "vision_end_token_id": 248054,
   "vision_start_token_id": 248053,
   "torch_dtype": "float16"
-}
+}

quantization_config.json

@@ -229,6 +229,15 @@
     "model.language_model.layers.26.linear_attn.in_proj_a",
     "model.language_model.layers.32.linear_attn.in_proj_a",
     "model.language_model.layers.56.linear_attn.in_proj_b",
-    "model.visual.merger.linear_fc1"
+    "model.visual.merger.linear_fc1",
+    "mtp",
+    "mtp.fc",
+    "mtp.layers.0.self_attn.q_proj",
+    "mtp.layers.0.self_attn.k_proj",
+    "mtp.layers.0.self_attn.v_proj",
+    "mtp.layers.0.self_attn.o_proj",
+    "mtp.layers.0.mlp.gate_proj",
+    "mtp.layers.0.mlp.up_proj",
+    "mtp.layers.0.mlp.down_proj"
   ]
-}
+}