Add Kimi K3.1 DSpark draft (r2, step 15,983, YaRN-32 serving config) and model card

README.md

@@ -0,0 +1,48 @@
+---
+tags:
+  - speculative-decoding
+  - dspark
+  - speculators
+  - vllm
+---
+
+# Kimi K3.1 DSpark
+
+DSpark draft model for Kimi K3.1.
+
+Five full-causal MLA layers conditioned on hidden states from five of the target's layers,
+trained at `block_size=7`: one forward pass drafts seven tokens. It trained on 32,768-token
+packs; `config.json` declares YaRN factor 32 (as in `Inferact/Kimi-K3-DSpark-Block5`) for
+serving up to 1,048,576 tokens.
+
+## Evaluation settings
+
+- **Server:** vLLM (Kimi K3.1 build `ab910db623`), target with TP8 on 8 GB300 GPUs, prefix
+  caching off. Speculative config
+  `{"method":"dspark","model":"Inferact/Kimi-K3.1-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}`:
+  probabilistic drafts (the draft samples its tokens and passes its probabilities to
+  verification) with block verification (Sun et al., 2024).
+- **Client:** `vllm-bench`, chat completions, temperature 1.0, top_p 0.95, chat template
+  defaults, one benchmark at a time.
+- **Metric:** acceptance length = 1 + accepted draft tokens / drafts (includes the bonus token),
+  from the server's spec-decode counters.
+- **Draft RoPE:** these results used the same weights with the plain RoPE declaration
+  (`rope_type: default`), not this repo's YaRN declaration.
+
+## Results
+
+| Benchmark | Prompts | Max tokens | Acceptance length | Accept rate, positions 1-7 |
+| --- | ---: | ---: | ---: | --- |
+| GSM8K | 256 | 1,024 | **6.020** | 0.94 0.87 0.79 0.71 0.64 0.57 0.50 |
+| HumanEval | 164 | 1,024 | 5.327 | 0.90 0.80 0.69 0.60 0.52 0.45 0.38 |
+| MBPP | 256 | 1,024 | 4.875 | 0.89 0.76 0.64 0.53 0.43 0.35 0.28 |
+| MATH-500 | 500 | 2,048 | 4.188 | 0.82 0.65 0.52 0.41 0.32 0.26 0.21 |
+| SPEED-Bench low entropy (10K-token inputs) | 512 | 1,536 | 3.863 | 0.81 0.62 0.47 0.35 0.27 0.20 0.15 |
+| MT-Bench | 80 | 1,024 | 3.605 | 0.79 0.59 0.42 0.30 0.22 0.16 0.12 |
+| SWE-bench Pro | 128 | 2,048 | 3.417 | 0.77 0.56 0.40 0.28 0.19 0.13 0.09 |
+| AIME 2026 | 30 | 4,096 | 2.912 | 0.70 0.47 0.30 0.19 0.12 0.08 0.05 |
+| **Pooled** | | | **4.012** | 0.82 0.64 0.49 0.38 0.29 0.23 0.18 |
+
+On the same benchmarks and settings, `Inferact/Kimi-K3-DSpark-Block5` (5 draft tokens) pools
+3.226. The five-shot GSM8K accuracy check (1,319 problems, temperature 0.7) scored 96.06% exact
+match with zero request errors.

config.json

@@ -0,0 +1,76 @@
+{
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 163584,
+  "eos_token_id": 163586,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 7168,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "kv_lora_rank": 512,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "num_attention_heads": 96,
+  "num_hidden_layers": 5,
+  "num_key_value_heads": 96,
+  "pad_token_id": 163839,
+  "q_lora_rank": 1536,
+  "qk_head_dim": 192,
+  "qk_nope_head_dim": 128,
+  "qk_rope_head_dim": 64,
+  "rms_norm_eps": 1e-05,
+  "rope_interleave": true,
+  "rope_parameters": {
+    "rope_type": "yarn",
+    "rope_theta": 1000000.0,
+    "factor": 32.0,
+    "original_max_position_embeddings": 32768,
+    "beta_fast": 32,
+    "beta_slow": 1,
+    "mscale": 1.0,
+    "mscale_all_dim": 0.0,
+    "attn_factor": 0.7426255848312643
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "use_cache": true,
+  "use_sliding_window": false,
+  "v_head_dim": 128,
+  "vocab_size": 163840,
+  "architectures": [
+    "K3DSparkModel"
+  ],
+  "model_type": "k3_dspark",
+  "max_position_embeddings": 1048576,
+  "rope_theta": 1000000.0,
+  "num_target_layers": 5,
+  "target_hidden_size": 7168,
+  "target_num_hidden_layers": 93,
+  "target_layer_ids": [
+    75,
+    79,
+    83,
+    87,
+    91
+  ],
+  "mask_token_id": 163839,
+  "markov_rank": 512,
+  "markov_head_type": "vanilla",
+  "enable_confidence_head": true,
+  "confidence_head_with_markov": true,
+  "sample_from_anchor": true,
+  "block_size": 7,
+  "sliding_window_non_causal": false,
+  "full_attention_causal": true,
+  "dflash_config": {
+    "causal": true
+  },
+  "draft_vocab_size": 163840,
+  "torch_dtype": "bfloat16"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27ad5181c8ea09d125615796b3d7e279dfe4300207b895de84a107611533d027
+size 5373511978