Add Kimi-K2.7-Code-DSpark-GGUF

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Kimi-K2.7-Code-DSpark-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Kimi-K2.7-Code-DSpark-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38ab345731142f801469d741cb7ab00737021ac3a8c8c1a9895c11f3388dba21
+size 1214800864

Kimi-K2.7-Code-DSpark-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edffc3904bb676c92ae73e283fc3902134fe8f05361428474620f72510b65490
+size 1528174048

Kimi-K2.7-Code-DSpark-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb82a13e6f1cfae2305509f220123aef742878bfcd0d118940a25136c8a84fd0
+size 1874079712

Kimi-K2.7-Code-DSpark-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49711b4f8902dcec1c9c15fc1d0a1f1c091a8bc90ca95cbb8652ae788062fec9
+size 2221658080

Kimi-K2.7-Code-DSpark-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3c9d50d2563a47871183f0f0065386fabccf1ebfe3560076c2e727231acd0f1
+size 2590960096

Kimi-K2.7-Code-DSpark-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0508805680517b76cee0e75e2efb27ee2fdd7485f610476712df1255261651d
+size 3353720800

Kimi-K2.7-Code-DSpark-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef7972da0008150567e57e48bec15c028a88884d88139e8312d2cc42dcd33881
+size 6306342880

README.md

@@ -0,0 +1,116 @@
+---
+base_model: novita/kimi-k2.7-code-dspark
+tags:
+- speculators
+- safetensors
+- speculative-decoding
+- dspark
+- text-generation
+- custom_code
+- base_model:moonshotai/Kimi-K2.7-Code
+- base_model:finetune:moonshotai/Kimi-K2.7-Code
+- region:us
+---
+
+# Kimi-K2.7-Code DSpark GGUF
+
+GGUF quantizations of [**novita DSpark draft model**](https://huggingface.co/novita/kimi-k2.7-code-dspark) for [**Kimi-K2.7-Code**](https://huggingface.co/moonshotai/Kimi-K2.7-Code).
+
+Use with [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp), a llama.cpp fork with advanced quantization features.
+
+---
+
+# Kimi-K2.7-Code DSpark speculator
+
+## Overview
+
+A DSpark speculator model for the `Kimi-K2.7-Code` base model, enabling faster
+inference through speculative decoding. DSpark extends the DFlash parallel draft
+backbone with two lightweight heads: a **Markov logit-bias head** (low-rank
+intra-block token dependency) and a **per-position confidence head** (accept-rate
+prediction). This checkpoint was trained in the Camelot-Ray online pipeline,
+where the draft consumes hidden states streamed from a live Kimi-K2.7-Code vLLM
+server.
+
+This export is from Camelot exp38 checkpoint 3.
+
+## Model Specifications
+
+- **Base Model**: Kimi-K2.7-Code
+- **Format**: Safetensors (single-file bf16, 6.3 GB, 44 tensors)
+- **Draft**: 3 layers (Qwen3-style GQA), hidden 7168, 56 heads / 8 KV heads,
+  head_dim 128, FFN 18432, rope_theta 50000, `block_size=8`
+- **Vocabulary**: pruned draft vocab 32,000 (d2t/t2d remap tables shipped in the
+  weights), target vocab 163,840; mappings reused from the Kimi-K2.6/K2.7
+  compatible tokenizer setup
+- **DSpark heads**: Markov rank 256 (vanilla), confidence head (with-markov),
+  `mask_token_id=163608`
+- **Aux hidden-state layers**: [1, 29, 57]
+- **Trained context**: seq 20000
+
+## Evaluation Results
+
+Online vLLM nightly spec-decode, greedy decoding, TP=8, Kimi-K2.7-Code verifier,
+`max_model_len=20000`, cudagraphs enabled, and
+`fuse_allreduce_rms=false`.
+
+The table also includes Novita's public Eagle3-MLA draft
+`novita/kimi-k2.7-code-eagle3-mla` under the same Kimi-K2.7-Code verifier,
+TP=8, cudagraph, and fusion-off serving setup. Cells show
+`tok/s / speedup / accept_len`. The standard rows use 6 prompts per benchmark;
+code-extra rows use the full LiveCodeBench and SPEED-Bench coding manifests
+with `max_tokens=512`.
+
+| benchmark | rows | baseline tok/s | DSpark n=3 | DSpark n=7 | Novita Eagle3 n=3 | Novita Eagle3 n=7 | best |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
+| gsm8k | 6 | 132.0 | 282.2 / 2.14x / 2.937 | 309.1 / 2.34x / 3.659 | 281.7 / 2.13x / 2.941 | 277.3 / 2.10x / 3.595 | DSpark n=7 |
+| math500 | 6 | 132.0 | 317.1 / 2.40x / 3.249 | 367.4 / 2.78x / 4.303 | 288.6 / 2.19x / 3.026 | 294.5 / 2.23x / 3.851 | DSpark n=7 |
+| aime | 6 | 131.5 | 276.8 / 2.10x / 2.778 | 318.4 / 2.42x / 3.716 | 263.2 / 2.00x / 2.766 | 275.3 / 2.09x / 3.626 | DSpark n=7 |
+| humaneval | 6 | 132.1 | 285.1 / 2.16x / 2.875 | 336.6 / 2.55x / 3.953 | 285.9 / 2.17x / 3.029 | 291.8 / 2.21x / 3.850 | DSpark n=7 |
+| livecodebench | 121 | 129.8 | 227.5 / 1.75x / 2.306 | 231.0 / 1.78x / 2.696 | 219.5 / 1.69x / 2.342 | 198.5 / 1.52x / 2.593 | DSpark n=7 |
+| speedbench_coding | 80 | 131.2 | 282.0 / 2.15x / 2.837 | 303.7 / 2.31x / 3.530 | 272.1 / 2.06x / 2.886 | 281.6 / 2.13x / 3.693 | DSpark n=7 |
+
+Use DSpark with `num_speculative_tokens=7` as the default for code, math, and
+most reasoning traffic.
+
+## Serving with vLLM
+
+Requires a vLLM nightly with DSpark support:
+
+```bash
+uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
+
+vllm serve moonshotai/Kimi-K2.7-Code \
+    --tensor-parallel-size 8 \
+    --max-model-len 20000 \
+    --trust-remote-code \
+    --compilation-config='{"pass_config": {"fuse_allreduce_rms": false}}' \
+    --speculative-config '{
+        "model": "novita/kimi-k2.7-code-dspark",
+        "num_speculative_tokens": 7,
+        "method": "dspark"
+    }'
+```
+
+Known vLLM-nightly caveats, with workarounds:
+
+1. Draft-side FA3 AOT scheduling can crash with `scheduler_metadata must have
+   shape (metadata_size)` because the GPU-worker spec-decode path misses
+   `fast_build=True` when building draft attention metadata. Patch
+   `vllm/v1/worker/gpu/spec_decode/speculator.py` and
+   `vllm/v1/worker/gpu/attn_utils.py` to pass `fast_build=True`.
+2. CUDA-graph capture can fail with a flashinfer allreduce workspace-size error
+   under spec-decode token expansion; disable the fusion:
+   `--compilation-config='{"pass_config": {"fuse_allreduce_rms": false}}'`.
+
+## Training Details
+
+- **Initialization**: continued from the Kimi-K2.7-Code DSpark exp37 checkpoint
+- **Data**: Kimi-K2.7-Code training mix with public Kimi-MTP data and hidden
+  states streamed from the live Kimi-K2.7-Code verifier; seq 20000
+- **Steps**: 20000 optimizer steps
+- **Schedule**: lr 3e-4 cosine, warmup 300, global batch 8, accumulation 2
+- **Loss**: 0.1 CE + 0.9 TV over block-diffusion anchors, decay_gamma 4.0,
+  max_anchors 3072
+- **Semantics**: `apply_verifier_norm=False`, hidden_states = concat of aux
+  layers [1, 29, 57]