chimcis/GLM-5.3-DSpark-fast-2026-10-27

GLM-5.3 DSpark drafter (2026-10-27)

A DSpark block drafter for speculative decoding with GLM-5.3 (NVFP4) as the target model.

Architecture

• 6 transformer layers, block size 16, sliding-window attention (4096)
• Conditions on target hidden states from layers 5, 19, 33, 47, 61, 75
• Markov head (rank 256) and a confidence head
• Shares the target model embedding and LM head (not included in this repo)

Usage (vLLM)

vllm serve <GLM-5.3 checkpoint> \
  --speculative-config '{"method":"dspark","model":"chimcis/GLM-5.3-DSpark-fast-2026-10-27","num_speculative_tokens":4,"kv_cache_dtype":"fp8","draft_sample_method":"probabilistic"}'

Use probabilistic draft sampling; the drafter was trained and evaluated that way. k=4 is the validated setting.

Training

Warm-started from an earlier DSpark checkpoint and trained on target-model regenerations of agentic, reasoning-heavy chat traffic (TV + cross-entropy loss against the target distribution, Muon optimizer).

Results

On a held-out set of agentic chat requests, at k=4 with probabilistic drafting, mean accepted length per verify step is about 8% higher than a DFlash2 drafter for the same target.