deepseek-ai

/

DeepSeek-V4-Flash-0731

Like 3.93k

Follow

DeepSeek 146k

Several threads here (#22, #39, #43) and a number of GitHub issues carried reports of output corruption when DSpark ran at draft depth 5, the checkpoint's dspark_block_size. We spent this week root-causing it on 4x RTX PRO 6000 (SM120, sglang) and are closing the loop where both DeepSeek and integrators will see it.

The model is innocent. The fault was an allocation hazard in one serving-stack code path: on sglang's SM120 fallback, the mHC combine ran an einsum whose large transients were allocated inside an NCCL symmetric-memory region, and collisions with in-flight collective buffers reached committed tokens. Depth 5 was never special to any code; that depth's verify batch shape just produced the pool occupancy that made collisions land in output. Full bisect ladder and evidence in sglang#33800.

For anyone serving this model, three practical things fall out:

1. If you're on sglang/SM120, take PR #29927 (its kernel rewrite fixes this as a side effect) or sgl-project/sglang#34021, the minimal one-line fix we filed upstream — validated: the corrupting build plus only that restructure ran 323/323 clean where the stock build corrupts ~39 per 1,000 requests. Our recipe image carries the fix: github.com/ombori/deepseek-v4-flash-0731-sglang-4x-rtx-pro-6000.
2. You can verify your own stack in two minutes: the corruption has a zero-false-positive signature — any server Decode-batch interval with accept length at or below 2.0 during load. We never saw one across 5,700+ clean intervals; broken stacks show dozens within the first minutes after a cold boot.
3. On depth selection, the checkpoint field is what misleads: acceptance is content-bound, not trained-length-bound, so dspark_block_size carries no serving significance. Whether a wide window pays depends on workload and hardware together: code/tool traffic saturates a 5-window and keeps paying at 7, reasoning-heavy prose accepts ~2-3 regardless of window, and the cost of unharvested rows scales with how expensive a verify row is on your deployment (nearly free on batch-amortized datacenter GPUs, meaningful on PCIe workstations, and decisive once experts are offloaded to system RAM — single-GPU setups running this checkpoint through offload forks, or the GGUF quants on llama.cpp, where the best-documented configs run depth 2-3).

And the suggestion for DeepSeek: dspark_block_size=5 reads like a recommended serving depth, while your vLLM recipe correctly passes 7 — and most stacks silently infer the serving depth from the checkpoint field, so users end up at 5 without choosing it. A one-line model-card note ("dspark_block_size is the draft head's training block size, not a recommended speculation depth; see the recipes for serving values") would let every deployment price its own window instead of inheriting a training constant.