JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

DeepSeek V4.1 Flash 1017 TrellisMX K2-Mix on two DGX Sparks

A Flare package for two DGX Sparks, serving the community brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix checkpoint at TP2. The routed experts are K2/K3/K4 TrellisMX sidecar streams and the Engram tables are 4-bit K4. The package keeps the official model's attention, shared experts, MTP, vision and tokenizer.

This revision contains 16 concurrent request slots, MTP with up to five verified draft tokens, 8192-token prefill chunks, and a 259 GiB signed payload. The serve default allocates about 14 GiB of KV in total (16.6 million tokens across the pair) and leaves the rest of each Spark's unified memory available. The per-request context limit is 1,048,576 tokens, subject to KV memory.

Status: build prerequisite. This package was produced by the ds41-trellismx engine branch (upstream PR NVIDIA/spark-local-inference#161). A stock flare-inference release before that PR cannot load it. Use a build from that branch, or wait for the PR to land in a release. The package is currently signed by the local-development publisher. The public profile is included as publisher.json (key 58cdfcd6...); this is not a production signature. Use the flags below only on a trusted host, or re-sign the package with a production key before a public deployment.

Requirements

• Two DGX Sparks (GB10/SM121, 48 SMs each) on Ubuntu 24.04-based DGX OS with working NVIDIA drivers and ConnectX-7 RoCE.
• The Flare build above on both hosts.
• The pinned vendor/ds41_trellismx and vendor/ds41_kern/b12x overlay on both hosts; the engine branch rejects TrellisMX checkpoints without it.
• About 259 GiB free on the first Spark for the package, and space for the rank-1 payload on the peer.
• TCP 29900 between hosts, TCP 29800 on the first Spark, TCP 8000 for clients.

Serve

On the first Spark, download this package and start rank zero. On the second, join the rendezvous. The engine distributes the rank-1 payload over the fabric.

# first Spark
package=$(flare download JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE --revision gb10-tp2)
flare run "${package}" --rank 0 --rendezvous 0.0.0.0:29800 \
  --embedding-table-storage disk --embedding-table-cache-size 1GiB \
  --mtp-draft-tokens 5 --max-concurrency 16 \
  --host 0.0.0.0 --port 8000 --served-model-name DeepSeek-V4.1-Flash \
  --publisher-keys ./publisher.json --dangerously-approve-key 58cdfcd6aaf8857ceb76d2bea7547762cc21ace35cd8cfebe673d89b9325cab4 \
  --dangerously-accept-publisher local-development

# second Spark
flare run --rank 1 --rendezvous FIRST_SPARK:29800 \
  --embedding-table-storage disk --embedding-table-cache-size 1GiB \
  --mtp-draft-tokens 5 --max-concurrency 16

Review and accept the model license. The package carries a development signature: the --dangerously-* flags above are required until a production signature replaces this revision; do not use them for an untrusted package.

API

OpenAI-compatible base URL: http://FIRST_SPARK:8000/v1, model DeepSeek-V4.1-Flash, no API key by default.

curl -sS http://FIRST_SPARK:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"DeepSeek-V4.1-Flash",
       "messages":[{"role":"user","content":"Hello"}],
       "max_tokens":128}'

Provenance

• Base weights: deepseek-ai/DeepSeek-V4.1-Flash-1017 (MIT).
• TrellisMX pack: brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix (MIT), revision 451e938fbf5094767349df6d1b7b127a376e5618.
• Routed-expert kernels and TrellisMX driver: the kit's pinned ds41_kern/ds41_trellismx overlay, plus the direct4 performance overlay.
• Engram 4-bit decoding and package support: the engine branch above.