respanai/span-02

Read the announcement: Introducing span-2 →

Span-02 is a frontier decision model. Send it a state (text or JSON, plus up to 4 images) and typed questions; it returns a calibrated probability distribution for each answer. No free text, no parsing.

• Typed answers: choice, yes/no (noul), score on a scale, or number.
• Fast: most requests answer in tens of milliseconds. When unsure, the model thinks briefly first.
• Efficient: Gemma 4 26B-A4B fine-tune (~4B active parameters), NVFP4/FP8 (16.4 GB), served by its own Rust/CUDA engine included here.
• One GPU: RTX PRO 6000 or RTX 5090 (Blackwell).

Decision Index

_{Official Decision Index: span-02 is on the latency–score frontier at 33 ms median.}

official result (1 × RTX PRO 6000)
Full score	62.6 (rank =4, tied with 4 other models)
Median latency	33 ms (p95 822 ms)
Requests answered	100% (110,201 of 110,201)

Quickstart

The weights run only with the included engine (engine/), not with transformers or vLLM.

Needs an NVIDIA driver with CUDA 12.9 support and the NVIDIA container toolkit:

hf download respanai/span-02 --local-dir span-02 && cd span-02
docker build -t span-02 .
docker run --gpus all -p 8000:8000 -v "$PWD":/model:ro -e DLLM_MODEL_DIR=/model -v span02-data:/data span-02

Ready in about 20 s, when GET /health returns {"ok": true, "model": "span-02"}. Without Docker, build the engine with cargo build --release -p dllm-server in engine/ (CUDA 12.8+) and run bash launch.sh.

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"customer": "My package arrived damaged and nobody answers my emails. I want a refund now."},
  "questions": {
    "dept":   {"type": "choice", "instructions": "Which team should handle this?",
               "criteria": {"returns": "Exchanges, wrong or damaged items",
                            "shipping": "Delivery status, delays, lost packages",
                            "billing": "Charges, invoices, payment problems"}},
    "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}
  }
}'

{"model": "span-02",
 "answers": {
   "dept":   {"type": "choice", "choice": "returns", "confidence": 0.945,
              "probabilities": {"returns": 0.963, "shipping": 0.024, "billing": 0.013}},
   "refund": {"type": "noul", "noul": 0.99999}},
 "usage": {"input_tokens": 410, "reasoning_tokens": 0, "output_tokens": 0, "latency_ms": 31.0}}

(Values are illustrative.) The API follows TypeSafe's System One API, so its clients work unchanged.

How it works

1. Read the answer directly: one forward pass per question reads the probabilities of the option labels. The shared state is computed once per request.
2. Think if unsure: if the top answer is below 0.8, the model writes a short thought, reads again, and averages the two distributions.
3. Stay on time: thoughts have a time budget (700 ms by default) and use speculative decoding.

API

POST /v1/systemone answers a request; GET /health reports readiness.

request field
state	required; any JSON value, treated as data
questions	required; {id: question}
assets	optional; images {"type": "image", "format": "png" | "jpg", "data_base64": "..."}, at most 4
reasoning	optional; {"mode": "auto" | "on" | "off", "budget_ms": 0-600000, "trace": bool}

question type	criteria	answer
choice	{key: description}, 1–679 options	choice, confidence, probabilities
noul	optional {"true": ..., "false": ...}	noul = probability of true
score	ordered list of 2–10 levels	score = expected level, legend, probabilities, confidence
number	none	number, text, confidence

The response has answers (plus reasoning for questions that thought) and usage (input, reasoning and output tokens, server latency). Errors are {"error": {"type", "message"}} with the usual HTTP statuses; 529 means busy, retry shortly.

Full API reference

Request

field
state	required	Any JSON value. The model treats it as data, not instructions.
questions	required	{id: question}. Answers come back in the same order.
assets	optional	Images keyed by id: {"type": "image", "format": "png" | "jpg" | "jpeg", "data_base64": "..."}. Mentioning an id in the state or a question is optional; see Images.
images	optional	A list of images as data URIs or base64 (used by the Decision Index kit). At most 4 images in total.
reasoning	optional	{"mode": "auto" | "on" | "off", "budget_ms": 0-600000, "trace": true | false}
model	optional	Accepted and ignored; responses always say span-02.

• mode: auto (default) thinks only when unsure; on always thinks; off never thinks.
• budget_ms: the thinking time limit for this request; 0 means no limit. If omitted, on thinks until the thought ends (at most 4,096 tokens) and auto uses the server default (--budget-ms, 700).
• trace: return each thought's text.
• Bodies can be JSON, multipart/form-data (a request part plus one file part per image, referenced as {"part": name}), or compressed with Content-Encoding: zstd or gzip. Unknown top-level fields are ignored.

Question types

type	criteria	answer
choice	{key: description}, 1 to 679 options	choice, confidence, probabilities
noul	optional {"true": ..., "false": ...}	noul = probability of true
score	ordered list of 2 to 10 levels, low to high	score = Σ i·pᵢ, legend, probabilities, confidence
number	none	number (null if the text is not a number), text, confidence

• choice confidence = (p_max − 1/n) / (1 − 1/n): 0 for a uniform guess, 1 for certainty.
• score confidence = 1 − (expected distance from the most likely level) / (the same for a uniform guess).
• number confidence = exp(mean log-probability of the answer tokens). Number questions think only with mode: "on", because the think gate is trained on option questions.

Response

• answers[id]: the typed answer. If the question thought, it also has reasoning: {"tokens": N} (plus trace when requested).
• usage: input_tokens (the prompt tokens of every question; the shared state counts once per question and each image its image tokens + 2), reasoning_tokens (thought tokens), output_tokens (number-answer tokens; other answers are read in one pass and generate none), latency_ms (server time).
• image_tokens: total image tokens, present only when the request has images.

Images

The model was trained with all images (up to 4) before the text, referred to as "Image 1", "Image 2", and so on. The server shows images first, then assets in the order sent, and replaces every mention of an asset id in the state, instructions and criteria with that image's label. For example {"state": {"photo": "a3f1c2e8-found"}, "assets": {"a3f1c2e8-found": ..., "77b0d4a1-poster": ...}} reaches the model as {"photo": "Image 1"}, and "the dog in this picture: 77b0d4a1-poster" becomes "the dog in this picture: Image 2". Assets that are never mentioned are still shown. Images are PNG or JPEG, at most 16 MiB each, and are resized to the image budget (280 tokens per image by default, 560 in the Decision Index settings).

Errors

Errors look like {"error": {"type": "...", "message": "..."}}.

status	type	when
400	invalid_request	invalid JSON, or a broken compressed or multipart body
400	invalid_asset	unknown asset type or format, bad base64, not a PNG/JPEG, more than 4 images
400	max_tokens_exceeded	the request does not fit the context or the KV cache ("maximum context length")
401	authentication_error	missing or wrong key (only when the server sets AR_API_KEY)
404	not_found	any other path
413	request_too_large	body over 64 MB, or an image over 16 MiB
415	unsupported_encoding	a Content-Encoding other than zstd or gzip
422	validation_error	missing fields, bad question types or criteria, too many options, bad reasoning
529	overloaded	the server is busy; retry shortly
500	server_error	an internal failure

Performance

one RTX PRO 6000
Decision Index median latency	33 ms (p95 822 ms)
Throughput, 64 requests in flight	167 req/s without thinking, 150 req/s with auto (p99 0.85 s)

On an RTX 5090 the engine is 1.3 to 7× faster than vLLM without thinking.

Requirements

• NVIDIA Blackwell GPU, compute capability 12.0: RTX PRO 6000 (96 GB) or RTX 5090 (32 GB).
• Driver with CUDA 12.9 support (R575+). About 28 GB of GPU memory by default.
• Quantization: MoE experts NVFP4; attention, dense MLP and LM head FP8; vision tower and KV cache bf16.
• Limits: 131,072-token context per question, 4 images, 64 MB body.

Decision Index settings

The Decision Index submission runs launch.sh as shipped:

--max-len 131072 --budget-ms 0 --tau 0.8 --cap 1024 --think-ms 750 --think-cost think_cost.json
--image-soft-tokens 560 --serial --no-prefix-cache --spec-max-branches 16 --pages 12000 --kv-gb 24

• Deterministic thinking budget: --budget-ms 0 --think-ms 750 --think-cost think_cost.json. The wall clock is never read. Instead, a fixed cost model turns a 750 ms target into a thought-token cap that depends only on the request (prompt length, images, number of thinking questions), so the result does not depend on the machine or its load. A short single question gets about 330 thought tokens; requests with many thinking questions or long prompts get fewer. --cap 1024 is the hard limit.
• One request at a time, no prefix cache: --serial --no-prefix-cache, so an answer does not depend on other requests. Requests where several questions think at once can still vary very slightly in their post-thought probabilities between runs.
• Image budget: --image-soft-tokens 560, about 1.3 MP with the aspect ratio kept.
• KV cache: --pages 12000 --kv-gb 24, sized so every Decision Index request fits on its own.

License

The weights derive from Google's Gemma 4 (google/gemma-4-26B-A4B-it) under the Gemma 4 license (Apache 2.0). The engine is Apache-2.0 and credits the projects it ports from (vLLM, SGLang, FlashInfer and others) in NOTICE.

Citation

@misc{li2026span02,
  title        = {span-02: a fast multimodal decision model},
  author       = {Li, Harvey},
  organization = {Respan},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/respanai/span-02}},
  note         = {Blog post: \url{https://www.respan.ai/blog/introducing-span-2}}
}

Contact: Harvey Li (harvey@respan.ai), Respan.