---
base_model: bottlecapai/ThinkingCap-Qwen3.8-Flash-Next
base_model_relation: quantized
library_name: transformers
tags:
  - fp4
  - nvfp4
  - compressed-tensors
  - vllm
  - token-efficient
  - efficient-thinking
license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
---

<a href="https://www.bottlecapai.com/"><img src="cap_header.png" alt="ThinkingCap — BottleCap AI" width="100%"></a>

# bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4

NVFP4 quantization of [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next), the ThinkingCap finetune of [Qwen3.8-Flash-Next (Qwen Team, 2026)](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) that reasons in fewer tokens.

➡️ Full model description, evaluation results, recommended sampling params, and citation: see the main model card at [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next).

## About this quantization

What is stored at which precision:

- the **routed experts**, in every layer and in the MTP head, in NVIDIA's [NVFP4](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/) format: 4-bit E2M1 values with an FP8 scale per 16 weights and one FP32 scale per expert projection, stored in the [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) `nvfp4-pack-quantized` format. Activations stay in bf16 (weight-only, `NVFP4A16`), and no calibration data was used. An expert's gate and up projections share one FP32 scale, because vLLM runs them as one matrix multiply;
- the **n-gram embedding table** in FP8 (E4M3) with one per-tensor scale, about 51 GB instead of 102 GB in bf16;
- everything else in bf16: attention and linear attention, the shared experts and router, the vision tower, embeddings and `lm_head`.

≈132 GB instead of ≈360 GB in bf16.

## Publish-time checks

- **Structure:** the shard index is complete; the MTP layer (4,637 tensors) and the vision tower (333 tensors) are kept; on 45 sampled experts, including the MTP head's, the gate and up projections share one FP32 scale.
- **Serving smoke test** on two RTX PRO 6000 Blackwell GPUs (tensor parallel 2, vLLM 0.31.0), loading straight from this repo: the model serves, answers an image question correctly (the vision tower works) and the reasoning parser separates the thinking. vLLM runs the weight-only 4-bit experts through its Marlin kernel.

- **Accuracy and thinking length against the bf16 model**, served on two H200s (tensor parallel 2, vLLM 0.31.0, no speculative decoding). Seeds are averaged within a question, and the 95% interval comes from resampling questions:

| Benchmark | Questions × seeds (bf16 / NVFP4) | bf16 | NVFP4 | Δ accuracy | Δ mean thinking tokens |
|---|---|---|---|---|---|
| AIME 2026 | 30 × 32 / 8 | 97.19% | 97.92% | +0.73 pp [−0.52, +1.98] | +17.8% [+11.0, +22.8] |
| RealWorldQA | 765 × 8 / 2 | 84.67% | 85.23% | +0.56 pp [−0.62, +1.76] | +4.5% [−6.2, +16.0] |

Accuracy is unchanged within noise; on AIME 2026 the NVFP4 model thinks 18% longer than the bf16 model, mostly on the longest problems. No answer hit the token cap.

- **MTP speculative decoding** on the same two H200s (3 draft tokens, probabilistic drafting, AIME 2026 and RealWorldQA): **2.76 tokens accepted per decoding step** on average, as for the bf16 and FP8 models (2.77). vLLM issue [#55357](https://github.com/vllm-project/vllm/issues/55357), MTP draft acceptance dropping to zero with repetition inside the thinking on this architecture, is still open; it did not reproduce in our test.

## Usage (vLLM)

Requires vLLM 0.31.0 or newer.

```bash
vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
```

Use the sampling settings from the main model card. vLLM keeps the n-gram table in pinned host RAM by default (about 51 GB). The checkpoint keeps the MTP head; to serve with MTP speculative decoding (measured under *Publish-time checks*):

```bash
vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
```

## License

Released under the base model's own license, the Qwen Community License 1.0 (full text in [LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4/blob/main/LICENSE)). Its conditions carry over to this model, including the separate license from Qwen that Model-as-a-Service and AI work-assistant businesses need for commercial use.