Instructions to use bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4
```

ThinkingCap-Qwen3.8-Flash-Next-NVFP4 / README.md

Adamiros

ThinkingCap: Qwen3.8-Flash-Next, NVFP4

df3922b 20 days ago

preview code

Raw History Blame Contribute Delete

4.84 kB

---
base_model: bottlecapai/ThinkingCap-Qwen3.8-Flash-Next
base_model_relation: quantized
library_name: transformers
tags:
  - fp4
  - nvfp4
  - compressed-tensors
  - vllm
  - token-efficient
  - efficient-thinking
license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
---
<a href="https://www.bottlecapai.com/"><img src="cap_header.png" alt="ThinkingCap — BottleCap AI" width="100%"></a>

# bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4

NVFP4 quantization of [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next), the ThinkingCap finetune of [Qwen3.8-Flash-Next (Qwen Team, 2026)](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) that reasons in fewer tokens.

➡️ Full model description, evaluation results, recommended sampling params, and citation: see the main model card at [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next).

## About this quantization

What is stored at which precision:

- the **routed experts**, in every layer and in the MTP head, in NVIDIA's [NVFP4](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/) format: 4-bit E2M1 values with an FP8 scale per 16 weights and one FP32 scale per expert projection, stored in the [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) `nvfp4-pack-quantized` format. Activations stay in bf16 (weight-only, `NVFP4A16`), and no calibration data was used. An expert's gate and up projections share one FP32 scale, because vLLM runs them as one matrix multiply;
- the **n-gram embedding table** in FP8 (E4M3) with one per-tensor scale, about 51 GB instead of 102 GB in bf16;
- everything else in bf16: attention and linear attention, the shared experts and router, the vision tower, embeddings and `lm_head`.

≈132 GB instead of ≈360 GB in bf16.

## Publish-time checks

- **Structure:** the shard index is complete; the MTP layer (4,637 tensors) and the vision tower (333 tensors) are kept; on 45 sampled experts, including the MTP head's, the gate and up projections share one FP32 scale.
- **Serving smoke test** on two RTX PRO 6000 Blackwell GPUs (tensor parallel 2, vLLM 0.31.0), loading straight from this repo: the model serves, answers an image question correctly (the vision tower works) and the reasoning parser separates the thinking. vLLM runs the weight-only 4-bit experts through its Marlin kernel.

- **Accuracy and thinking length against the bf16 model**, served on two H200s (tensor parallel 2, vLLM 0.31.0, no speculative decoding). Seeds are averaged within a question, and the 95% interval comes from resampling questions:

| Benchmark | Questions × seeds (bf16 / NVFP4) | bf16 | NVFP4 | Δ accuracy | Δ mean thinking tokens |
|---|---|---|---|---|---|
| AIME 2026 | 30 × 32 / 8 | 97.19% | 97.92% | +0.73 pp [−0.52, +1.98] | +17.8% [+11.0, +22.8] |
| RealWorldQA | 765 × 8 / 2 | 84.67% | 85.23% | +0.56 pp [−0.62, +1.76] | +4.5% [−6.2, +16.0] |

Accuracy is unchanged within noise; on AIME 2026 the NVFP4 model thinks 18% longer than the bf16 model, mostly on the longest problems. No answer hit the token cap.

- **MTP speculative decoding** on the same two H200s (3 draft tokens, probabilistic drafting, AIME 2026 and RealWorldQA): **2.76 tokens accepted per decoding step** on average, as for the bf16 and FP8 models (2.77). vLLM issue [#55357](https://github.com/vllm-project/vllm/issues/55357), MTP draft acceptance dropping to zero with repetition inside the thinking on this architecture, is still open; it did not reproduce in our test.

## Usage (vLLM)

Requires vLLM 0.31.0 or newer.

```bash
vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
```

Use the sampling settings from the main model card. vLLM keeps the n-gram table in pinned host RAM by default (about 51 GB). The checkpoint keeps the MTP head; to serve with MTP speculative decoding (measured under *Publish-time checks*):

```bash
vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
```

## License

Released under the base model's own license, the Qwen Community License 1.0 (full text in [LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4/blob/main/LICENSE)). Its conditions carry over to this model, including the separate license from Qwen that Model-as-a-Service and AI work-assistant businesses need for commercial use.

	---
	base_model: bottlecapai/ThinkingCap-Qwen3.8-Flash-Next
	base_model_relation: quantized
	library_name: transformers
	tags:
	- fp4
	- nvfp4
	- compressed-tensors
	- vllm
	- token-efficient
	- efficient-thinking
	license: other
	license_name: qwen-community-license-1.0
	license_link: LICENSE
	---

	<a href="https://www.bottlecapai.com/"><img src="cap_header.png" alt="ThinkingCap — BottleCap AI" width="100%"></a>

	# bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4

	NVFP4 quantization of [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next), the ThinkingCap finetune of [Qwen3.8-Flash-Next (Qwen Team, 2026)](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) that reasons in fewer tokens.

	➡️ Full model description, evaluation results, recommended sampling params, and citation: see the main model card at [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next).

	## About this quantization

	What is stored at which precision:

	- the routed experts, in every layer and in the MTP head, in NVIDIA's [NVFP4](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/) format: 4-bit E2M1 values with an FP8 scale per 16 weights and one FP32 scale per expert projection, stored in the [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) `nvfp4-pack-quantized` format. Activations stay in bf16 (weight-only, `NVFP4A16`), and no calibration data was used. An expert's gate and up projections share one FP32 scale, because vLLM runs them as one matrix multiply;
	- the n-gram embedding table in FP8 (E4M3) with one per-tensor scale, about 51 GB instead of 102 GB in bf16;
	- everything else in bf16: attention and linear attention, the shared experts and router, the vision tower, embeddings and `lm_head`.

	≈132 GB instead of ≈360 GB in bf16.

	## Publish-time checks

	- Structure: the shard index is complete; the MTP layer (4,637 tensors) and the vision tower (333 tensors) are kept; on 45 sampled experts, including the MTP head's, the gate and up projections share one FP32 scale.
	- Serving smoke test on two RTX PRO 6000 Blackwell GPUs (tensor parallel 2, vLLM 0.31.0), loading straight from this repo: the model serves, answers an image question correctly (the vision tower works) and the reasoning parser separates the thinking. vLLM runs the weight-only 4-bit experts through its Marlin kernel.

	- Accuracy and thinking length against the bf16 model, served on two H200s (tensor parallel 2, vLLM 0.31.0, no speculative decoding). Seeds are averaged within a question, and the 95% interval comes from resampling questions:

	\| Benchmark \| Questions × seeds (bf16 / NVFP4) \| bf16 \| NVFP4 \| Δ accuracy \| Δ mean thinking tokens \|
	\|---\|---\|---\|---\|---\|---\|
	\| AIME 2026 \| 30 × 32 / 8 \| 97.19% \| 97.92% \| +0.73 pp [−0.52, +1.98] \| +17.8% [+11.0, +22.8] \|
	\| RealWorldQA \| 765 × 8 / 2 \| 84.67% \| 85.23% \| +0.56 pp [−0.62, +1.76] \| +4.5% [−6.2, +16.0] \|

	Accuracy is unchanged within noise; on AIME 2026 the NVFP4 model thinks 18% longer than the bf16 model, mostly on the longest problems. No answer hit the token cap.

	- MTP speculative decoding on the same two H200s (3 draft tokens, probabilistic drafting, AIME 2026 and RealWorldQA): 2.76 tokens accepted per decoding step on average, as for the bf16 and FP8 models (2.77). vLLM issue [#55357](https://github.com/vllm-project/vllm/issues/55357), MTP draft acceptance dropping to zero with repetition inside the thinking on this architecture, is still open; it did not reproduce in our test.

	## Usage (vLLM)

	Requires vLLM 0.31.0 or newer.

	```bash
	vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
	--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
	```

	Use the sampling settings from the main model card. vLLM keeps the n-gram table in pinned host RAM by default (about 51 GB). The checkpoint keeps the MTP head; to serve with MTP speculative decoding (measured under Publish-time checks):

	```bash
	vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
	--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml \
	--speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
	```

	## License

	Released under the base model's own license, the Qwen Community License 1.0 (full text in [LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4/blob/main/LICENSE)). Its conditions carry over to this model, including the separate license from Qwen that Model-as-a-Service and AI work-assistant businesses need for commercial use.