Instructions to use navispace/Qwen3.6-35B-A3B-W8A16-GPTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use navispace/Qwen3.6-35B-A3B-W8A16-GPTQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="navispace/Qwen3.6-35B-A3B-W8A16-GPTQ")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("navispace/Qwen3.6-35B-A3B-W8A16-GPTQ")
model = AutoModelForMultimodalLM.from_pretrained("navispace/Qwen3.6-35B-A3B-W8A16-GPTQ", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use navispace/Qwen3.6-35B-A3B-W8A16-GPTQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/navispace/Qwen3.6-35B-A3B-W8A16-GPTQ

SGLang

How to use navispace/Qwen3.6-35B-A3B-W8A16-GPTQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "navispace/Qwen3.6-35B-A3B-W8A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use navispace/Qwen3.6-35B-A3B-W8A16-GPTQ with Docker Model Runner:
```
docker model run hf.co/navispace/Qwen3.6-35B-A3B-W8A16-GPTQ
```

navispace commited on 20 days ago

Commit

f2dd3fa

verified ·

1 Parent(s): 85b7d64

W8A16 GPTQ (RTN, no calibration) — see model card

Browse files

Files changed (23) hide show

.gitattributes +2 -0
README.md +200 -0
config.json +137 -0
configuration.json +1 -0
generation_config.json +12 -0
merges.txt +0 -0
model-00001.safetensors +3 -0
model-00002.safetensors +3 -0
model-00003.safetensors +3 -0
model-00004.safetensors +3 -0
model-00005.safetensors +3 -0
model-00006.safetensors +3 -0
model-00007.safetensors +3 -0
model-00008.safetensors +3 -0
model-00009.safetensors +3 -0
model-00010.safetensors +3 -0
model-00011.safetensors +3 -0
model.safetensors.index.json +3 -0
preprocessor_config.json +21 -0
tokenizer.json +3 -0
tokenizer_config.json +305 -0
video_preprocessor_config.json +21 -0
vocab.json +0 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,200 @@

+---
+library_name: transformers
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE
+pipeline_tag: image-text-to-text
+base_model: Qwen/Qwen3.6-35B-A3B
+base_model_relation: quantized
+tags:
+  - gptq
+  - int8
+  - w8a16
+  - quantized
+  - moe
+  - rocm
+  - vllm
+---
+# Qwen3.6-35B-A3B — W8A16 GPTQ (INT8 routed experts, group size 32)
+An 8-bit weight-only quantization of
+[Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), with the
+**routed experts quantized and everything on the always-active path left in
+BF16**. **67 GiB → 39.3 GiB**.
+## How this was quantized
+**Round-to-nearest (RTN), no calibration data.** Each weight is rounded
+independently to the nearest INT8 level with a symmetric per-group scale — no
+Hessian, no error compensation, no forward pass over a dataset. The only
+refinement over naive RTN is searching ~21 scale candidates per group and
+keeping the one that minimises squared error.
+At 8 bits this costs little: relative Frobenius error of the quantized weights
+is ~0.63%. GPTQ's error compensation is what rescues *4-bit* quantization; at
+8 bits there is not much left to rescue.
+> **Note on the format.** `quant_method` is `"gptq"` because that is the *file
+> layout* the inference kernels read. The *algorithm* is RTN.
+### Only the routed experts are quantized — and that is the point
+In an A3B mixture-of-experts only 8 of 256 experts fire per token, so weight
+mass and *active* weight per token are very different quantities:
+| group | total weight | % of total | active/token | **% of active** |
+|---|---|---|---|---|
+| routed experts | 60.00 GiB | 95.8% | 1.88 GiB | 41.7% |
+| `linear_attn` | 1.88 GiB | 3.0% | 1.88 GiB | **41.7%** |
+| `self_attn` | 0.51 GiB | 0.8% | 0.51 GiB | 11.3% |
+| `shared_expert` | 0.23 GiB | 0.4% | 0.23 GiB | 5.2% |
+`linear_attn` alone contributes as much to each token's computation as *all
+eight active experts combined*, while being 3% of the file. Judged by total
+weight, quantizing it looks free; judged by what the model actually computes per
+token, it is over 40% of the work.
+So everything outside the routed experts stays in **BF16**, at a cost of
+1.21 GiB. That takes the quantized share of the active path from 100% down to
+**41.7%**.
+Quantized (31,488 modules): `mlp.experts.{0..255}.{gate,up,down}_proj` across
+all 40 layers, plus the MTP block's experts.
+Left in **BF16**: `linear_attn`, `self_attn`, `shared_expert`, the router
+(`mlp.gate`), the vision tower, embeddings, `lm_head`, and all norms. The router
+is excluded on purpose — its output decides *which* experts serve a token, so
+error there swaps whole experts rather than perturbing a value, for ~40 MB of
+savings.
+### Settings
+| Setting | Value | Reason |
+|---|---|---|
+| `bits` | 8 | weight-only; activations stay BF16 |
+| `group_size` | 32 | `down_proj` has K=512, which is only 4 groups at g128 and 1 per partition at TP=4 — g32 leaves real margin |
+| `sym` | `true` | vLLM's `AutoGPTQConfig.TYPE_MAP` only accepts `(4, True)` and `(8, True)` |
+| `desc_act` | `false` | required by the `moe_wna16` MoE kernel |
+### A note on the source layout
+The base checkpoint stores experts as fused 3-D tensors —
+`mlp.experts.gate_up_proj` `[256, 1024, 2048]` and `mlp.experts.down_proj`
+`[256, 2048, 512]`. Those are split into per-expert 2-D tensors here, because
+vLLM's `build_expert_params_mapping` looks for
+`experts.{id}.{gate,up,down}_proj` when weights are quantized; its pre-fused
+shortcut maps only to `weight`, never `qweight`.
+## Serving
+Verified on 4 × AMD Radeon RX 7900 XTX (gfx1100, RDNA3) with vLLM 0.25.1 built
+from source for `gfx1100`.
+```bash
+vllm serve <path> \
+  --quantization gptq \
+  --dtype bfloat16 \
+  --tensor-parallel-size 4 \
+  --safetensors-load-strategy eager \
+  --max-parallel-loading-workers 64
+```
+**Use `--dtype bfloat16`, not float16.** The experts run on the `moe_wna16`
+Triton kernel, which accepts both (`get_supported_act_dtypes` returns
+`[bfloat16, half]`), and no dense linear layer is quantized here, so the Exllama
+kernel's FP16-only constraint does not apply to this model.
+This matters in practice. Serving this checkpoint with `--dtype float16` passes
+short prompts but degenerates on long reasoning traces, first into repetition
+loops and then into token-level garbage (runs of `[][][]`, `((((((`, stray
+`<|im_end|>`). The base model is trained in BF16; FP16 saturates at 65504 while
+BF16 shares FP32's exponent range, and Qwen activation outliers overflow it once
+a reasoning trace accumulates. Short generations do not surface this.
+### Measured on 4 × RX 7900 XTX
+- weights: ~9.8 GiB per card at TP=4
+- KV cache: 5.72 GiB per card with `--kv-cache-dtype fp8`, prefix caching and
+  chunked prefill; 1,183,035 tokens
+- served at `--max-model-len 524288` with 2.26× concurrency
+- generation throughput 65–71 tok/s in interactive use — the A3B design moves
+  only ~3 B parameters per decode step
+## Caveats
+Quality was verified by generation, not by benchmark. **No evaluation against
+the original BF16 weights has been run.** The claim that leaving the active path
+in BF16 improves output quality is a reasoned inference from the active-weight
+table above, not a measurement.
+---
+# Original model card — Qwen/Qwen3.6-35B-A3B
+Qwen3.6-35B-A3B is the inaugural open-weight variant in the Qwen3.6 lineup. It
+emphasises practical utility and developer experience, with particular strengths
+in agentic workflows and extended reasoning. The architecture is a
+mixture-of-experts with 35 billion total parameters, activating only 3 billion
+per inference pass.
+## Capabilities
+- **Agentic coding**: frontend development and repository-level code comprehension
+- **Extended reasoning**: preserves reasoning context across multi-turn interactions
+- **Multimodal**: text, image and video inputs
+- **Long context**: natively supports context lengths up to 262,144 tokens
+## Architecture
+**Language model**
+- Parameters: 35B total / 3B activated
+- Hidden dimension: 2,048
+- Layers: 40, structured as 10 × [3 × Gated DeltaNet→MoE + 1 × Gated Attention→MoE]
+- Attention heads: 16 (Q), 2 (KV), head dim 256
+- Linear attention heads: 32 (V), 16 (QK), head dim 128
+- MoE: 256 experts total, 8 routed + 1 shared active per token
+- Context window: 262,144 tokens, extensible to 1,010,000 with YaRN scaling
+**Vision**
+- Integrated vision encoder for image and video understanding alongside text
+## Performance highlights
+- **Coding agents**: SWE-bench Verified 73.4, Terminal-Bench 51.5
+- **Mathematical reasoning**: AIME26 92.7%, HMMT Feb 25 90.7%
+- **Knowledge**: MMLU-Redux 93.3%, C-Eval 90.0%
+- **Multimodal vision**: RealWorldQA 85.3%, MMBench 92.8%
+- **Video understanding**: VideoMMMU 83.7%, MLVU 86.2%
+## Deployment
+Specialised serving engines are advised: SGLang ≥ 0.5.10, vLLM ≥ 0.19.0, or
+KTransformers for CPU-GPU heterogeneous setups. Hugging Face Transformers is
+suitable for lightweight testing only.
+### Sampling parameters
+| mode | parameters |
+|---|---|
+| thinking, general | `temperature=1.0, top_p=0.95, top_k=20, presence_penalty=1.5` |
+| thinking, precise coding | `temperature=0.6, top_p=0.95, top_k=20, presence_penalty=0.0` |
+| non-thinking | `temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5` |
+### Memory
+If you hit out-of-memory errors, consider reducing the context window — though
+keeping at least 128K tokens preserves extended reasoning capabilities.
+## Citation
+```bibtex
+@misc{qwen36_35b_a3b,
+    title = {{Qwen3.6-35B-A3B}: Agentic Coding Power, Now Open to All},
+    url = {https://qwen.ai/blog?id=qwen3.6-35b-a3b},
+    author = {{Qwen Team}},
+    month = {April},
+    year = {2026}
+}
+```

config.json ADDED Viewed

	@@ -0,0 +1,137 @@

+{
+  "architectures": [
+    "Qwen3_5MoeForConditionalGeneration"
+  ],
+  "image_token_id": 248056,
+  "model_type": "qwen3_5_moe",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": 248044,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_5_moe_text",
+    "moe_intermediate_size": 512,
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_experts": 256,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 2,
+    "output_router_logits": false,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "router_aux_loss_coef": 0.001,
+    "shared_expert_intermediate_size": 512,
+    "tie_word_embeddings": false,
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.1",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 27,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_5_moe",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053,
+  "quantization_config": {
+    "quant_method": "gptq",
+    "bits": 8,
+    "group_size": 32,
+    "sym": true,
+    "desc_act": false,
+    "lm_head": false,
+    "checkpoint_format": "gptq",
+    "dynamic": {
+      "+:.*layers\\.\\d+\\.mlp\\.experts$": {
+        "bits": 8,
+        "group_size": 32,
+        "sym": true,
+        "desc_act": false
+      },
+      "-:.*": {}
+    }
+  }
+}

configuration.json ADDED Viewed

	@@ -0,0 +1 @@


1	+ {"framework":"Pytorch","task":"visual-question-answering"}

generation_config.json ADDED Viewed

	@@ -0,0 +1,12 @@

+{
+    "bos_token_id": 248044,
+    "do_sample": true,
+    "eos_token_id": [
+        248046,
+        248044
+    ],
+    "pad_token_id": 248044,
+    "temperature": 1.0,
+    "top_k": 20,
+    "top_p": 0.95
+}

merges.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

model-00001.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:423e795853c326fe7168605d6b506ec426ae0119f02fd38315b2ad2ef5f9c0ed
+size 4297958040

model-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cc1962d800bb3d8a30580420ca768ddb23bae49d62c676a7fc46462cbbf7c205
+size 4297964192

model-00003.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ea9893f54bb6e88742fd81ac2b69b6a8c084a0869b7675750fcf45a89d9feaab
+size 4297961240

model-00004.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cff5902a245f005c212fcf14cef5c3d7536026e16dca711fd6dc902b6734b151
+size 4297964648

model-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b5d21e8dfec8c9f07d7961c2de9a086235f3b2137c1eb22a0c14966869213852
+size 4297168000

model-00006.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a1a869321bb48e7cd2ae510bd6d553ffe5d9911c7c94dd8eafa9af32ad0eae78
+size 4297514624

model-00007.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:12eec0ce30b3f91a92f1078977a9fcb23b9f29feb305e1a5d4ddd8c2ec7246d2
+size 4298467944

model-00008.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dab3126f19bfda9b7131f302b8ea11c30830ecaafa7ca91776cdf350e4590361
+size 4297949416

model-00009.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:590f42f0495e384cb2b44e9c519f8a6cab94796456714b26ef697037ebee3bf4
+size 1940538712

model-00010.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e08bc58c364eb4b0dcddd5c572693a4ae0a096750d3dccb14ce5bf589a27459
+size 4300152368

model-00011.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:14fff30518a757852c3dc34b4a12c0efa3f4a440d4daf808b2f9ac25cf07096f
+size 1568491584

model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:83923c42aad217a4f762c6cb5a1844d3efa3e76da03c6b96ea7b676bd3df47f4
+size 11693288

preprocessor_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42
+size 12807982

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,305 @@

+{
+    "add_prefix_space": false,
+    "added_tokens_decoder": {
+        "248044": {
+            "content": "<|endoftext|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248045": {
+            "content": "<|im_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248046": {
+            "content": "<|im_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248047": {
+            "content": "<|object_ref_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248048": {
+            "content": "<|object_ref_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248049": {
+            "content": "<|box_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248050": {
+            "content": "<|box_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248051": {
+            "content": "<|quad_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248052": {
+            "content": "<|quad_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248053": {
+            "content": "<|vision_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248054": {
+            "content": "<|vision_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248055": {
+            "content": "<|vision_pad|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248056": {
+            "content": "<|image_pad|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248057": {
+            "content": "<|video_pad|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248058": {
+            "content": "<tool_call>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248059": {
+            "content": "</tool_call>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248060": {
+            "content": "<|fim_prefix|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248061": {
+            "content": "<|fim_middle|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248062": {
+            "content": "<|fim_suffix|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248063": {
+            "content": "<|fim_pad|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248064": {
+            "content": "<|repo_name|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248065": {
+            "content": "<|file_sep|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248066": {
+            "content": "<tool_response>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248067": {
+            "content": "</tool_response>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248068": {
+            "content": "<think>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248069": {
+            "content": "</think>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": false
+        },
+        "248070": {
+            "content": "<|audio_start|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248071": {
+            "content": "<|audio_end|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248072": {
+            "content": "<tts_pad>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248073": {
+            "content": "<tts_text_bos>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248074": {
+            "content": "<tts_text_eod>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248075": {
+            "content": "<tts_text_bos_single>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "248076": {
+            "content": "<|audio_pad|>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        }
+    },
+    "additional_special_tokens": [
+        "<|im_start|>",
+        "<|im_end|>",
+        "<|object_ref_start|>",
+        "<|object_ref_end|>",
+        "<|box_start|>",
+        "<|box_end|>",
+        "<|quad_start|>",
+        "<|quad_end|>",
+        "<|vision_start|>",
+        "<|vision_end|>",
+        "<|vision_pad|>",
+        "<|image_pad|>",
+        "<|video_pad|>"
+    ],
+    "bos_token": null,
+    "chat_template": "{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n    {%- if content is string %}\n        {{- content }}\n    {%- elif content is iterable and content is not mapping %}\n        {%- for item in content %}\n            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n                {%- if is_system_content %}\n                    {{- raise_exception('System message cannot contain images.') }}\n                {%- endif %}\n                {%- if do_vision_count %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                {%- endif %}\n                {%- if add_vision_id %}\n                    {{- 'Picture ' ~ image_count.value ~ ': ' }}\n                {%- endif %}\n                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n            {%- elif 'video' in item or item.type == 'video' %}\n                {%- if is_system_content %}\n                    {{- raise_exception('System message cannot contain videos.') }}\n                {%- endif %}\n                {%- if do_vision_count %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                {%- endif %}\n                {%- if add_vision_id %}\n                    {{- 'Video ' ~ video_count.value ~ ': ' }}\n                {%- endif %}\n                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n            {%- elif 'text' in item %}\n                {{- item.text }}\n            {%- else %}\n                {{- raise_exception('Unexpected item type in content.') }}\n            {%- endif %}\n        {%- endfor %}\n    {%- elif content is none or content is undefined %}\n        {{- '' }}\n    {%- else %}\n        {{- raise_exception('Unexpected content type.') }}\n    {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n    {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- if tools and tools is iterable and tools is not mapping %}\n    {{- '<|im_start|>system\\n' }}\n    {{- \"# Tools\\n\\nYou have access to the following functions:\\n\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\" }}\n    {{- '\\n\\nIf you choose to call a function ONLY reply in the following format with NO suffix:\\n\\n<tool_call>\\n<function=example_function_name>\\n<parameter=example_parameter_1>\\nvalue_1\\n</parameter>\\n<parameter=example_parameter_2>\\nThis is the value for the second parameter\\nthat can span\\nmultiple lines\\n</parameter>\\n</function>\\n</tool_call>\\n\\n<IMPORTANT>\\nReminder:\\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\\n- Required parameters MUST be specified\\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\\n</IMPORTANT>' }}\n    {%- if messages[0].role == 'system' %}\n        {%- set content = render_content(messages[0].content, false, true)|trim %}\n        {%- if content %}\n            {{- '\\n\\n' + content }}\n        {%- endif %}\n    {%- endif %}\n    {{- '<|im_end|>\\n' }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {%- set content = render_content(messages[0].content, false, true)|trim %}\n        {{- '<|im_start|>system\\n' + content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" %}\n        {%- set content = render_content(message.content, false)|trim %}\n        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}\n            {%- set ns.multi_step_tool = false %}\n            {%- set ns.last_query_index = index %}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if ns.multi_step_tool %}\n    {{- raise_exception('No user query found in messages.') }}\n{%- endif %}\n{%- for message in messages %}\n    {%- set content = render_content(message.content, true)|trim %}\n    {%- if message.role == \"system\" %}\n        {%- if not loop.first %}\n            {{- raise_exception('System message must be at the beginning.') }}\n        {%- endif %}\n    {%- elif message.role == \"user\" %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- set reasoning_content = reasoning_content|trim %}\n        {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}\n            {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content + '\\n</think>\\n\\n' + content }}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if tool_call.function is defined %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {%- if loop.first %}\n                    {%- if content|trim %}\n                        {{- '\\n\\n<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n                    {%- else %}\n                        {{- '<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n                    {%- endif %}\n                {%- else %}\n                    {{- '\\n<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n                {%- endif %}\n                {%- if tool_call.arguments is defined %}\n                    {%- for args_name, args_value in tool_call.arguments|items %}\n                        {{- '<parameter=' + args_name + '>\\n' }}\n                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}\n                        {{- args_value }}\n                        {{- '\\n</parameter>\\n' }}\n                    {%- endfor %}\n                {%- endif %}\n                {{- '</function>\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.previtem and loop.previtem.role != \"tool\" %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if not loop.last and loop.nextitem.role != \"tool\" %}\n            {{- '<|im_end|>\\n' }}\n        {%- elif loop.last %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- else %}\n        {{- raise_exception('Unexpected message role.') }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- else %}\n        {{- '<think>\\n' }}\n    {%- endif %}\n{%- endif %}",
+    "clean_up_tokenization_spaces": false,
+    "eos_token": "<|im_end|>",
+    "errors": "replace",
+    "model_max_length": 262144,
+    "pad_token": "<|endoftext|>",
+    "split_special_tokens": false,
+    "tokenizer_class": "Qwen2Tokenizer",
+    "unk_token": null,
+    "add_bos_token": false,
+    "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+    "extra_special_tokens": {
+        "audio_bos_token": "<|audio_start|>",
+        "audio_eos_token": "<|audio_end|>",
+        "audio_token": "<|audio_pad|>",
+        "image_token": "<|image_pad|>",
+        "video_token": "<|video_pad|>",
+        "vision_bos_token": "<|vision_start|>",
+        "vision_eos_token": "<|vision_end|>"
+    }
+}

video_preprocessor_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+    "size": {
+        "longest_edge": 25165824,
+        "shortest_edge": 4096
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "video_processor_type": "Qwen3VLVideoProcessor"
+}

vocab.json ADDED Viewed

The diff for this file is too large to render. See raw diff