Instructions to use TheShed/BigBang-v1-W4A16-NVFP4-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TheShed/BigBang-v1-W4A16-NVFP4-MTP with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="TheShed/BigBang-v1-W4A16-NVFP4-MTP")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("TheShed/BigBang-v1-W4A16-NVFP4-MTP")
model = AutoModelForMultimodalLM.from_pretrained("TheShed/BigBang-v1-W4A16-NVFP4-MTP", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TheShed/BigBang-v1-W4A16-NVFP4-MTP with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TheShed/BigBang-v1-W4A16-NVFP4-MTP"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TheShed/BigBang-v1-W4A16-NVFP4-MTP",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/TheShed/BigBang-v1-W4A16-NVFP4-MTP

SGLang

How to use TheShed/BigBang-v1-W4A16-NVFP4-MTP with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TheShed/BigBang-v1-W4A16-NVFP4-MTP" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TheShed/BigBang-v1-W4A16-NVFP4-MTP",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TheShed/BigBang-v1-W4A16-NVFP4-MTP" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TheShed/BigBang-v1-W4A16-NVFP4-MTP",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use TheShed/BigBang-v1-W4A16-NVFP4-MTP with Docker Model Runner:
```
docker model run hf.co/TheShed/BigBang-v1-W4A16-NVFP4-MTP
```

BigBang-v1 W4A16 NVFP4 + MTP

NVIDIA ModelOpt quantization of endless-frontier/BigBang-v1, based on Qwen3.6-35B-A3B. This checkpoint includes the native MTP module and has been tested with vLLM on one NVIDIA DGX Spark-class system.

This is a quantization of the upstream model, not an additional fine-tune.

Quantization

Component	Format
MoE and shared-expert MLP weights	W4A16 NVFP4, group size 16
Attention projections	FP8
Language-model output head	W4A16 NVFP4
Vision and other excluded modules	BF16
Native MTP module	BF16

The checkpoint was produced with NVIDIA ModelOpt 0.45.0 and is approximately 21.8 GiB across four safetensor shards. Exact layer assignments are recorded in hf_quant_config.json.

vLLM

Tested with vLLM 0.25.1, FlashInfer attention, the Marlin MoE backend, a BF16 KV cache, and Qwen reasoning/tool parsers.

vllm serve TheShed/BigBang-v1-W4A16-NVFP4-MTP \
  --trust-remote-code \
  --kv-cache-dtype bfloat16 \
  --attention-backend flashinfer \
  --moe-backend marlin \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1,"moe_backend":"triton"}' \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml

MTP is optional. To serve the target model without speculative decoding, omit the --speculative-config argument. A separate non-MTP checkpoint is not required.

Notes

The architecture supports up to 262,144 tokens. Usable context and concurrency depend on available memory and KV-cache configuration.
Although the quantization metadata advertises FP8 KV cache, BF16 KV was used for testing because calibrated FP8 KV scales are not included.
No formal quality, throughput, or multimodal comparisons against the upstream BF16 checkpoint are provided.
The upstream model's limitations and safety considerations still apply.

Attribution and license

Base model: endless-frontier/BigBang-v1
Base architecture: Qwen/Qwen3.6-35B-A3B
Quantization: NVIDIA ModelOpt

Released under the Apache License 2.0, matching the upstream model.

Downloads last month: 245

Safetensors

Model size

20B params

Tensor type

BF16

F8_E4M3

Model tree for TheShed/BigBang-v1-W4A16-NVFP4-MTP

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

endless-frontier/BigBang-v1

Quantized

(37)

this model