Instructions to use btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16")
model = AutoModelForMultimodalLM.from_pretrained("btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16

SGLang

How to use btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16 with Docker Model Runner:
```
docker model run hf.co/btbtyler09/Qwen3.8-Flash-Next-NVFP4-LIL-PLEbf16
```

btbtyler09 commited on Oct 9

Commit

4fe9233

verified ·

1 Parent(s): d92663a

LIL NVFP4 with BF16 PLE tables (dequantized from LIL's own NVFP4) (part 2)

Browse files

Files changed (21) hide show

.gitattributes +1 -0
model-00010-of-00041.safetensors +3 -0
model-00011-of-00041.safetensors +3 -0
model-00012-of-00041.safetensors +3 -0
model-00013-of-00041.safetensors +3 -0
model-00014-of-00041.safetensors +3 -0
model-00015-of-00041.safetensors +3 -0
model-00016-of-00041.safetensors +3 -0
model-00017-of-00041.safetensors +3 -0
model-00018-of-00041.safetensors +3 -0
model-00019-of-00041.safetensors +3 -0
model-00020-of-00041.safetensors +3 -0
model-00021-of-00041.safetensors +3 -0
model-00022-of-00041.safetensors +3 -0
model-00023-of-00041.safetensors +3 -0
model-00024-of-00041.safetensors +3 -0
model-00025-of-00041.safetensors +3 -0
model-00026-of-00041.safetensors +3 -0
model.safetensors.index.json +3 -0
ple-bf16-00.safetensors +3 -0
ple-bf16-01.safetensors +3 -0

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b5b935561f39184f2be07a0bab4902caff01135aae75a13c12f7c5c5583de32
+size 4248647096

+version https://git-lfs.github.com/spec/v1
+oid sha256:699749bfe7b369cdc1d8651efaa1c39852927bfe95669923a040ff5741097803
+size 2609490248

+version https://git-lfs.github.com/spec/v1
+oid sha256:6c658ba6a1c7988664cbdc09a88310416670795c407dc2b9c628f6671e98fa90
+size 591938104

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f815e784dedba1d7ae62dad023d79d59b0a1035f9fd33d7c140e27e7c551a02
+size 360698832

+version https://git-lfs.github.com/spec/v1
+oid sha256:724e104ba8e24134270e39ae06e37cf0654240269d47977ef75c9bf8ee64f08e
+size 128587568

+version https://git-lfs.github.com/spec/v1
+oid sha256:0d0ec7505ca780c66d6f1c2fbf90211a77731acf263c2c388edbf5c1de3c49c6
+size 9669968

+version https://git-lfs.github.com/spec/v1
+oid sha256:77de96023b11762f626f05a5a100f7e7113209e7cd38854a023a9023158b4b0a
+size 1652473080

+version https://git-lfs.github.com/spec/v1
+oid sha256:95aacd5f0f446ae17269ec34fe8e71df3f6ac3f6d4813110ed833482e0b668bf
+size 123264

+version https://git-lfs.github.com/spec/v1
+oid sha256:536aa18a2789deb47a8c1599cbcc896490faf4de84d5373c283f6886c23a6a90
+size 33249830

+version https://git-lfs.github.com/spec/v1
+oid sha256:2f3924a833565463d9ea87998d94b710577118d64bb37be02ec03469c3faf605
+size 2400012032

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@