Instructions to use DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1")
model = AutoModelForMultimodalLM.from_pretrained("DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1

SGLang

How to use DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 with Docker Model Runner:
```
docker model run hf.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1
```

Mtp quality? 🤔

by LinkuStarto - opened Aug 16

Discussion

LinkuStarto

Aug 16

would like to know if you could modify the default MTP drafter somehow. The current drafter accepts only max 1 efficiently... Anything above just slows it down into oblivion... Actually this model needs Dspark or Dflash badly ! Especially considering how crazy long his thinking can get ! I tried some generic Dspark / DFlash drafters but they were either incompatible or made the model even slower. After experiencing Muse Glimmer insane speeds, I can't really find myself using Qwen3.8 for longer workflows.

Do you think we might see some improvement in that field at some time?

DavidAU

Owner Aug 16

My test copies are working here correctly with 2 tokens, and 51-60% acceptance.

MTP:
Temp must be 1 or below 1 , but not too low as you will get loops and other issues.
Rep pen must be 1 [off].

Temp above 1 or rep pen on [even a little] wipes out MTP benefits.
Also ; more creative prompts -> MTP (2 tokens) will drop to SUB "non MTP" speeds.

PotatoOcelot

Aug 16

Mtp2 for General reasoning, mtp4 for coding at 0.7 temp.

DavidAU

Owner Aug 18

Hey all:

GGUFS released:

https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF

Source released too;
Thank you for your patience and support.

windowsxp811203

Aug 23

•

edited Aug 24

@LinkuStarto — an update on what you asked for, because the landscape changed twice since 2026-08-16 and the honest answer isn't "good news".

What now exists: incoai/Qwen3.8-27B-DFlash2 (mirror: z-lab/Qwen3.8-27B-DFlash2), published 08-19 — a 3.8 GB block-diffusion drafter for Qwen3.8-27B, drafting a whole block per pass with a path selector, lossless by construction. And RadixArk/Qwen3.8-27B-DSpark, a SpecForge-trained DSpark drafter. Exactly the class of thing you asked about, three days after you asked.

Why I'd still hold off for your use case:

Measured head-to-head, the built-in MTP head wins on quantized targets. In RadixArk's own discussion #3, josch15366 reports NVFP4 target: MTP n=3 87.2% acceptance vs DSpark k=7 62.4% (−24.8 pp); FP8 target: 83.4% vs 65.0%. His note: "the high acceptance rates in some posts are caused by simple prompts. Breaks down with real data." A second reporter on SGLang saw DSpark steady-state slightly below EAGLE, plus a ~12× cold-start collapse on the first few requests.
These drafters are trained against base Qwen3.8 hidden states. Cold-Fusion is a different set of weights; transfer is unmeasured, and the same thread shows rank/hidden-state mismatches costing double-digit acceptance points even with an identical tokenizer.
DFlash2 currently breaks vision in llama.cpp. With --spec-type draft-dflash plus an mmproj, image requests hard-fail (llama.cpp #27408/#26108); with the community zero-fill patch they succeed but the drafter never engages on image input. The built-in MTP head handles mmproj fine on the same build — and Cold-Fusion has a vision tower.
Stack status as of today: llama.cpp DFlash2 support is still an open PR (#27342); DSpark is merged per-architecture. vLLM merged DFlash2 on 08-21 (#52816), which also means the install instructions on incoai's card (pointing at a PR branch) are stale.

So for llama.cpp today your realistic options are the built-in MTP head with the settings DavidAU posted above (temp ≤ 1, rep-pen off), or ngram speculation stacked on top of it — veldierin's config earlier in #3 on the GGUF repo does exactly that, --spec-type draft-mtp,ngram-mod,ngram-map-k4v, and reports 70–85% acceptance in real coding sessions.

If you do try DFlash2 or DSpark against Cold-Fusion on vLLM, the number worth posting back is draft acceptance at your normal sampling settings, not tokens per second — acceptance tells you whether the drafter understands this checkpoint; t/s mixes that up with batch size and context length. And note all of these fade at high batch: in incoai's own batch-32 numbers on one H200, GSM8K still shows MTP 1.04× / DSpark 1.13× / DFlash2 1.45×, while the other four tasks drop to or below 1.0× for MTP and DSpark with DFlash2 holding a modest edge — "degrading gracefully", not "fast at scale".

(Edited: the batch-32 sentence originally overstated the drop — GSM8K stays slightly above 1× for MTP and DSpark in incoai's table.)

veldierin

Aug 23

Isn't it also an issue at higher context sizes where dflash breaks down as well? So for those of us doing coding work, after 130k or some ceiling, MTP keeps going where dflash falls apart?

i try to do 200-262k context if/when i can, just so i can retain enough context to work through a lot of code, and mtp is amazing for me with the ngram side of it.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment