Instructions to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2")
model = AutoModelForMultimodalLM.from_pretrained("Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2

SGLang

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2 with Docker Model Runner:
```
docker model run hf.co/Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2
```

Swift-Qwen3.8-27B · huihui-style abliterated · NVFP4 + DFlash 2 (built-in draft)

NVFP4 + FP8 quantized checkpoint of the abliterated Swift model, published in standard Hugging Face safetensors format, with the DFlash 2 draft model bundled in this repository. The drafter lives in dflash2/ and config.json carries the matching speculative_config block (relative dflash2 path): engines that auto-read that key activate the drafter out of the box, while vLLM currently needs the explicit flag from the Usage section.

The target weights, tokenizer and behaviour are the standard-HF NVFP4 checkpoint — the full provenance chain (abliteration → quantization → config repair) is documented in the base repository.

Layout

Path	What	Size
`model-00001-of-00002.safetensors`	target NVFP4+FP8 weights (shard 1)	19.93 GB
`model-00002-of-00002.safetensors`	target weights (shard 2)	1.72 GB
`config.json`	target config + `speculative_config` (`method: dflash`, `model: dflash2`, 7 draft tokens)	17.0 KB
`dflash2/model.safetensors`	DFlash 2 drafter, BF16, 1.92B params (Inco AI / Z Lab)	3.85 GB
`dflash2/config.json`	drafter config (`DFlash2DraftModel`, 5 layers, block size 8)	1.2 KB
`tokenizer.json`, `chat_template.jinja`, `preprocessor_config.json`, …	standard HF artifacts of the target	—
`recipe/recipe.yaml`	llm-compressor NVFP4/FP8 recipe (unsloth allocation)	16.8 KB

DFlash 2 draft

Source: z-lab/Qwen3.8-27B-DFlash2 @ rev 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 (Apache-2.0)
Block-diffusion drafter: 1.92B params, BF16 (hence 3.85 GB), 5 layers, sliding window 2048, block size 8; conditions on target layers [5, 19, 33, 47, 61]
dflash2/model.safetensors sha256 67fc76d68dc5a9415511a4f394ef744d67510cd20e93b37cc2cc7d28e4bab65c
dflash2/config.json sha256 873e3556509b0da06e29654ba00d4944888d4b5e8a33afde25f7eb27d321e980

The draft proposes up to 7 tokens per step; the target verifies the whole block in one forward pass, so the sampling distribution is exactly the target's — lossless.

Usage (vLLM)

pip install "vllm>=0.29"        # dflash2 drafter needs >= 0.29 (0.27.1 has no DFlash2DraftModel)
export VLLM_USE_FLASHINFER_SAMPLER=0
huggingface-cli download dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2   --local-dir ./model          # target + dflash2/ drafter in one folder

vllm serve ./model   --max-model-len 32768 --gpu-memory-utilization 0.9   --speculative-config '{"method": "dflash", "model": "./model/dflash2", "num_speculative_tokens": 7}'

config.json also carries the same speculative_config block (with the relative dflash2 path) so that engines which auto-load it activate the drafter out of the box; vLLM 0.29.0 currently resolves model against the working directory / HF repo ID, hence the explicit flag above.

SGLang:

python -m sglang.launch_server   --model-path ./model   --speculative-algorithm DFLASH   --speculative-draft-model-path ./model/dflash2   --speculative-num-draft-tokens 8

Verified on target

Setting	Throughput (tok/s)	vs baseline
Baseline (autoregressive)	260.3	—
DFlash 2 (7 draft tokens)	538.4	2.07×

Measured on one RTX PRO 6000 (sm_120a), vLLM 0.27.1, 8 concurrent prompts × 512 max tokens, temp 1.0 / top_p 0.95 / top_k 20, thinking xhigh; draft acceptance rate n/a. Throughput is inference-only (engine startup excluded). Greedy/sampled output is lossless vs the target.

Provenance

Source: ukisai/Swift-Qwen3.8-27b @ 1b30aaaf753fe5c1cb51ada2ea0367a53445359c
Abliteration: huihui-style (refusal-direction projection, Arditi et al. 2024 family), transferred by per-tensor weight difference from the Qwen/Qwen3.8-27B ↔ huihui-ai/Huihui-Qwen3.8-27B-abliterated pair — format/weight change, no fine-tuning
Quantization: llm-compressor oneshot, NVFP4 + FP8 allocation copied verbatim from unsloth/Qwen3.8-27B-NVFP4, 32 calibration samples
Config repair: nested multimodal config restored + quantization_config grafted (llm-compressor flattens it)
DFlash 2 draft: bundled from z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c (byte-identical, sha256 above); speculative_config added to config.json
Verification: vLLM 0.29.0 smoke (baseline vs dflash) on RTX PRO 6000, Modal

License

The target is a derivative of the Swift checkpoint — Swift Open License v1.0 (LICENSE), not Apache. The bundled DFlash 2 draft is Apache-2.0 (z-lab); the Qwen base model underneath is also Apache-2.0. Practical terms of the Swift Open License (free use, commercial included, under $1,000,000 FY gross-revenue threshold; see the base repository for the full breakdown):

Component	Licence
Qwen/Qwen3.8-27B (base model)	Apache-2.0
ukisai/Swift-Qwen3.8-27b (Swift Contribution)	Swift Open License v1.0
Target checkpoint in this repo (abliteration + quantization of Swift)	derivative work — Swift Open License v1.0
`dflash2/` draft (z-lab)	Apache-2.0

This is a description of what the licences say, not legal advice.

Downloads last month: 246

Safetensors

Model size

19B params

Tensor type

BF16

F8_E4M3

Model tree for Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-DFlash2

Base model

Qwen/Qwen3.8-27B

Finetuned

ukisai/Swift-Qwen3.8-27b

Quantized

(60)

this model