Qwen3.8-Flash-Next-Uncensored-GGUF

GGUF quants (2-bit → 6-bit) of the abliterated (refusal-removed) Qwen3.8-Flash-Next — for llama.cpp

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Ollama · Discord · X

---

GGUF conversions of the abliterated (refusal-removed) build of Qwen's Qwen3.8-Flash-Next — a large Mixture-of-Experts (512 experts, 10 routed + 1 shared active) preview of the Qwen4 architecture (qwen4_exp): Gated DeltaNet linear attention + Qwen Sparse Attention (QSA) at the micro-block level, HyperConnections in place of layer norms, PLE n-gram hash embeddings, native vision-language, reasoning, and tool-calling. These files run in llama.cpp (CPU / CUDA / Metal / ROCm), quantized from 2-bit to 6-bit, with a separate mmproj file that restores vision. Browse all models in the OrcaRouter Model Catalog.

---

Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). It will comply with harmful, unethical, or illegal requests the original Qwen3.8-Flash-Next would refuse. Released strictly for legitimate research — interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You assume full responsibility for how you use it and everything it generates; add your own safety and moderation layers before any deployment. Use must comply with the Apache 2.0 License inherited from the base model and all applicable law. The authors accept no liability for misuse.

---

Requirements — build a recent llama.cpp

The qwen4_exp architecture (Gated DeltaNet + QSA + HyperConnections + PLE n-gram) was merged into mainline llama.cpp on 2026-08-27 (PR #27742, model: add Qwen3.8-Flash-Next (qwen4exp), plus follow-up #27880). Any llama.cpp built from master at/after that date loads these files. Older or bundled runtimes that predate it will not — e.g. LM Studio's current build stops at qwen35moe and fails with unknown architecture 'qwen4_exp'; those will work once they bump their bundled llama.cpp. Build a current one yourself:

git clone https://github.com/ggml-org/llama.cpp/
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
  --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

(Drop -DGGML_CUDA=ON for a CPU-only build.)

Note — no MTP head. These GGUFs do not include the multi-token prediction (mtp) speculative-draft head, because llama.cpp's qwen4exp backend has no MTP inference path yet (its graph builder never wires one up, unlike the qwen35moe / qwen3next siblings that do). The draft head would just be dead weight in the file, so the converter drops it — all text + vision capability is unaffected. If you want speculative decoding today (e.g. on Apple Silicon), use the MLX build orcarouter/Qwen3.8-Flash-Next-Uncensored-MLX, which ships the mtp/ drafter.

Files

Files larger than ~48 GB are split with llama-gguf-split into multiple parts — download the whole set for that quant and point llama.cpp at the …-00001-of-000NN.gguf part; it loads the rest automatically. Sizes are approximate.

Standard K-quants

File	Bits	Size	Notes / recommendation
…-Q2_K	2-bit	~74 GB	Smallest K-quant; noticeable quality drop — low-VRAM only
…-Q3_K_S	3-bit	~78 GB
…-Q3_K_M	3-bit	~87 GB	Good small option
…-Q3_K_L	3-bit	~93 GB
…-Q4_K_S	4-bit	~103 GB
…-Q4_K_M	4-bit	~110 GB	Recommended default — best quality/size balance
…-Q5_K_S	5-bit	~117 GB	High quality
…-Q5_K_M	5-bit	~125 GB	Near-max quality; every shard <50 GB (no Xet needed)
…-Q6_K	6-bit	~168 GB	Highest fidelity offered. The PLE table falls back to Q8_0 → one ~54 GB shard, so it needs a Xet-capable downloader (default hf / recent llama.cpp)

Q6_K and higher. The model's n-gram (PLE) embedding table is a single 51.2B-parameter tensor (per_layer_token_embd, shape [160, 320001536]; 102.4 GB in BF16). Its quantized dimension (160) is not divisible by 256, so it cannot be a K-quant — at Q6_K it falls back to Q8_0 (~54.4 GB) and occupies its own shard. That shard is above CloudFront's 50 GB single-request download limit (a delivery constraint, not a storage limit), so it is served via Xet / ranged requests, which the default hf client and recent llama.cpp downloaders handle transparently — the same approach upstream Unsloth uses for its Q6_K/Q8_0/BF16 builds. Q6_K is provided above as the highest-fidelity option; Q8_0 / BF16 are omitted as impractically large — for full precision use the BF16 safetensors in orcarouter/Qwen3.8-Flash-Next-Uncensored.

IQ quants (imatrix)

Lower-bit quants built with an importance matrix (computed on English + Chinese + code calibration text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.

File	Bits	Size	Notes / recommendation
…-IQ4_XS	~4.25-bit	~97 GB	Best low-bit pick — approx Q4_K_S quality at smaller size
…-IQ3_M	~3.7-bit	~82 GB	Solid 3-bit
…-IQ3_XXS	~3.1-bit	~73 GB	Smaller 3-bit
…-IQ2_M	~2.7-bit	~63 GB	Runs in lower VRAM; some quality loss
…-IQ2_XXS	~2.1-bit	~52 GB	Smallest runnable; most degraded

Vision

File	Size	Notes
mmproj-…-F16.gguf	~0.9 GB	Vision projector — download this too for image input

Usage (llama.cpp — PR #27742 build)

Download

hf download orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF \
  --include "Qwen3.8-Flash-Next-Uncensored-Q4_K_M*" "mmproj-*" \
  --local-dir ./qwen-flashnext-uncensored

Chat (text)

./llama-cli -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf --jinja -c 8192 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 -p "Hello!"

OpenAI-compatible server (tool calling + reasoning + vision)

./llama-server -m Qwen3.8-Flash-Next-Uncensored-Q4_K_M-00001-of-00003.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf \
  --host 0.0.0.0 --port 8000 -c 8192 --jinja

• Recommended sampling: --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0.
• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Tool calling: --jinja enables the Qwen tool template; use standard OpenAI tools + tool_calls.
• Reasoning (thinking): thinking is on by default; toggle per request via chat_template_kwargs.enable_thinking. The reasoning trace is returned in reasoning_content — give max_tokens room (e.g. >= 2048) so the final answer isn't truncated by the thinking budget.

Evaluation

Abliteration was measured on this build (served with vLLM) vs the official Qwen/Qwen3.8-Flash-Next, using the same scripts. Harmful-prompt refusal collapses from 64–100% (base) to ~0–3.3%; benign over-refusal stays near 0%; capability stays within ±2 pts of the base across MMLU-Pro / GSM8K / CMMLU-style checks; vision (image + OCR via mmproj) and multi-turn tool calling verified working. GGUF quants are deterministic derivations and inherit these behaviours; lower bits trade some quality (most visible at Q2_K / IQ2).

Hardware

Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. As a large MoE, only ~10 of 512 experts are active per token, so decode is far faster than the total parameter count suggests, but the full weights must fit in RAM/VRAM (or be memory-mapped): budget ≈ the file size + KV cache + (for vision) the ~0.9 GB mmproj. Multi-GPU (-ngl split) and CPU+GPU offload both work.

License

Apache 2.0, inherited from Qwen/Qwen3.8-Flash-Next. Abliteration and quantization do not change the underlying license obligations.