Nex-N2.5-mini-Uncensored-GGUF

imatrix GGUF quants (2-bit → 8-bit) of the abliterated (refusal-removed) Nex-N2.5-mini — for llama.cpp

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

---

GGUF conversions of the abliterated (refusal-removed) build of Nex-AGI's Nex-N2.5-mini — a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context. These files run in llama.cpp (CPU / CUDA / Metal / ROCm), quantized from 2-bit to 8-bit with an importance matrix, with a separate mmproj file that restores vision. Browse all models in the OrcaRouter Model Catalog.

Derived releases:  •  Nex-N2.5-mini-Uncensored (BF16 source)  •  …-FP8 (block-FP8, mirrors Qwen's scheme)  •  …-NVFP4 (experts-only NVFP4, for vLLM)  •  …-MLX (2 / 3 / 4 / 6 / 8-bit, Apple Silicon)  •  …-GGUF (this repo).

---

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Nex-N2.5-mini would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Nex-AGI.

By downloading or using this model you acknowledge and accept the above.

---

Requirements — a recent llama.cpp

The qwen35moe (Qwen3.5-MoE) architecture is in mainline llama.cpp as of September 2026, including the vision path. An older runtime will fail with unknown architecture 'qwen35moe'.

git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j \
  --target llama-cli llama-server llama-mtmd-cli llama-quantize

(Drop -DGGML_CUDA=ON for a CPU-only build.)

Note — no MTP head. Upstream nex-agi/Nex-N2.5-mini ships zero mtp.* tensors — Nex never released a multi-token-prediction head for this model — so these GGUFs have none and MTP speculative decoding is unavailable. Nothing was dropped in conversion. If you convert from the HF weights yourself, --no-mtp is mandatory: the config still declares mtp_num_hidden_layers: 1, so without the flag the converter writes block_count = 41 for 40 layers of weights and every llama.cpp tool then fails with tensor 'blk.40.attn_norm.weight' not found. Setting the config field to 0 does not work either — the converter asserts against it.

Files

File	Bits	Size	Notes / recommendation
…-Q2_K.gguf	2-bit	12.94 GB	Smallest; real quality drop — low-VRAM only
…-Q3_K_M.gguf	3-bit	16.76 GB	Good small option
…-IQ4_XS.gguf	4-bit	18.73 GB	Recommended default — best quality/size balance
…-Q6_K.gguf	6-bit	28.51 GB	Within noise of the source
…-Q8_0.gguf	8-bit	36.90 GB	Highest fidelity offered
mmproj-…-F16.gguf	—	0.90 GB	Vision projector — download this too for image input
imatrix.dat	—	0.19 GB	The importance matrix these were built with, for re-quantizing

No file exceeds the Hub's split threshold, so each tier is a single .gguf.

IQ4_XS is the 4-bit tier, not Q4_K_M, which lands at ~4.85 bpw here (about 21.3 GB) and misses a 20 GB target; with an imatrix, IQ4_XS at 4.32 bpw is the better-quality option at that size anyway.

Every tier was quantized directly from BF16 — no Q8_0 intermediate — with an imatrix from a 1200-chunk corpus interleaving English prose, Chinese and real source code in even thirds. Interleaving matters because llama.cpp uses the first --chunks worth of tokens; so does length, on a 256-expert MoE: the expert tensors report 95–98 % "partial data" at 8 chunks but a 99.2 % median at 1200 (252 of 369 tensors ≥ 99 %). The few experts the corpus never routes to fall back to default weighting.

For full precision, use the safetensors builds in …-FP8 or …-NVFP4.

Usage (llama.cpp)

Download

hf download orcarouter/Nex-N2.5-mini-Uncensored-GGUF \
  --include "*IQ4_XS.gguf" "mmproj-*" --local-dir ./nex-n25-mini-uncensored

Chat (text)

./llama-cli -m Nex-N2.5-mini-Uncensored-IQ4_XS.gguf -ngl 999 -c 8192 --jinja \
  -p "Explain gated delta-net attention in two sentences."

Vision

./llama-mtmd-cli -m Nex-N2.5-mini-Uncensored-IQ4_XS.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Uncensored-F16.gguf \
  -ngl 999 --image screenshot.png -p "What is on screen?"

OpenAI-compatible server

./llama-server -m Nex-N2.5-mini-Uncensored-IQ4_XS.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Uncensored-F16.gguf \
  --host 0.0.0.0 --port 8000 -ngl 999 -c 16384 --jinja

• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Reasoning: the chat template gates its <think> block on reasoning_effort, not enable_thinking — send "chat_template_kwargs": {"reasoning_effort": "none"} for direct answers, and otherwise give generation enough budget to reach </think> or the answer is truncated inside the scratchpad.
• Stop tokens: neither this build nor upstream ships a generation_config.json. With --jinja llama.cpp takes EOS from the GGUF metadata; if you drive /completion directly, stop on <|im_end|> yourself.

Evaluation — quality vs the source

Perplexity, KL-divergence and top-1 token agreement of each quant against the BF16 GGUF of the same checkpoint, run through the identical llama.cpp forward so the quantization is the only variable. Measured on wikitext-2, 12 chunks × 2048 tokens. Lower KLD and higher Top-1 = closer to the source.

Build	Size	bpw	PPL	ΔPPL	Mean KLD	Top-1 agreement
BF16 (ref)	69.4 GB	16.01	6.704	—	—	100 %
Q8_0	36.90 GB	8.50	6.710	+0.13 %	0.027	94.2 %
Q6_K	28.51 GB	6.56	6.717	+0.24 %	0.032	93.6 %
IQ4_XS	18.73 GB	4.32	6.802	+1.51 %	0.074	89.0 %
Q3_K_M	16.76 GB	3.91	7.042	+5.09 %	0.150	84.5 %
Q2_K	12.94 GB	3.02	7.550	+12.66 %	0.266	79.0 %

All three signals agree on the ordering and on where the cliff is: Q8_0 and Q6_K are within noise of the source, IQ4_XS is the sweet spot at 27 % of the BF16 size for +1.5 % perplexity, Q3_K_M degrades gently, and Q2_K costs a lot — pick it for fit, not quality.

Top-1 agreement is lower than these bit-widths would give on a dense model, and that is the architecture rather than the quantizer: with 256 fine-grained experts and top-8 routing, a small perturbation of the hidden state flips which experts a token is routed to, and expert selection is a discrete function. PPL, which does not depend on argmax, moves 0.13 % at Q8_0.

Uncensoring

Abliteration is baked into the weights and inherited by every quant — but on this architecture the refusal direction lives almost entirely in the routed-expert down_proj matrices, the very weights these tiers quantize, so it was measured on each tier's own bytes. On JailbreakBench (100 harmful + 100 benign prompts, greedy, reasoning_effort=none, served from these exact files): explicit refusal is 0.000 at every tier, down to Q2_K, and benign over-refusal is 0.000 at every tier. Refusal is reduced, not removed — where the model resists it names the harm and answers a different, safe question rather than declining outright (0.08–0.13 across tiers against 0.15 for the BF16 source, which is sampling noise at n=100, not a trend). See the base …-FP8 card for the full method and scoring caveats.

Hardware

Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. As an MoE, only 8 of 256 routed experts are active per token, so decode is far faster than 35 B suggests, but the full weights must fit in RAM/VRAM (or be memory-mapped): budget ≈ the file size + KV cache + (for vision) the ~0.9 GB mmproj. IQ4_XS fits a single 24 GB card; Q8_0 needs 40 GB or a multi-GPU -ngl split. Only 10 of 40 layers use full attention — the rest are gated delta-net — so long-context KV growth is modest for a model this size.

License

Apache 2.0, inherited from nex-agi/Nex-N2.5-mini. Abliteration and quantization do not change the underlying license obligations.