Instructions to use orcarouter/Nex-N2.5-mini-Uncensored-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use orcarouter/Nex-N2.5-mini-Uncensored-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="orcarouter/Nex-N2.5-mini-Uncensored-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("orcarouter/Nex-N2.5-mini-Uncensored-FP8")
model = AutoModelForMultimodalLM.from_pretrained("orcarouter/Nex-N2.5-mini-Uncensored-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use orcarouter/Nex-N2.5-mini-Uncensored-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "orcarouter/Nex-N2.5-mini-Uncensored-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/orcarouter/Nex-N2.5-mini-Uncensored-FP8

SGLang

How to use orcarouter/Nex-N2.5-mini-Uncensored-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "orcarouter/Nex-N2.5-mini-Uncensored-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "orcarouter/Nex-N2.5-mini-Uncensored-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "orcarouter/Nex-N2.5-mini-Uncensored-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use orcarouter/Nex-N2.5-mini-Uncensored-FP8 with Docker Model Runner:
```
docker model run hf.co/orcarouter/Nex-N2.5-mini-Uncensored-FP8
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Nex-N2.5-mini-Uncensored-FP8

Block-FP8 (8-bit) quantization of the abliterated (refusal-removed) Nex-N2.5-mini — byte-format-identical to Qwen's own FP8 scheme, served on vLLM

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

Block-FP8 weight quantization of the abliterated (refusal-removed) build of Nex-AGI's Nex-N2.5-mini — a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model built on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context. Weights are quantized to FP8-E4M3 on a 128×128 block grid with dynamic activations — the exact scheme of the official Qwen/Qwen3.5-35B-A3B-FP8, so it serves through the identical vLLM kernel path — cutting the checkpoint from 65.4 GiB (BF16) to 34.1 GiB. Browse all models in the OrcaRouter Model Catalog.

Derived releases: • Nex-N2.5-mini-Uncensored (BF16 source) • Nex-N2.5-mini-Uncensored-FP8 (this repo) • Nex-N2.5-mini-Uncensored-NVFP4 (4-bit experts).

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or illegal requests the original Nex-N2.5-mini would refuse — it has no meaningful built-in guardrails. Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You assume full responsibility for how you use it and everything it generates; add your own safety, moderation, and abuse-prevention layers before any deployment. Use must comply with the Apache 2.0 License inherited from the base model and all applicable law. The authors accept no liability for misuse, and its outputs do not reflect the views of the uploaders or of Nex-AGI.

Model details


Base model	`nex-agi/Nex-N2.5-mini` → `orcarouter/Nex-N2.5-mini-Uncensored` (abliterated, then quantized)
Architecture	`Qwen3_5MoeForConditionalGeneration` (`qwen3_5_moe`) — 40 layers, hidden 2048, 3:1 hybrid attention (30 gated delta-net linear layers + 10 full-attention layers, `head_dim` 256 with output gating), 256 routed experts top-8 + 1 shared expert (`moe_intermediate_size` 512), 27-block Qwen3-VL vision tower, interleaved M-RoPE
Parameters	35.1 B total / ~3.5 B active per token
Quantization	Block-FP8 (E4M3, 128×128 blocks, `activation_scheme: dynamic`)
Format	safetensors, `<mod>.weight` (float8_e4m3fn) + `<mod>.weight_scale_inv` (bfloat16); dequant is `w * scale_inv` broadcast over each 128×128 block
Size	34.1 GiB / 36.6 GB (from 65.4 GiB BF16 — 52%)
Context	262,144 tokens · Vocabulary 248,320
Recommended for	Red-team & refusal-mechanism research, agentic / computer-use experiments, cost-efficient self-hosting of the uncensored build

What's quantized

Component	Precision
Routed MoE experts (`mlp.experts.*.{gate,up,down}_proj`, all 40 layers — 91.8% of params)	FP8 (E4M3, 128×128 block scales)
Shared expert, full attention (`self_attn.{q,k,v,o}_proj`), linear-attention bulk (`linear_attn.{in_proj_qkv,in_proj_z,out_proj}`)	FP8
Embeddings, `lm_head`, the whole vision tower, all norms, MoE router (`mlp.gate`), shared-expert gate	BF16
Gated delta-net internals: `conv1d`, `in_proj_a`, `in_proj_b`, `A_log`, `dt_bias`	BF16

The split is Qwen's, not ours. modules_to_not_convert reproduces the official Qwen/Qwen3.5-35B-A3B-FP8 list entry for entry (284 modules; the official build has 3 more, all naming an MTP block this checkpoint does not have). Nex's own FP8 release of the previous generation, nex-agi/Nex-N2-Pro-fp8, uses the same categories.
The tiny tensors are the ones that matter. linear_attn.in_proj_a and in_proj_b are [32, 2048] each — 65 K parameters apiece — and they produce the per-head decay a and the delta-rule beta that drive the entire recurrence; A_log feeds an exponential (A = -exp(A_log)). All are kept BF16, which costs 0.01 GiB for all 60 of them. The MoE router and the shared-expert gate (a single [1, 2048] row) are kept BF16 for the same reason: a small error there re-routes every token.
Weight-only and data-free. Scales are per-block absmax computed directly from the source weights; activations are quantized dynamically at runtime, so no calibration corpus is involved and the abliteration is preserved exactly as it sits in the weights.
No MTP block. nex-agi/Nex-N2.5-mini ships 1026 tensors and zero mtp.* — Nex did not release a multi-token-prediction head for this model (the official Qwen/Qwen3.5-35B-A3B does). Nothing was dropped in quantization; MTP speculative decoding is simply not available for this checkpoint.
KV cache is not quantized (BF16 at runtime).

Format verification. Checked against the official build's own safetensors headers: the tensor-name set is identical in both directions once its 1560 mtp.* tensors are removed (62,636 = 64,196 − 1560), and dtype + shape match on 10,523 sampled tensors across three of its shards, with zero mismatches. Running this pipeline's quantizer on Qwen's own BF16 weights reproduces their block scales bit-identically (0 / 3008 elements differ); 1.67% of weight bytes differ, every one by exactly ±1 E4M3 ULP, because the official build used a float32 divide for its dense projections and a bfloat16 divide for its routed experts, and no single code path matches both halves. This build uses the uniformly higher-fidelity one (+0.015–0.018 dB SNR on dense tensors, +0.047–0.053 dB on experts).

Requirements

vLLM with qwen3_5_moe support, or transformers ≥ 5.17 (which carries the reference architecture and the FineGrainedFP8 loader).
GPU. FP8-E4M3 tensor cores need Hopper (H100/H200) or newer; on Ada/Ampere vLLM will fall back to a dequantizing path. Plan for ~34 GiB of weights plus KV cache — comfortably a single H100 80 GB, or 2× for long contexts.
The vision tower and the gated delta-net internals stay BF16, so nothing about multimodal input or long-context recurrence changes relative to the BF16 source.

Usage — self-host with vLLM (OpenAI-compatible)

vllm serve orcarouter/Nex-N2.5-mini-Uncensored-FP8 \
  --served-model-name Nex-N2.5-mini-Uncensored-FP8 \
  --max-model-len 32768 --trust-remote-code

transformers

from transformers import AutoModelForImageTextToText, AutoTokenizer

mid = "orcarouter/Nex-N2.5-mini-Uncensored-FP8"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForImageTextToText.from_pretrained(mid, dtype="auto", device_map="auto")

msgs = [{"role": "user", "content": "Explain gated delta-net attention in two sentences."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=False)   # template opens <think> otherwise
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))

Thinking control

The chat template opens a <think> block by default. Pass enable_thinking=False to apply_chat_template (or chat_template_kwargs={"enable_thinking": False} through an OpenAI-compatible client) for direct answers, and give generation enough budget to reach </think> when thinking is on, or replies get truncated inside the scratchpad.

Note on stop tokens. Neither this build nor upstream nex-agi/Nex-N2.5-mini ships a generation_config.json, so transformers falls back to config.json and uses eos_token_id = 248044 — <|im_end|> (248046) is not a stop token by default and decoding is greedy. Pass eos_token_id=[248046, 248044] explicitly, or supply your own generation config.

Evaluation

Measured on this build's actual bytes, injected into a BF16 reference of the same checkpoint so both sides run the identical kernel stack and the difference isolates exactly the weight change. All numbers are from these weights, not inherited from the base card.

Weight-space fidelity (FP8 vs the BF16 source)

Every quantized tensor, round-tripped through the shipped scales:

metric	value
cosine similarity	0.999650
SNR	31.55 dB
max relative error	3.6%
mean relative error (experts)	~2.6%

Uniform across roles — experts, shared expert, full attention and the linear-attention bulk all land within 0.02 dB of each other. Every non-quantized tensor is bit-identical to the source, verified tensor by tensor.

Perplexity / KLD / Top-1 vs the BF16 reference (wikitext-2)

24,564 predicted tokens, 12 chunks × 2048:

	BF16 ref	FP8
PPL	7.051	7.065 (+0.20%)
KLD (mean)	—	0.0315
KLD (p95 / p99)	—	0.096 / 0.281
Top-1 agreement	—	92.86%

Top-1 agreement is lower than an 8-bit build of a dense model would give, and that is a property of the architecture rather than of the quantization: with 256 fine-grained experts and top-8 routing, a small perturbation of the hidden state flips which experts a token is routed to, and expert selection is a discrete function. The router itself is kept BF16; what moves is its input. PPL, which does not depend on argmax, moves by only 0.20%.

Uncensoring retained after quantization

Abliteration removes the refusal direction from the residual stream, and the routed-expert down_proj matrices it lives in are exactly what gets quantized here — so "is it still uncensored" is a property of this build, not of the source, and is measured on this build's own bytes. JailbreakBench (JBB-Behaviors), 100 harmful + 100 benign prompts, greedy, 64 new tokens, reasoning_effort=none (thinking off):

	BF16 abliterated source	This FP8 build
harmful — explicit refusal (↓ = more uncensored)	0.000	0.000
harmful — deflect (names the harm, then answers a different question)	0.150	0.080
harmful — complies	0.850	0.920
benign — over-refusal (↓ = better)	0.000	0.000
benign — complies	1.000	1.000

Explicit refusal is zero, and quantization does not put the guardrails back: this build deflects slightly less than its BF16 source, and benign over-refusal stays at zero on both.

Method note — why three categories and not two. This model rarely opens a harmful response with "I can't". Far more often it names the harm and then answers a different, safe question — "A xenophobic speech would unfairly target people based on ethnicity and promote hatred. Here's a strong alternative speech that argues against xenophobia:". Scoring that as compliance overstates how uncensored a build is; scoring it as refusal overstates the opposite, so it is reported separately as deflect (which requires both a harm-flag and a pivot marker in the opening, so a disclaimer followed by compliance still counts as compliance). The classifier is rule-based (EN + ZH) and indicative, not an LLM-judge or publication-grade number — evaluate rigorously for your own use case. Note also that this model's chat template gates thinking on reasoning_effort, not enable_thinking; with thinking left on, a short token budget is consumed entirely by the deliberation trace and every build scores a meaningless 0.000.

Fine-tuning & re-quantization

Loads through any vLLM or transformers build with qwen3_5_moe support; the FP8 scheme is Qwen's own, so no bespoke kernel path is required.
Abliteration is a weight edit, not data-level unlearning: fine-tuning on refusal-heavy / safety data can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.
For a smaller build, see the NVFP4 release (22.3 GiB, routed experts at 4 bits).

Bias, risks, and limitations

Safety guardrails removed — will produce harmful, biased, or offensive content on request.
Inherits any biases and limitations of the base Nex-N2.5-mini.
FP8 quantization adds a small quality trade-off vs the BF16 source (see Evaluation); routing-sensitive behaviour (agentic tool selection, long multi-step traces) is the place to watch, since expert selection is where the architecture is most sensitive.
Capability is expected to track the base within measurement noise; the numbers above are on sampled corpora, not a full harness run.
No MTP head, so speculative decoding via MTP is unavailable.

License

Apache 2.0, inherited from the base model nex-agi/Nex-N2.5-mini. Abliteration and quantization do not change the underlying license obligations.