GLM-5.3-Flash-FP4-Abliterated

NVIDIA's nvidia/GLM-5.3-Flash-FP4-Local (revision 76348ddb2be2e35e39ecfc00e75dc07703f4f3d5) with the Huihui refusal-direction edit re-applied to its own weights and stored in its own formats. It answers where the original refuses or deflects, with the same speed and, in our tests, the same quality.

Everything except 44 matrices is byte-identical to NVIDIA's checkpoint: 40 of the 42 weight files are unchanged, the two others differ only inside the 44 edited tensors (checked byte by byte outside them). repack-manifest.json lists every changed matrix with its measurements and the SHA-256 of both rewritten files.

What was changed

• Matrices: self_attn.o_proj and mlp.shared_experts.down_proj in layers 15 to 36, the same 44 projections the Huihui abliteration of GLM-5.3-Flash edits (edp1096/Huihui-GLM-5.3-Flash-abliterated-NVFP4, MIT).
• Why not just add Huihui's delta: NVIDIA trained this checkpoint further with quantization-aware distillation, so its weights are no longer the originals the delta was computed against. The edit is a rank-1 change along one output direction, so it is re-applied as such: T = D - s · r (rᵀ D), where D is this checkpoint's dequantized weight, r the leading left singular vector of (Huihui-edited − unedited), and s the strength the original edit had on the original weight (about 2.4).
• Formats kept exactly: 28 of the matrices are E4M3 with an FP32 scale per 128×128 block, 16 are MXFP8 (E4M3 with an E8M0 scale per 32 inputs, scale = 2^ceil(log2(amax/448))). Before writing, the encoder was required to reproduce NVIDIA's own codes and scales from NVIDIA's own dequantized weights for all 44 matrices; it did, with zero differences. The routed experts (NVFP4 with compressed scales) are not touched.
• After re-encoding, 91 to 95 % of the edit's strength remains (the rounding of the 8-bit formats), comparable to earlier FP8 builds of the same edit.

Tests

Both variants on 2× RTX PRO 6000 Blackwell (300 W each), Local Inference Lab vLLM release-10152026, TP2 with decode context parallel 2, MTP 3, 8 sequences, --max-model-len 262144 (FP8 KV cache of 1,107,332 tokens), same harness and prompts for both.

	NVIDIA FP4-Local	This model
Functional checks (tool calls single / parallel / none / follow-up, JSON, vision, German, effort low/high/max, length stop)	11 / 11	11 / 11
GSM8K, 96-problem sample, thinking high	94 / 96	95 / 96
Refusal probe, 12 prompts (Tiananmen, Taiwan, criticism of Xi, Xinjiang, adult fiction, profanity, lock picking, paywalls, fake review, fictional ransom note, offensive joke, card counting)	2 refused, 3 answered with the state line	0 refused, 0 deflected
Decode tok/s, prose, 1 / 3 / 6 / 8 requests	176 / 271 / 353 / 409	174 / 276 / 356 / 413
Decode tok/s, code, 1 / 3 / 6 / 8 requests	211 / 324 / 420 / 490	211 / 323 / 420 / 478
240K-token prompt, three identifiers recalled	yes (91 s)	yes (91 s)
Three 240K prompts at once / six 200K prompts at once	all recalled	all recalled

This is a small test set, not a benchmark suite. The refusal probe is meant to show the edit works, not to rank models.

Running it

The NVFP4 scale factors use the lossless compressed format of the original checkpoint, so this needs the same containers NVIDIA lists for the original; upstream vLLM cannot read it yet. Use NVIDIA's command for 2× RTX PRO 6000 from the original model card with this repository (or a local copy) as the model. We serve it with --max-model-len 1048576 (the model's native length), --max-num-seqs 8, --decode-context-parallel-size 2, --kv-cache-memory-bytes 7000000000 and MTP 3.

The KV cache gets more room from the larger --max-model-len: with the same 7 GB per card, vLLM reports 1,687,708 tokens at --max-model-len 1048576 and 1,107,332 at 262144. We take that room for many parallel chats and still limit a single request to 262,144 tokens in front of vLLM; the long-context checks above went up to 240K, and single requests beyond 262,144 tokens were not tested with this build. The functional checks pass 11 / 11 in this layout as well.

Responsibility

The refusal behaviour of the original has been removed on purpose. You are responsible for how you use the model and for what it produces; apply the safeguards your use case needs.

License

Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement (see LICENSE). The base model GLM-5.3-Flash by Z.ai is MIT-licensed; the Huihui edit it builds on is MIT-licensed. Changes: the 44 matrices described above.