Instructions to use drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45")
model = AutoModelForMultimodalLM.from_pretrained("drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45

SGLang

How to use drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 with Docker Model Runner:
```
docker model run hf.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45
```

Acknowledge the Responsible Use Agreement to access this repository

Access is granted automatically after you agree to the terms below and submit the form.

Responsible Use Agreement

This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.

Prohibited uses (you must agree before access is granted):

Anything involving the sexual exploitation or endangerment of minors.
You must be of age 18 years or older to use and download this model.
You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
Content promoting self-harm or suicide.
Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
Any use prohibited by the upstream Z.AI / GLM MIT license.

You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream Z.AI GLM-5.3-Flash MIT license — review and comply with it before use or redistribution.

keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45

Abliterated RedHatAI GLM-5.3-Flash NVFP4 (compressed-tensors). RedHat body, Dealign o_proj on L15–43 + MTP L45, L0–14 stock, L44 stock (anchor-safe). Repo id: ablit-l15-43-mtp-l45. Previous ids redirect.

GitHub (method + scripts): drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 · METHOD.md.

See RESPONSIBLE_USE.md and the gate form above. Access is gated with automatic approval after you agree.


HF	https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45
GitHub (method + scripts)	https://github.com/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45
Keys 0731 ancestor	HF anchored-tensors · GitHub 1M recipe
Stock NVFP4 (this parent)	`RedHatAI/GLM-5.3-Flash-NVFP4`
Format guide (not shipped)	`orcarouter/GLM-5.3-Flash-Uncensored-NVFP4`
Ablit source (`o_proj`)	`dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4`
Upstream	`zai-org/GLM-5.3-Flash` / BF16
Direction reference (not shipped)	`Blackfrost-Research/GLM-5.3-Flash-DERISKED-NVFP4`
Ablit	L15–43 + L45 `self_attn.o_proj` byte-copy · 30 tensors · L44 stock
Anchors	L0–14 stock (0731) · L44 stock (anchor-safe) · experts / vision / QKV RedHat stock
Gate	32/32 bypass, 0 refuse, 0 garble
Layout	10 shards + `model_mtp.safetensors`, `quant_method=compressed-tensors`

Preferred method: 0731 safety-anchors (enhance, don't hinder)

Keys learned this on DeepSeek-V4-Flash 0731. Projecting residual writes through early layers made the target stop refusing and made the stock drafter keep proposing refusal-shaped tokens.

Spare early layers. That is still our default. On GLM-5.3-Flash the in-checkpoint drafter is MTP layers.45, and residual refusal lived late + MTP. Rank-1 projection of Blackfrost’s V (nearly orthogonal to Dealign, cos ≈ 0.09) never cleared 9/32, even widened to L2–45.

The previous publish of this same repo id was a LibertAI ModelOpt 120-shard body with Dealign o_proj including L44. Two changes in this drop:

Parent is RedHat compressed-tensors (Tony: 0/0/0 U+FFFD on GB10 vs ModelOpt 4–9). OrcaRouter Uncensored NVFP4 is the format guide (experts-only NVFP4, attention BF16). We keep RedHat’s MTP; we do not take OrcaRouter’s 62-shard no-MTP tree.
L44 stays RedHat stock. Dealign L44 is Δrel 0.74 (4–6× every other layer) and is the garble risk on a clean RedHat body. MTP L45 is still transplanted.

	Blackfrost DERISKED-NVFP4 (reference)	Dealign / previous keys (LibertAI body)	OrcaRouter Uncensored NVFP4 (guide)	This checkpoint
Stock quant	Their NVFP4 of derisked BF16	LibertAI ModelOpt 120-shard	compressed-tensors, no MTP	RedHat compressed-tensors
Edited layers	`skip_early=2` → L2–44 `o_proj`	L15–45 `o_proj` (incl. L44)	full uncensored swap	L15–43 + L45 `o_proj` (30 tensors)
L44	edited	Dealign Δrel 0.74	n/a	RedHat stock (anchor-safe)
MTP L45	BF16 exact passthrough (~10 GiB extra)	Dealign `o_proj`	dropped	RedHat NVFP4 experts, Dealign `o_proj`
Refusal32	not our gate	32/32 (previous publish)	their card	32/32 bypass, 0 refuse, 0 garble

Abliteration does not change FLOPs. The intended win is direct completions instead of refuse/hedge loops, without soup.

Credit: RedHat (stock NVFP4 we actually edit)

RedHatAI/GLM-5.3-Flash-NVFP4 is the parent: LLM-Compressor compressed-tensors NVFP4 on routed-expert FFNs, attention / vision / embeddings BF16, ~193 GiB, 10 shards + MTP. Experts, vision, QKV, embeddings, L0–14 o_proj, and L44 o_proj remain theirs.

Credit: OrcaRouter (format guide)

orcarouter/GLM-5.3-Flash-Uncensored-NVFP4 — compressed-tensors experts-only NVFP4, attention BF16. We used it as a guide for the quant split. We did not ship their checkpoint (they drop MTP).

Credit: Dealign (the o_proj we copied)

Full credit to dealignai / @dealignai (compute @jordanschenck) for GLM-5.3-Flash-UNCENSORED-NVFP4.

We byte-copied BF16 self_attn.o_proj for L15–43 and MTP L45 onto RedHat shards. L44 is not copied. We did not ship their full checkpoint as a swap.

Credit: Blackfrost (direction we measured and did not ship)

Full credit to Blackfrost / @Blackfrost_AI for publishing GLM-5.3-Flash-DERISKED-NVFP4 and the BF16 master it comes from.

We recovered their rank-1 residual V by SVD of (Blackfrost − LibertAI) o_proj. It is nearly orthogonal to Dealign’s write (cos ≈ 0.09). Projecting it on L10–35, L2–35, or L2–45 never passed 9/32. Credited as a measured reference. Do not treat this repo as a Blackfrost build.

Credit: Z.ai and the Spark vLLM recipe

Z.ai / zai-org — GLM-5.3-Flash (Glm5NextForConditionalGeneration, hybrid KDA+DSA, mHC, native MTP).
vLLM PR #53906.
tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark — SM121 DFlash2 image, --block-size 2304; barrydeen GMU 0.85 floor.

Abliteration recipe (published)

Byte-copy Dealign o_proj into RedHat shards. Not a λ-projection of a rank-1 V. Offsets from RedHat headers — never the LibertAI 120-shard map.


Tensor	`model.language_model.layers.{L}.self_attn.o_proj.weight` (BF16)
Layers	15–43 and 45 (30 tensors, includes MTP `layers.45`)
Skip	L44 RedHat stock (Dealign L44 Δrel 0.74)
Safety	L0–14 byte-identical to RedHat stock
Experts	NVFP4 passthrough (even in rewritten shards)
Gate	32/32 bypass, 0 refuse, 0 garble, raw vLLM + thinking-off template

Variation table: GitHub METHOD.md. Artifacts: ABLIT_META.json, VARIATIONS.json.

Reproduce:

python3 scripts/apply_oproj_l15_45.py \
  --src /path/to/GLM-5.3-Flash-NVFP4-RedHat \
  --dst /path/to/dest \
  --bins ./oproj_bins \
  --skip-layers 44 \
  --fresh

Thinking leak (template, not ablit)

Stock GLM-5.3-Flash always opens <think> and injects Reasoning Effort: Max. enable_thinking=false is a silent no-op — CoT lands in content. Mount chat_template.thinking-off.jinja over chat_template.jinja at serve time. Stock chat_template.jinja in this repo is unchanged from RedHat.

Files

Path	Purpose
`model-00001-of-00010.safetensors` … `00010` + `model_mtp.safetensors` + index	Full NVFP4 checkpoint (RedHat layout, L15–43+L45 `o_proj` from Dealign)
`ABLIT_META.json`	Edit stats / recipe fingerprint
`VARIATIONS.json`	Refusal32 table
`chat_template.thinking-off.jinja`	Recommended serve overlay (closed `<think></think>`)
`tokenizer.json` / `chat_template.jinja` / processor	Unchanged from RedHat / Z.ai

config.json is RedHat stock (index_topk 2048, quant_method=compressed-tensors). Honor it — do not pass --quantization modelopt_fp4.

The previous 120-shard LibertAI ModelOpt files (model-*-of-00120.safetensors) are removed from this repo.

Download

# after you agree to the gate (automatic approval)
hf download drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45 \
  --local-dir ~/models/GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45

Serve with Tony’s 2× DGX Spark DFlash2 recipe (marlin MoE, DFlash2 k=7, fp8 KV, --block-size 2304). GPU memory utilization ≤ 0.85. Mount the thinking-off template.

License

MIT, inherited from Z.AI GLM-5.3-Flash (also the RedHat NVFP4 card). You must still comply with the Responsible Use gate above.

Downloads last month: 850

Safetensors

Model size

169B params

Tensor type

F32

BF16

F8_E4M3

Model tree for drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45

Base model

zai-org/GLM-5.3-Flash

Quantized

RedHatAI/GLM-5.3-Flash-NVFP4

Finetuned

(2)

this model