steampunque/GLM-Z1-9B-0414-MP-GGUF

Mixed Precision GGUF layer quantization of GLM-Z1-9B-0414 by THUDM

Original model: https://huggingface.co/THUDM/GLM-Z1-9B-0414

The hybrid quant employs different quantization levels on a per layer basis to enable both high performance and small file size at the same time. An extended layer definition E quant Q4_E_H for the model is defined as follows (updated 10/12/2026):

   LAYER_TYPES='[
   ["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
   ["MAP","VOD","0","QN_K","2","Q2_K","3","Q3_K","4","Q4_K","5","Q5_K","6","Q6_K","8","Q8_0","h","F16","f","F32"],
   [0 ,"Q6_K_886"],[1 ,"Q5_K_666"],[2 ,"Q4_K_665"],[3 ,"Q4_K_655"],[4 ,"Q4_K_654"],[5 ,"Q4_K_654"],[6 ,"Q4_K_654"],[7 ,"Q4_K_654"],
   [8 ,"Q5_K_554"],[9 ,"Q4_K_554"],[10,"Q4_K_554"],[11,"Q4_K_554"],[12,"Q4_K_554"],[13,"Q4_K_554"],[14,"Q4_K_554"],[15,"Q4_K_554"],
   [16,"Q5_K_654"],[17,"Q4_K_654"],[18,"Q4_K_654"],[19,"Q4_K_654"],[20,"Q4_K_654"],[21,"Q4_K_654"],[22,"Q4_K_654"],[23,"Q4_K_654"],
   [24,"Q5_K_665"],[25,"Q4_K_665"],[26,"Q4_K_665"],[27,"Q4_K_665"],[28,"Q4_K_665"],[29,"Q4_K_665"],[30,"Q4_K_665"],[31,"Q4_K_665"],
   [32,"Q5_K_665"],[33,"Q5_K_665"],[34,"Q5_K_665"],[35,"Q5_K_665"],[36,"Q5_K_666"],[37,"Q5_K_666"],[38,"Q5_K_668"],[39,"Q6_K_886"]
   ]'
   FLAGS="--token-embedding-type Q6_K --output-tensor-type Q6_K --layer-types-high  --tensor-pad [[13696,13824],[27392,27648,2]] --override-kv glm4.feed_forward_length=int:13824"

The quant pads FFN dimension to even 256 so K quants can be used with it. The layer quants were optimized for strong performance across a curated set of reasoning prompts with a minimum quant across layers at Q4_K to minimize entropy loss. It is sized to ~Q4_K_M bpw.

Comparison:

Quant	size	PPL	Comment
Q4_K_M	6.2e9	14.9	-
Q4_E_H	6.2e9	15.2	Hybrid quant with Q6_K embedding Q6_K output padded FFN tensors for K quants

Usage:

This is a RL trained thinking model. The layer quants for this model were optimized across a set of test/eval reasoning prompts.

The model can be speculated using Qwen3 0.6B if the inference engine can support dynamic vocab translation between draft and target models. Approx performance using a downstream speculator with llama.cpp on a 4070 (12G VRAM) with layers and context fully in GPU:

Q	QKV	ND	NKV	gen tps	Comment
Q4_E_H	F16	0	32k	68	No draft
Q4_E_H	F16	4	32k	101	Spec 4

Code:

Testing with some code problems show the model is extremely weak on code generation problems.

Benchmarks:

Math benchmarks for the model are given here: https://huggingface.co/spaces/steampunque/benchlm

Download the file from below:

Link	Type	Size/e9 B	Notes
GLM-Z1-9B-0414.Q4_E_H.gguf	Q4_E_H	6.2e9 B	~Q4_K_M size

A discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:

https://github.com/ggml-org/llama.cpp/discussions/13040