Instructions to use zai-org/GLM-5.3-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zai-org/GLM-5.3-Flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="zai-org/GLM-5.3-Flash")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("zai-org/GLM-5.3-Flash")
model = AutoModelForMultimodalLM.from_pretrained("zai-org/GLM-5.3-Flash", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zai-org/GLM-5.3-Flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zai-org/GLM-5.3-Flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/zai-org/GLM-5.3-Flash

SGLang

How to use zai-org/GLM-5.3-Flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5.3-Flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zai-org/GLM-5.3-Flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use zai-org/GLM-5.3-Flash with Docker Model Runner:
```
docker model run hf.co/zai-org/GLM-5.3-Flash
```

Reading GLM 5.3 Flash from config.json: 6.0 KB of KV Cache per Token

#55

by laxmimerit - opened Sep 16

Discussion

laxmimerit

Sep 16

A header-level comparison of the GLM 5.3 and GLM 5.3 Flash checkpoints

This note documents what the published config.json files and safetensors headers of GLM 5.3 and GLM 5.3 Flash contain. All values were derived from 194,737 tensor shapes. No weight files were downloaded, and every parameter count and cache size below is arithmetic on those shapes.

GLM 5 to GLM 5.3

The base architecture carries over: 78 layers, identical hidden size, 256 routed experts and an identical vocabulary. The diff is limited to three entries.

max_position_embeddings: 202,752 → 1,048,576
rope_theta: 1,000,000 → 8,000,000, extending RoPE to the longer window
index_topk_freq: 4: the indexer runs in 21 layers and 57 layers reuse the latest selection. Indexer tensors are present in exactly 22 layers, the 21 main layers and the MTP draft layer.

GLM 5.3 Flash

Flash is a distinct architecture.

Attention: 34 Kimi Delta Attention layers and 11 DeepSeek Sparse Attention layers, interleaved at 3:1.
Positional encoding: qk_rope_head_dim: 0 and mla_use_nope: true. kv_a_proj_with_mqa has shape [512, 4096], compared with [576, 6144] in GLM 5.3.
Indexer: index_kpool: 4, index_kpool_compress: true, index_kpool_always_select_tail: true.
Residual stream: hc_mult: 4 and hc_sinkhorn_iters: 20, 35M parameters in total.
Experts: 288 routed and 1 shared per layer, 25.17M parameters each.
Vision: 0.56B tower, 24 blocks, 448×448 input, 14×14 patches, temporal_patch_size: 2.

KV cache derivation

MLA caches a single latent per layer per token. With FP8 storage:

GLM 5.3   78 × (512 + 64) + 21 × 128      = 47,616 bytes/token
Flash     11 × 512        + 11 × 128 / 4  =  5,984 bytes/token

The ratio is 8.0x if key pooling applies to the stored indexer keys, and 6.8x if it applies only to the scan. The vendor reports approximately 4.4x, and the serving method behind that figure is not documented. For a 1,048,576-token sequence this corresponds to 49.9 GB for GLM 5.3, and 6.3 GB plus 0.07 GB of KDA state for Flash.

Full article: https://kgptalkie.com/tutorials/llm-benchmarking/glm-5-3-vs-glm-5-3-flash-architecture-teardown

Video: https://youtu.be/zWkc4iOi_c8

androidli

24 days ago

Nice arithmetic. One practical datapoint on the memory side: with a pack that quantizes only the routed experts to 3.0bpw EXL3 and keeps attention/shared/dense/head/vision at source-native precision, the non-expert weight savings are large enough that on 2× DGX Spark the model serves a 3,891,200-token KV pool (≈ 4 full 1M-token sessions) with no measurable quality loss vs 4bpw (EvalPlus HumanEval+/MBPP+ (542, thinking on) 468/542 = 86.3% — on par with the 4bpw packs).

Pack: huggingface.co/androidli/GLM-5.3-Flash-Uncensored-EXL3-3.0bpw-mcg-TensorFold · TensorFold recipe with the 3-bit gate patch + custom expert kernel: github.com/greatyingzi/GLM-5.3-Flash-EXL3-3bit-TensorFold

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment