brandonmusic/GLM-5.3-EXL3-TR3-3bpw

GLM-5.3 EXL3/TR3 uniform-K3 (3.0 bpw routed) — DRAFT

Measured quality — two KLD measurements, and why there are two

An initial reading of mean KLD 0.0979 on this repo's own sealed panel looked far worse than davidsyoung/GLM-5.3-EXL3-TR3-3.25bpw's reported 0.0240 — but the two figures were scored on disjoint window sets with zero token-SHA overlap, so they were never comparable. Measurement 2 below rescoures this same quant on his exact four published windows to make the comparison apples-to-apples. Result: the numbers reconcile; there is no measurement error on either side. The original gap was (a) window-set composition — this panel's encyclopedic windows score 0.21–0.25 at flat K3 — and (b) flat 3-bit vs his mixed/tiered allocation at 3.42 bpw effective.

Measurement 1 — this repo's sealed panel (16 confirmation windows × 2047 positions)

Axis	Value
KLD mean	0.097893
KLD median	0.007149
KLD p90 / p95 / p99	0.186 / 0.501 / 1.752
KLD max	8.256
Top-1 agreement (teacher vs student)	91.79%
Top-8 overlap	84.28%

The median-vs-mean spread shows the flat-K3 signature: the typical token is nearly untouched; a thin tail of heavily-damaged experts drives the mean.

Measurement 2 — the published panel-v1 confirmation windows (davidsyoung's exact lane)

Teacher logits and token arrays from brandonmusic/GLM-5.3-BF16-full-logits: reference-full-panel/logits/confirmation/confirmation-000{0..3}.safetensors and reference-full-panel/calibration/panel-v1/arrays/confirmation-000{0..3}.tokens.npy — the same dataset his measurement used. Identity proof: each token array's SHA-256 (below) matches the token_ids_sha256 embedded in the corresponding published teacher file, and was verified before capture.

Window	token_ids_sha256 (npy file)	This quant (clean bf16 replay)	davidsyoung 3.42 bpw (+fp8-KV serving)
confirmation-0000	11c149e8…445ece4	0.02234	0.0148
confirmation-0001	2def7320…9e771318	0.08992	0.0542
confirmation-0002	7f2728b7…484de05e	0.02010	0.0135
confirmation-0003	967f7232…837666eb	0.01782	0.0134
Mean		0.03754	0.0240

His figures are from kld/cn3/GLM53-EXL3-KLD-SUMMARY.csv in his repo. Asymmetry note: his numbers were measured through a serving runtime with fp8 KV cache (they include that error); ours is the clean teacher-forced bf16 replay with no KV quantization. The same window (0001) is the hard one for both quants — composition, not process. Ratio on identical text: ~1.56× at 3.0-flat vs 3.42-mixed; the clean-vs-clean encoder gap is somewhat larger, consistent with the missing 0.42 bpw and absent allocation.

Method (both measurements)

• Student = the BF16 model with all 57,600 routed expert matrices (layers 3–77 × 256 experts × gate/up/down) replaced in place by their authenticated uniform-K3 reconstructions; every decode passes a per-matrix codec-fp16 + stored-bf16 SHA-256 closure before installation.
• Forward: Transformers TP8, bfloat16 execution, sdpa, teacher-forced full 2048-token windows, no KV quantization, no sampling; logits stored float32, full 154,880-token vocabulary.
• Scoring: KL(teacher ‖ student), float64 log-softmax over the full stored vocabulary, per-position, then per-window and global means.
• Loader canary (measurement 2): every published teacher window scored against itself must yield exactly 0.0 at every position — passed for all four before any student scoring.

Reproduce it

Scripts in evidence/: kld_student_capture.py (patch + capture), kld_compare.py (float64 scoring), kld_axes.py (quantiles/agreement axes), xcheck_capture.py (published-window replay with token-SHA verification). Receipts: evidence/kld-student-uniform-k3.json, evidence/kld-axes-uniform-k3.json, evidence/xcheck-report.json (all input SHA-256s: token arrays, teacher files, candidate inventory, capture receipts).

Contents

• Routed experts, layers 3–77: EXL3/TR3 trellis-encoded at uniform K3 from calibrated Hessians (fit/conditional-fit lanes, seed 20260823 panel). materialized-uniform-k3/: per-layer consolidated shards + per-expert receipts + sealed manifest, remote-verified by size and SHA-256.
• mtp78-k5/: the MTP layer-78 head trellis-encoded at K5 (decode-verified, 256 experts). All other non-routed tensors: source BF16 by reference to zai-org/GLM-5.3-BF16 (revision pinned in evidence/source-inventory.json).
• Campaign resume artifacts (candidates, Hessian-source captures, seals): brandonmusic/GLM-5.3-BF16-shapleymcg-resume.

Licenses

ShapleyMCG License 1.0 (LICENSE). Derivatives must declare it with the exact identifier fixed in the LICENSE appendix: license_name: shapleymcg-license-1.0. Base model license: THIRD_PARTY_LICENSES/ZAI-GLM-5.3-LICENSE.txt (GLM-5.3 License — not MIT).