brandonmusic

/

GLM-5.2-EXL3-TR3v4-3.5bpw-MTP78

Like 9

Summary

Store the R7 shared experts in original BF16 form so the current Gilded
Gnosis runtime can encode one merged K6 gate_up_proj from each gate/up pair.
The routed per-expert K3/K4/K5 payloads and rotation metadata remain unchanged.

This avoids requantizing the already encoded separate K6 matrices, removes 75
kernel launches per target step, and leaves the checkpoint flexible for future
shared-expert runtime formats.

Checkpoint change

• Replace layers 3-77 shared-expert gate/up/down storage with 228 BF16 tensors
  from willfalco/GLM-5.2-EXL3-TR3-3.42bpw@a350292.
• Remove 225 stale shared-expert EXL3 ownership records.
• Rebuild the safetensors index and quantization metadata.
• Preserve the original R7 assembly manifest as SOURCE_R7_MANIFEST.json.
• Remove the old manifest seals because they cannot describe the derived
  checkpoint.
• Include the converter, focused tests, exact derivation hashes, and measured
  results.

All 228 donor tensors were checked byte-for-byte. The resulting index contains
186,905 tensors and no stale shared-expert EXL3 records.

Runtime result

TP4/DCP1, MTP0, A16, online K6, NVFP4 DS-MLA KV:

Prefill was at parity at 8k (3,280 vs 3,265 tok/s). The single 64k sample was
3,034 vs 3,254 tok/s and is documented as indicative rather than a release
claim.

The paired 2,047-position full-vocabulary KLD gate measured 0.064467 for the
original R7 runtime and 0.065339 for this revision. The +0.000873 difference is
smaller than run-to-run variation and the distributions overlap.

MTP0 and MTP3 ran in full CUDA graphs with no eager fallback or new K6/profile
bottleneck. Focused converter tests: 3 passed; Ruff check and format: passed.

Merge dependency

Merge or deploy this checkpoint revision together with the companion Gilded
Gnosis R7 BF16 shared-expert runtime. The intended launch contract is:

MODEL_FAMILY=glm52-exl3 QUANTIZATION=exl3 ONLINE_QUANT=exl3-b6

Use a persistent JIT_CACHE so only the first load performs K6 encoding. The
existing v26 image in the parent README targets parent revision c55c1cd and
should remain pinned there until the companion runtime is released.

Superseded by #3. While this upload was running, the canonical BF16-shared layout landed directly on main as 9ab9579774cc432df91567a36f6e9e863e0d4c9f. That made this parent-based 75-shard rewrite conflict with the new layout. PR #3 is rebased on the canonical commit, changes no model shard, and carries the metadata corrections, reproducible converter, focused tests, and independent validation.