Gemma 4 12B — v4: the TIED-TABLE build (one fp16 vocab file, two consumers; verify at the rungs that load)
Supersedes tiered-v3/. Four changes, each measured before it shipped:
1. The head AND the embedding leave the graph as ONE bound file
Gemma 4 ties lm_head.weight to embed_tokens.weight, so v4 binds a single fp16 sidecar
(head.f16, [262144 x 3840]) as a graph INPUT serving both the in-graph embedding gather
(times the model's own rounded embed_scale = 61.96773353931867) and the head matmul.
A bound input's value is unknown at compile time, so the compiler cannot pre-transpose it —
the mechanism that sank the fp16-head E4B (+1.34 GB of graph constant per entrypoint,
measured). Measured on this blob: graph constant -12,907,432 B (~0), resources.bin /
main.mlirb ratio 0.9979, and main.mlirb is 5.72 GiB against v3's 7.60 — the fp16
head's 1.88 GiB no longer serialized.
This also retires v3's int4 in-graph embedding and its measured 7.43 pp top-1 cost: the
gather now reads fp16 rows, bit-exact against the reference ScaledEmbedding (the fp32
round-trip variant measured rel=2.5e-2 on 6-layer logits and was REJECTED by the gate; the
shipped op order is fp16 gather x fp16 scalar, max_abs = 0.0).
2. Verify moved to the rungs a 16 GB machine actually loads
v3 pinned verify_c262144 to the top rung on the claim that every rung fits. The demand
model falsified it: with the mapped table counted, c262144 demands ~13.8 GiB against a
12.575 GiB budget, while c131072 demands ~11.8. So v4 ships 2 verify
entrypoints — verify_c131072 and verify_c65536 — each S=8, FULL logits
[1,8,262144] plus the post-final-norm hidden states as activations [1,8,3840] (the
assistant drafter's input; oracle-parity-gated port already in hand).
3. accepted_count — speculation that cannot corrupt the cache
Verify takes an int32 [1] accepted_count bounding how many of the block's rows COMMIT to
the sliding ring: 0 speculates (ring bit-identical afterwards — gated), n commits the
accepted prefix, 8 commits all. The blend is a runtime VALUE over the literal ring extent —
never a shape — so every entrypoint stays shape-stable (ZERO symbolic extents, checked at
save). Global-cache rows are always written and self-heal: absolute positions mask them and
later writes overwrite them. Rows the blend skips keep exactly the keys a future window
still admits — the wrap arithmetic is in the module comment.
4. Same ladder, same states, same kernels
7 rungs (c4096..c262144) x (main + prefill S=64), the length-bounded Metal flash-decode
kernel and its prefill sibling, fp16 KV. function_map = {main: 7, prefill: 7, verify: 2}
written by the exporter — the role v3 needed a metadata amendment for is now in the
artifact from birth.
Loader contract
capacity_tier_notes declares head_table_file / head_table_shape / head_table_dtype
AND embed_from_head_table: true. The second declaration matters: it tells the loader
this bundle has NO PLE pair, so the family-default static inputs must be cleared before
binding the head (a loader that skips this refuses on embed_per_layer.* files that never
existed for this model). Absence of both declarations still means an in-graph head — every
pre-v4 bundle keeps loading.
Bundle — v4/gpu-pipelined/gemma4_12b_qat_decode_int4linsym_hsc_msdpa_g8_wkv_stable_pf64_tiered_c4k_c8k_c16k_c32k_c64k_c128k_c256k (6,177,845,160 B)
gemma4_12b_qat_decode_int4linsym_hsc_msdpa_g8_wkv_stable_pf64_tiered_c4k_c8k_c16k_c32k_c64k_c128k_c256k.aimodel/main.hash 32 B 3ea6c629221724c402f151cd73b0f107981ba3a377c995f25cdcce281e42ca59
gemma4_12b_qat_decode_int4linsym_hsc_msdpa_g8_wkv_stable_pf64_tiered_c4k_c8k_c16k_c32k_c64k_c128k_c256k.aimodel/main.mlirb 6,145,628,244 B 40e06a7b5f0d71a2138e32722fbff3a940b79b788b8b13561ec1594b324c7fa3
gemma4_12b_qat_decode_int4linsym_hsc_msdpa_g8_wkv_stable_pf64_tiered_c4k_c8k_c16k_c32k_c64k_c128k_c256k.aimodel/metadata.json 105 B 289d1e4bfcd080febff255b763fb85de7f93ea1767d00e86c04b9276199b6c4b
metadata.json 5,058 B d6f7f28732454182a4b29293e80e24bd7e73fd0f293a2c13a3e8f532619d34b6
tier_report.json 20,322 B 773dea6665588288f4c1a28987fc56e7bdfc7d9b78893ee16e0ae8882eef39ba
tokenizer/chat_template.jinja 18,683 B ae53464bf3be25802b3a5b37def7fd89667067d7577049b3b2d74c4d8de4c6d4
tokenizer/tokenizer.json 32,169,626 B cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
tokenizer/tokenizer_config.json 3,090 B c3fd34e321798e184e53894aaa1d8c4b37d08daf1fb1826dbceb3dcda0c62d06
Tables — v4/ios-frontend/gemma4_12b_tied_vocab (2,013,265,920 B)
head.f16 2,013,265,920 B fb21807f7117be132acbb4fcc548d3d0479fb84dce64f9f19e8c4e3af01f1a21
Export: export_gemma4_12b_tiered.py --tiers 4096,8192,16384,32768,65536,131072,262144 --prefill 64 --verify-len 8 --verify-tiers 131072,65536 --head-sidecar --quant-embed none --split-g 8 (2026-08-28). Gates: 6-layer torch parity bit-exact (decode logits, verify
logits, hidden, commit semantics all max_abs = 0.0); compiled-blob constant ~0; Mac decode
run under the 3-slot loader.