apocryphx/gemma-4-E2B-it-qat-q4-apml

gemma-4-E2B-it-qat-q4.apml — Apertura model bundle (lattice-exact QAT)

Replaced 2026-10-09 — source checkpoint changed. Revisions before this date were a post-training MLX-affine group-64 quantization of the plain google/gemma-4-E2B-it release (Q8 head, Q8 per-layer table). This revision is built from Google's QAT checkpoint with the lattice-exact recipe below; bundle layout and name are unchanged. 2.9 GB on disk (was 4.0 GB).

Gemma 4 E2B (elastic 2B) instruction-tuned, Google's QAT-Q4 weights (*-qat-q4_0-unquantized) converted to the Apertura .apml bundle format with the lattice-exact recipe: MLX-affine 4-bit, group size 32, where each block's scale is the QAT-learned step itself (scale = d, bias = -8·d), so the 4-bit weights the kernel sees are the trained lattice points — not a second, misaligned quantization of them. The tied embedding / LM head AND the per-layer embedding table (embed_tokens_per_layer, half of the model's parameters) sit on the same lattice and are stored exact at 4 bits. tokenizer.json + chat_template.jinja included, runtime mlx, architecture gemma4.

Consumed by Apertura — an Objective-C++ transformer engine + macOS chat app on MLX.

Why this recipe

google/gemma-4-E2B-it-qat-q4_0-unquantized is bf16 weights that already sit on an int4 lattice: per 32-element block, w = bf16(k·d) with codes k ∈ [-8, 7] and a learned per-block step d. Running a min/max affine quantizer over that (as a plain post-training quantizer must) picks a different scale and zero point and lands most weights off their trained values — the same failure Unsloth documented for naive Q4_0 GGUFs of these checkpoints. Here d is recovered from the lattice structure per block and written as the affine scale.

Measured against the bf16 QAT checkpoint, over all 4.59 B quantized weights (the old row is measured against ITS source, the plain bf16 -it release):

recipe	bit-exact	within 1 bf16 ulp	top-1 vs bf16, 3 probes	top-1, 2176-tok prompt ×256
plain -it source, affine g64 + Q8 head/table (previous revisions)	14.2%	33.2%	131/144	240/256
QAT source, lattice g32, exact Q4 head + table (this revision)	90.4%	100.0000%	116/116	251/256

The weights that are not bit-identical differ by one bf16 rounding step: the checkpoint stores bf16 roundings of k·d, so no single bf16 step reproduces every element. The bundle and the checkpoint are two equally faithful bf16 renderings of the trained lattice. Group 32 cost ~5% decode vs group 64 on the 31B (Apertura roadmap §1d); not re-measured on this model; fidelity was the aim. Requires an Apertura build from 2026-10-09 or later: the QAT elastic checkpoints omit the unused K/V projections on the shared-KV layers, and older builds expect them.

weights/mlx-q4/quantization.json records lattice: qat-int4-g32, embed_bits: 4, ple_bits: 4 and the fit statistics. Head/table re-quantization is disabled by the loader on lattice bundles.

Download

hf download apocryphx/gemma-4-E2B-it-qat-q4-apml --local-dir gemma-4-E2B-it-qat-q4.apml

License

Gemma is provided under and subject to the Gemma Terms of Use. By downloading this model you agree to those terms, including the Gemma Prohibited Use Policy. This repository redistributes a quantized Model Derivative of google/gemma-4-E2B-it-qat-q4_0-unquantized; the same terms and use restrictions apply to it and to any further derivatives.