nikimori/gemma-4-e4b-it-6bit-slim

Gemma 4 E4B-it, 6-bit MLX (slim)

A slimmer re-pack of mlx-community/gemma-4-e4b-it-6bit, a conversion of google/gemma-4-E4B-it:

• The audio tower is removed. The vision tower stays, because Pretype's speculative-decoding load (with Google's MTP drafter) needs it.
• Both embedding tables are 4-bit (group size 64): the shared input/output table and the much larger per-layer table. Pretype's loader already re-quantized them to 4-bit in memory, so these are the weights it ran. A table is only a lookup, so this costs no speed.
• All other weights are unchanged.

The weights are 5.53 GB instead of 7.01 GB (−21%). In Pretype's evals the outputs match the original (details). The license is the upstream one (Apache-2.0).

Made for Pretype, on-device autocomplete for macOS. It is the Instruct-style model of Pretype's Gemma E4B tiers (the default on Macs with 32 GB or more): it writes their completions, fixes a selection (⌥Tab) and drafts replies.

On Pretype's held-out sets (2026-10-11, 17–19 languages), it restores 69% [65–73] of single-typo sentences exactly, proposes a change on 18% of clean lines, and drafts a usable reply for 97% [94–98] of on-screen conversations.

from mlx_lm import load, generate
model, tokenizer = load("nikimori/gemma-4-e4b-it-6bit-slim")
messages = [{"role": "user", "content": "Fix the typo: I recieved your letter."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=40))