wave-gemma4-E2B-q4f16_1-MLC

Three MLC-compiled Gemma 4 E2B variants, all quantized to q4f16_1 for WebGPU (plus a Metal dylib for the fine-tuned variant). Each subfolder is a self-contained MLC model — shards, tokenizer, mlc-chat-config.json, and the compiled .wasm library — usable by @mlc-ai/web-llm.

Compiled against MLC PR #3485. The google-it and finetuned builds use the <|turn|> conv_template patch from that PR; unsloth uses the original <start_of_turn> template.

Variants

Folder	Source	Status
google-it/	google/gemma-4-E2B-it	reference baseline
unsloth/	Unsloth-merged WAVE fine-tune	broken — pad-token failures downstream; kept for reproducibility
finetuned/	PEFT-merged WAVE fine-tune	working

google-it/

Google's Gemma 4 E2B (instruction-tuned), compiled as a baseline. WASM: gemma-4-E2B-it-q4f16_1-webgpu.wasm.

unsloth/

Built from an Unsloth-merged Gemma 4 + WAVE LoRA. Downstream loaders hit pad-token / tokenizer mismatches with this checkpoint (notice bos_token_id: 1 and the original <start_of_turn> template); the PEFT re-merge in finetuned/ resolves it. Kept here as a reproducibility artifact only. WASM: wave-r32-q4f16_1-webgpu.wasm.

finetuned/

PEFT re-merge of the same WAVE LoRA. This is the variant you actually want to use. WASM: wave-r32-q4f16_1-webgpu.wasm. Also includes a macOS Metal library: wave-r32-q4f16_1-metal.dylib.

Usage (@mlc-ai/web-llm)

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const REPO = "https://huggingface.co/Maelstrome/wave-gemma4-E2B-q4f16_1-MLC/resolve/main";

const engine = await CreateMLCEngine("wave-finetuned", {
  appConfig: {
    model_list: [{
      model_id: "wave-finetuned",
      model: `${REPO}/finetuned`,
      model_lib: `${REPO}/finetuned/wave-r32-q4f16_1-webgpu.wasm`,
    }],
  },
});

Tested with @mlc-ai/web-llm@0.2.83. web-llm has a per-engine state-leak; reload the engine per task call as a workaround.

Files per subfolder

• mlc-chat-config.json — MLC chat config (conv_template, quantization, tokenizer ids, etc.)
• params_shard_*.bin — quantized weight shards (46 shards)
• tensor-cache.json — shard manifest
• tokenizer.json, tokenizer_config.json
• *-q4f16_1-webgpu.wasm — compiled MLC model library for WebGPU
• wave-r32-q4f16_1-metal.dylib (finetuned only) — compiled Metal library for macOS

Quantization

• Weights: q4f16_1 (group-quantized 4-bit weights, fp16 activations)
• Context window: 131072 tokens
• Sliding window: 512