Maple-Preview — hipfire HFQ (arch 15)

A hipfire build of deepgrove/maple-preview, a 20B-A1B natively-ternary reasoning MoE, for AMD RDNA GPUs.

File	maple-preview.mq2lloydu
Size	6,499,340,288 bytes (6.05 GiB)
sha256	7fb52fe72c1a0a4455d0fe3a8109b0df66fa53782f41d8b257140d3e966645db
arch_id	15
Body carrier	MQ2G256LloydU (qt=51), 2.250 bpw
lm_head	Q8
Context	131,072

hipfire run maple-preview "Prove that sqrt(2) is irrational."

The body is a packing, not a quantization

Every linear weight in Maple-Preview is already exactly {-s, 0, +s} with one bf16 scale per output row. MQ2G256LloydU (qt=51) is an unrotated MQ2-Lloyd carrier built for this: it stores those three levels exactly.

Verified end-to-end against the original deepgrove/maple-preview safetensors — not against our own dequantizer, which could only confirm a shared misunderstanding:

model.layers.0.mlp.experts.0.gate_proj.weight   max|err| = 0
model.layers.0.mlp.experts.0.down_proj.weight   max|err| = 0
model.layers.0.self_attn.q_proj.weight          max|err| = 0
model.layers.0.self_attn.o_proj.weight          max|err| = 0

Every bitwise difference is a signed zero (-0.0 → +0.0), which is numerically irrelevant. The converter refuses a non-ternary row rather than falling back to lossy quantization, so a clean convert is itself evidence the property holds across all 18,528 ternary tensors (19,579,011,072 weights).

The accuracy this model loses relative to full precision therefore comes from the runtime, not the weights.

Measured quality

Against a BF16 reference — deepgrove/maple-preview converted to a BF16 GGUF and scored with llama.cpp on CPU — over 2,048 teacher-forced wikitext tokens, identical tokens and protocol on both sides:

KV tier	lm_head	mean KL	top-1 agreement
bf16 (default)	Q8	0.0511	91.9%
q8	Q8	0.0842	90.8%

The lm_head is free: a bf16 head gives the identical 0.0511, so Q8 costs nothing and decodes 23% faster.

Interpreting 0.0511. It is close to the floor for this model rather than a defect. llama.cpp rounds activations to bf16 before every matmul (vec_dot_type = GGML_TYPE_BF16, ε = 2⁻⁸), so the reference is itself precision-limited: an F16 reference built from the same weights differs from the BF16 reference by mean KL 0.0357 — the same order as our entire divergence from it — and this build sits closer to the more precise reference (0.0477) than to the bf16 one (0.0511). Maple is unusually sensitive to activation precision, plausibly because ternary weights and 256-expert top-8 routing make expert selection easy to perturb.

At position 0, where attention has no history to read, KL is 1.2e-4.

Performance and the lm_head

gfx1151 (Radeon 8060S / Strix Halo), bf16 KV, greedy, 1,500 tokens, 3 paired interleaved repetitions. KL is against the BF16 reference on the same 2,048 tokens as above.

lm_head	bpw	mean KL	top-1	decode
q8 (shipped in the base)	8.5	0.0511	91.9%	~145 tok/s
q4k (overlay)	4.5	0.0640	90.1%	~168 tok/s
bf16 (overlay)	16	0.0511	91.7%	~118 tok/s

Prefill is roughly 800 tok/s on short prompts.

q8 is the default because bf16 is strictly dominated — identical KL, 23% slower. A bf16 head costs throughput and buys exactly zero accuracy.

q4k is offered for throughput, not recommended by default: +16% decode for +25% mean KL and −1.8pp top-1. Note DeepGrove do ship a Q4_K head in their own llama.cpp example, and that is the right call there — on their CPU path the same swap buys 49% (252.7 vs 169.8 tok/s) rather than 16%, because the MoE body dominates decode differently on a GPU. Same decision, different machine.

Head variants ship as overlays, not as separate models

The head is 2.7% of the file, so shipping a whole model per carrier would duplicate the identical 6.17 GB body every time. Instead the base carries the recommended q8 head and runs standalone, and alternative carriers are single-tensor .hfq overlays that shadow lm_head.weight at load:

file	size
maple-preview.mq2lloydu (base, q8 head)	6.50 GB
maple-head-q4k.hfq	188 MB
maple-head-bf16.hfq	635 MB

hipfire run maple-preview --head q4k "..."     # by registry name
hipfire run <model.hfq> --head <head.hfq> ""   # or by path

Each overlay is a normal .hfq carrying a single tensor at the same logical shape as the base's head, which is what lets hipfire attach it at load time and shadow lm_head.weight. They are listed in the registry entry under heads, with their own sha256, and are rebuilt with:

hipfire-quantize --format maple --input <safetensors dir> \
  --output maple-head-q4k.hfq --head-quant q4k --head-only

An overlay is not an approximation of a full build — it reproduces one exactly (q4k overlay 0.0640 KL / 90.1% top-1; a monolithic q4k build measures the same). Three full variants would be 19.63 GB; this is 7.30 GB, and switching heads costs 188–635 MB rather than a 6.5 GB re-download.

Recommended settings

Use the vendor's values — they are not the Qwen defaults, despite the shared tokenizer:

temperature 1.0
top_p       0.95

Source: DeepGrove's own llama.cpp fork, github.com/deepgrove-ai/llama.cpp, which deepgrove/maple-preview-GGUF links to as the official setup. There is no generation_config.json upstream and the model card names no sampler, so that README is the only first-party source. top_k / min_p values circulating elsewhere are not from DeepGrove.

The chat template matters. Maple's embedded template ends its generation prompt with <|im_start|>assistant\n<think>\n — including the thinking prefix. Omitting it makes the model open its own reasoning block, starting every generation off-distribution inside the block where degenerate loops occur. On a fixed prompt across 8 seeds, the loop rate went from 5/8 to 0/8 once the prefix and temperature were corrected. hipfire applies the template from the HFQ metadata automatically.

KV cache

Defaults to bf16, overridable with --kv-mode q8.

Maple is 3:1 sliding-window(512) / global-NoPE attention. Q8 KV costs 39% of the measured divergence, and the damage is concentrated in the tail rather than spread as uniform blur — the median position moves only 24%, but the worst goes 10.36 → 4.21 nats. Rare catastrophic positions are what derail long generations.

bf16 KV costs 1.88× the KV bytes (26,112 → 49,152 B/token; +2.81 GiB at 131k context) and ~2% decode. Use --kv-mode q8 to trade that fidelity back for memory.

Limitations

These are the model's, not the port's — see the upstream card:

This preview received minimal post-training for agentic tasks and only small-scale general reinforcement learning.

Wikitext perplexity is high (~57 on the standard protocol) where a comparable peer scores ~7.7. That is the model, not the packing: KL against a full-precision reference is 0.05, and DeepGrove's own ternary GGUF scores within 0.2% of a BF16 run of the same weights. Maple is not uncertain on raw prose — its entropy matches its peer — it is confidently wrong on it, which is consistent with a reasoning-focused preview.

Give it room to think. Maple spends a large share of its budget reasoning, and a short cap truncates it mid-derivation in a way that reads like a repetition loop but is not one. On a five-house logic puzzle, 8/8 sampled runs hit a 12,000-token cap without converging — while at 40,000 tokens they terminated cleanly with the correct answer after ~84–134 KB of reasoning. If output looks stuck, raise max_tokens before suspecting the model.

Most of what otherwise looked like intrinsic looping was configuration: with the vendor template and temperature above, the loop rate on a fixed prompt went from 5/8 to 0/8 across 8 seeds.

Provenance

MIT, inherited from deepgrove/maple-preview. Weights are DeepGrove's, carried into HFQ losslessly. Converted with:

hipfire-quantize --format maple \
  --input <maple-preview safetensors dir> \
  --output maple-preview.mq2lloydu \
  --head-quant q8