Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q8

Kimi-K3.1-Pro, REAP expert-pruned to 20% of experts with a code-weighted calibration, converted to MLX. 326 GiB: sized to load on a 512 GB Apple Silicon machine.

Base: moonshotai/Kimi-K3.1-Pro (1.56 TB, MXFP4 experts, Kimi Delta Attention + gated MLA with a DSA indexer, Attention Residuals, latent MoE). Highest-precision tier of the REAP80 code builds; q6 is within 0.2% perplexity of it and faster.

What this is

• Pruning: REAP saliency (gate x ||expert_output||) over a code-heavy calibration corpus (60% code, with English-web and Chinese slices). 179 of 896 routed experts kept per MoE layer (92 MoE layers), top-16 routing unchanged, 2 shared experts untouched.
• Experts: the surviving routed experts are a bit-exact byte copy of Moonshot's MXFP4 weights (MLX mxfp4 uses the identical encoding). Pruning is the only loss on the expert side.
• Everything else (attention, shared experts, latent projections, layer-0 dense MLP, embeddings, lm_head): MLX affine 8-bit, group size 64. The DSA indexer (wq_b, wk, weights_proj) and norms/routers stay bf16/f32 on purpose: indexer error only shows past 2048 tokens, so it would not show up in short evals.
• The residual stream is float32 from the first MoE layer on (as in the reference loader); only stored weights are quantized.

Measured quality

Held-out perplexity at 2048 tokens, identical token windows, paired bootstrap (REAP80, code-calibrated):

non-expert precision	overall ppl	vs q8
q8	5.725	baseline
q6	5.736	+0.2% (code +0.0 to 0.1%)
q4	8.81	+53.7% (python +44%, code-multi +17%)

q8 is the reference tier for the comparison above.

Greedy continuation of def merge_intervals(intervals): (13-token prompt, verbatim, from the MLX reference run):

    if not intervals:
        return []
    
    intervals.sort(key=lambda x: x[0])
    merged = [

This is an aggressive prune (top-16 of 179 experts). Expect real degradation versus the full model, especially outside code. For example, in one chat test on the q6 build, a greedy "What is 17 times 23?" produced a reasoning trace that settled on 323 (the answer is 391). Treat it as a "fits on one Mac" code model, not a quality build, and verify anything that matters.

Speed (512 GiB M3 Ultra)

Each decoded token reads roughly 70-85 GB of weights (about 26 GB of routed experts plus all non-expert tensors), so decode is memory-bandwidth-bound.

path	decode	notes
mlx-lm (mlx_lm.generate)	~5.5 tok/s (about 10% slower than q6)
native Metal engine (pipefold, unreleased)	~6.5 tok/s	prefill about 60 tok/s

Pruning buys memory, not speed: decode traffic depends on top-k and non-expert precision, not on how many experts are stored.

The native engine's greedy tokens were checked against MLX on this build: 24/24 greedy tokens identical on a code prompt.

Usage

Requires mlx-lm and the bundled kimi_k31.py loader (the kimi_k31 architecture is not upstream). On a 512 GB machine raise the GPU wired limit first (the default is below this model's footprint):

sudo sysctl iogpu.wired_limit_mb=480000
pip install mlx-lm

python - <<'PY'
import os, shutil, mlx_lm
from huggingface_hub import hf_hub_download
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "kimi_k31.py")
shutil.copy(hf_hub_download("pipenetwork/Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q8", "kimi_k31.py"), dst)
PY

mlx_lm.generate --model pipenetwork/Kimi-K3.1-Pro-REAP80-code-MLX-mxfp4-q8 --trust-remote-code \
    --max-tokens 256 --prompt "Write a Python function that merges overlapping intervals."

Notes:

• Tokenizer is the model's tiktoken (tiktoken.model + tokenization_kimi.py, needs trust-remote-code). tokenizer.json and chat_template.jinja are also included for engines that read them: verified token-for-token against tiktoken, and the template reproduces encoding_k3.py for plain system/user/assistant messages only (tool calls and images are not covered by it).
• Only the text path was validated. The vision files ship alongside but were not exercised.
• Safetensors shards are 16-byte aligned so engines can map them zero-copy.
• Long prompts: peak memory grows with context. Test your longest prompt before relying on it.

License

Weights remain under the Kimi K3.1-Pro License (included as LICENSE). Note its conditions for large-scale commercial use (a separate agreement above 20M USD / 12 months for Model-as-a-Service businesses; prominent "Kimi K3.1-Pro" display above 100M MAU or 20M USD/month); read it before commercial use.