anuppillai/Kimi-K3-REAP-720-MXFP4

Kimi K3 REAP-720 MXFP4

Repository: anuppillai/Kimi-K3-REAP-720-MXFP4.

This is an unofficial community derivative of Moonshot AI's Kimi K3 MXFP4 checkpoint. It removes 176 of the 896 routed experts in every MoE layer using router-weighted expert activation pruning (REAP), retaining 720 experts per layer. The checkpoint remains natively multimodal: the vision tower, multimodal projector, attention layers, shared experts, tokenizer, and processor were not pruned or requantized.

Documentation-only update. This revision corrects the model card and evaluation metadata. No model weights or configuration files are changed: all 96 safetensors shards (1,276,563,666,168 bytes total; 1,188.89 GiB) remain as previously published.

Evaluation verdict for the intended workload

Project decision: acceptable for practical use. The full 198-item GPQA-Diamond run produced 155/198 = 78.28% (approximately 78.4% in the project summary). We accept this result for the intended workload of coding, tooling, vision, and agentic use. GPQA-Diamond is retained here as a transparent reasoning datapoint, not as the release gate for this derivative.

This model is not presented as a universal replacement for the strongest frontier reasoning systems. Its purpose is to make Kimi K3's native multimodal, tool-capable behavior practical on a single 8×B200 node after gentle expert pruning, while preserving a 262,144-token serving profile.

GPQA-Diamond context

The comparison below is contextual rather than a leaderboard or a controlled A/B test. These published results use different model versions, prompts, sampling settings, harnesses, and evaluation dates. They are included to show that the REAP-720 result is in the broad neighborhood of several capable frontier or frontier-adjacent systems, not to claim protocol equivalence.

Model / variant	GPQA-Diamond result	Primary source
Kimi-K3-REAP-720-MXFP4	155/198 = 78.28%	This repository's full test split; exact result in evaluation/gpqa_diamond_full.json
Gemini 2.5 Flash Preview (09-2025)	78.3%	Google Gemini 2.5 Flash model card
Gemini 2.5 Flash (GA, extended thinking)	80.8%	Google Gemini 2.5 Flash model card
DeepSeek-R1-0528	81.0%	Official DeepSeek-R1-0528 model card
OpenAI o4-mini	81.4%	Reported in the Google Gemini 2.5 Flash model card; treat as provider-reported context, not a matched run

The Google model card explicitly warns that provider results are not necessarily directly comparable. The DeepSeek card likewise reports its own decoding and evaluation setup. The exact REAP-720 measurement and raw per-item records remain the authoritative evidence for this checkpoint.

Intended use

The primary target is real-world engineering work, especially:

• Coding: repository changes, code generation, debugging, test writing, and long-context software work.
• Tooling: structured function calls, terminal-style workflows, API orchestration, and tool-aware task execution.
• Vision: native image understanding and vision-in-the-loop coding or engineering workflows; the vision tower and projector are retained unchanged.
• Agentic use: multi-step coding and knowledge-work workflows where local deployment, long context, tool access, and controllable latency matter more than maximizing a single academic reasoning score.

Use the model card's serving configuration and run task-specific acceptance tests before production deployment. Formal protocol-matched coding, tool-use, and vision scores are not claimed here; the next validation should use representative real-world workloads in the intended domains.

What changed

Property	Upstream Kimi K3	This checkpoint
Routed experts per MoE layer	896	720
Experts selected per token	16	16
MoE layers evaluated for saliency	92	92
Routed-expert count removed	—	19.64%
Weight format	MXFP4	MXFP4, unchanged for retained tensors
Native maximum position setting	1,048,576	1,048,576
Vision tower / projector	present	not targeted by pruning (vision_tower_modified: false; mm_projector_modified: false)
Checkpoint size	about 1,453 GiB	1,188.89 GiB (96 shards; 1,276,563,666,168 bytes)

REAP saliency was accumulated per expert as sum(router_probability * l2_norm(expert_output)). Calibration streamed one MoE layer at a time as balanced multidomain text plus 16 image-text samples. Each of the 92 scored MoE layers retained its 720 highest-saliency routed experts (a 0.1964 removed fraction). Router rows and expert-correction-bias entries were pruned and renumbered consistently. No retained expert was missing activation coverage in the completed saliency run (retained_experts_never_activated = 0); retained weights remain in mxfp4-pack-quantized format.

This is structural expert pruning, not a newly trained or distilled checkpoint. It should reduce resident weight memory; because K3 still routes to 16 experts per token, it does not by itself reduce the number of active experts used for each token.

Evaluation

Reasoning — verified full-split result

The GPQA-Diamond run used the full 198-item test split and scored 155/198 = 78.28% (approximately 78.4% in the project summary), with 0 infrastructure errors and 0 unscorable outputs; 3 items required a retry. This is accepted for the project's intended coding, tooling, vision, and agentic workload.

This remains a single run (runs = 1), not an average over repeated runs. It is also not a causal paired-pruning estimate because Full K3 was not run through the same prompt/choice/seed/serving harness. The result should therefore be read as an honest full-split REAP-720 measurement and practical acceptance decision, not as a claim of benchmark parity with Full K3.

Protocol (from evaluation/gpqa_diamond_full.json):

• Dataset: hendrydong/gpqa_diamond_mc, config default, split test, expected items 198; completed items 198, resumed items 0.
• Source order was preserved and structural-first schemas were shuffled.
• Settings: temperature 1.0, top-p 0.95, reasoning_effort="max", max_tokens 32,768, timeout 900 s.
• official: true; protocol fingerprint 1654304ead05affa3c6ee2549ebebf100470c0f5c9c3481dc46790f241cb049f.

The published Full-K3 GPQA-Diamond reference is 93.5%, but that number is not a matched control for this run. The project accepts the REAP-720 result because GPQA is not the primary objective; the primary objective is a usable local model for coding, tooling, vision, and agentic workflows.

Machine-readable aggregate and per-item records are included as evaluation/gpqa_diamond_full.json and evaluation/gpqa_diamond_full_records.jsonl.

Vision — invalidated run; no valid final score

There is no valid vision score. A full MMMU-Pro attempt was run but invalidated and excluded from baseline comparison; do not report its 195/246 raw count as a benchmark score.

Invalidated run details (operator-verified, from the correction ledger):

Field	Value
Benchmark	MMMU-Pro
Completed records	246
Correct records	195 (excluded — not a benchmark score)
Retried records	34
Unscorable records	15
Terminal-length finishes	46 of 182 requests in the active server metrics window
Reason invalidated	16,384-token output ceiling caused material length truncation, retries, and unscorable responses
Checkpoint SHA-256	e097d9cc96bd136a9a55314ceb70ea1a3c91c0842f4cb06e2157b3ce934cfe2e
Use	Excluded from baseline comparison

Machine-readable audit disclosure: evaluation/mmmu_pro_invalidated_run.json.

A separate five-item evaluator smoke at 80% exists, but it is diagnostic only — it validates the evaluator and end-to-end generation path, not a floor estimate. It is too small to establish the vision floor. Published MMMU-Pro Full-K3 reference: 81.6%.

Coding — no protocol-matched score

There is no DeepSWE score (final_deepswe_score = null; status: NO PROTOCOL-MATCHED SCORE). An earlier compatibility check found that three generated-code tasks passed the same isolated graders for both REAP-720 and the existing Q2 K3 deployment. A newer same-prompt comparison against K3 via the Moonshot API is summarized below. These are qualitative generation checks, not DeepSWE retention. Published Full-K3 DeepSWE reference: 67.5 (run on the Kimi Code harness, whose exact historical harness revision, task snapshot, seeds, and trajectories are not all public).

Tool use — no terminal score

There is no Toolathlon-Verified score (final_toolathlon_score = null; status: NO TERMINAL SCORE). An official private-mode Toolathlon client (repository hkust-nlp/Toolathlon, commit 2aed2468858f15818acafa178518390cc4b0f5cb) connected its remote execution service to the local OpenAI-compatible endpoint. The selected find-alita-paper task did not reach a terminal score: two long model turns exhausted their output limit, and the job was cancelled after approximately 20 minutes (to free the single-sequence server for GPQA and MMMU-Pro runs). This is compatibility evidence only, not a Toolathlon score or a Full-K3 comparison. Published Full-K3 Toolathlon-Verified reference: 76.5 (full run needs 108 stateful task environments plus external service accounts; the historical K3 trajectories are gated and the immutable configuration behind 76.5 is not public).

Attempt record (evaluation/toolathlon_compatibility.json):

Field	Value
Job ID	job_4419ec6d80d2
Terminal status	cancelled
Model requests started	13
Model responses HTTP 200	12
Completed task artifacts	0
Score	null
Interpretation	protocol compatibility only

Accordingly, no coding/tool retention ratio is reported and no four-dimension pass claim is made.

Benchmark comparability limits

The published Full-K3 references used for the requested floors are 93.5 (GPQA-Diamond), 81.6 (MMMU-Pro), 67.5 (DeepSWE), and 76.5 (Toolathlon-Verified). The coding and tool-use references cannot honestly be compared with the lightweight evidence above: the historical harnesses, task snapshots, seeds, and trajectories behind the DeepSWE and Toolathlon-Verified numbers are not fully public.

Qualitative paired generation spot checks — not formal benchmarks

The same strict prompts were run once through the live REAP-720 deployment and K3 via the Moonshot API. Outputs were saved and tested locally. These single-generation comparisons are useful for examining practical strengths and failure modes, but they are not statistically powered benchmarks, protocol-matched retention measurements, or evidence that the four-dimension quality gate passed.

Test	REAP-720	K3 via Moonshot API	Takeaway
Python: balanced-bracket validator, hand-built O(1) LRU cache, and concurrent token-bucket limiter	3/3 compiled; all included and independent tests passed. Working review: 9.8/10	3/3 compiled; all included and independent tests passed. Working review: 9.7/10	Functional tie. REAP supplied broader tests and clearer internal-behavior documentation; the API version was slightly leaner and added stricter zero-capacity/refill validation.
2D aquarium — visual fidelity	7.0/10	7.5/10	API output had the clearer radial crack, water jet/mist, puddle, and mobile scaling. REAP had richer fish, plants, graded water, glass, and settled shards.
2D aquarium — implementation/runtime	8.1/10	7.2/10	REAP had no uncaught runtime errors and stronger pooling. The API output repeatedly produced NaN-geometry console errors.
2D aquarium — equal-weight overall	7.6/10	7.4/10	Narrow REAP win for this sample; both outputs still had Reset/replay correctness issues.

2D aquarium comparison

_{Figure: Static matched aftermath frame at approximately 10 seconds. Left animation: generated by K3 via the Moonshot API. Right animation: generated by Kimi-K3-REAP-720-MXFP4 (720/896 routed experts). Both WebGL animations were generated from the same prompt.}

What was encouraging about REAP-720 in these checks:

• It retained fully runnable Python generation across data structures, concurrency, edge cases, and strict output formatting after 19.64% routed-expert removal.
• Its aquarium was the more ambitious desktop illustration, with notably better fish, plants, water treatment, rotating triangular glass fragments, and settled debris.
• Its aquarium implementation was cleaner at runtime than the API sample and narrowly won the combined review despite losing the visual-fidelity subscore.

Known REAP aquarium limitations remain: Reset does not fully restore the ambient clock, fish, and bubbles; a live resize does not refresh the renderer dimensions/pixel ratio; the crack can escape the glass; and the mobile composition is weaker. These observations do not change the QUALITY GATE NOT VALIDATED verdict above.

LiveCodeBench (date-windowed direct-code result)

Kimi-K3-REAP-720-MXFP4: 78.85% pass@1 (41/52).

Evaluated on LiveCodeBench release_v6 (code generation), restricted to problems released 2025-03-15 to 2025-04-30 (52 tasks), with greedy decoding (temperature=0, n=1), a 28,000-token generation cap, served with vLLM and scored through the official LiveCodeBench OpenAI-compatible-endpoint harness. This is a single-sample, non-agentic, direct code-generation result (not agentic/tool-use, self-repair, or best-of-n). The run completed all 52 tasks with 0 harness exceptions and 0 timeouts.

Setting	Value
Task	LiveCodeBench code generation
Dataset release	release_v6
Date window	2025-03-15 to 2025-04-30 (inclusive)
Evaluated tasks	52
Metric	pass@1 (greedy)
Decoding	temperature=0, n=1
Max generation length	28,000 tokens
Serving	this checkpoint on vLLM (FP8 KV)
Harness	LiveCodeBench OpenAI-compatible endpoint
Result	41/52 = 78.85%

Context: official LiveCodeBench leaderboard — a DIFFERENT, larger window

These rows are not the same 52 problems. They are the official public leaderboard's default view — 454 problems, 2024-08-01 to 2025-05-01 — recomputed from the leaderboard's own per-problem data and independently verified. They are shown only for rough context and must not be read as a head-to-head ranking against the REAP-720 number above (different task set, sampling, serving stack, and possibly harness).

Model	pass@1	Window / note	Source
O4-Mini (High)	80.2	454 problems, 2024-08 to 2025-05	LiveCodeBench leaderboard
O3 (High)	75.8	same window	LiveCodeBench leaderboard
DeepSeek-R1-0528	73.1	same window	LiveCodeBench leaderboard
Qwen3-235B-A22B	65.9	same window	LiveCodeBench leaderboard
Claude Opus 4 (Thinking)	56.6	same window	LiveCodeBench leaderboard
Claude Sonnet 4 (Thinking)	55.9	same window	LiveCodeBench leaderboard
GPT-4o (2024-08-06)	29.5	same window	LiveCodeBench leaderboard
DeepSeek-V3	27.2	same window	LiveCodeBench leaderboard
Kimi K2 Instruct	53.7	v6, "Aug 24 - May 25" (vendor self-reported)	Moonshot K2 model card

Full Kimi K3 has no published LiveCodeBench score — the official K3 card reports DeepSWE, ProgramBench, Terminal-Bench 2.1, SciCode, etc. instead — so no direct base-vs-pruned LiveCodeBench comparison is available. The REAP-720 result above is a custom recent six-week window and differs from every row in task set, sampling, serving, and possibly harness; treat it as a standalone date-windowed datapoint, not a ranking.

Sources: LiveCodeBench leaderboard and repository; Kimi K2 Instruct card.

Verified vLLM serving configuration

The checkpoint was loaded successfully on one 8× NVIDIA B200 node with the official vllm/vllm-openai:kimi-k3 image. SGLang is not validated for this derivative; only vLLM is claimed.

Short-context configuration (verified boot)

VLLM_ALLREDUCE_USE_FLASHINFER=0 vllm serve /model \
  --served-model-name kimi-k3-reap720 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --disable-custom-all-reduce \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3 \
  --max-model-len 8192 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.92 \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[1],"pass_config":{"fuse_allreduce_rms":false}}'

Observed steady memory was about 168.99 GiB per GPU. A coherent arithmetic probe and multimodal preprocessing path passed. Standard safetensors loading was used; the fastsafetensors loader exceeded available staging memory on this single-node configuration.

Concurrent throughput profile (32K context, 8 sequences — verified)

A separate concurrent-throughput profile was verified with eight concurrent real MMMU image requests. Relative to the short-context command above, the differing settings were:

  --max-model-len 32768 \
  --max-num-seqs 8 \
  --kv-cache-dtype fp8_e4m3 \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[1,2,4,8],"pass_config":{"fuse_allreduce_rms":false}}'

Measured result over a 20-second window: 9,420 generation tokens → 471.0 aggregate generation tokens/second, with 8 requests running, 0 waiting, and 0 server errors. This is aggregate concurrent throughput, not single-request decode speed.

Single-sequence mixed workload (verified)

With --max-num-seqs 1, the live vLLM Prometheus counter vllm:inter_token_latency_seconds recorded 124,017 inter-token intervals totaling 1,585.267 s across the 262K validation and GPQA workload: 12.783 ms mean inter-token latency, or 78.23 decode tokens/s. This is a Prometheus aggregate over mixed successful requests (including the 260K-context validation and the official-protocol GPQA evaluation), not a synthetic fixed-length throughput suite. The machine-readable counter snapshot is evaluation/throughput_262k.json.

Serving configuration for this measurement: tensor-parallel-size 8, max_model_len 262144, max_num_seqs 1, max_num_batched_tokens 2048, gpu_memory_utilization 0.95, kv_cache_dtype fp8_e4m3, mla_prefill_backend FLASHINFER, prefill-query quantization enabled, cudagraph_mode FULL_DECODE_ONLY.

Long context — 262,144 tokens (verified)

The same checkpoint passed a 262,144-token FP8-KV boot and a near-limit end-to-end request. Relative to the short-context command, the verified long-context settings are:

  --max-model-len 262144 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.95 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-config '{"use_prefill_query_quantization":true,"mla_prefill_backend":"FLASHINFER"}'

The request contained 260,095 API-counted prompt tokens (260,094 locally counted) plus 53 completion tokens and finished normally in 54.78 s with finish_reason="stop", returning READY. Machine-readable evidence: evaluation/long_context_262k.json.

For this B200 image, forcing the FLASHINFER MLA prefill backend is required for FP8 prefill-query quantization; the auto-selected FlashAttention MLA prefill backend does not support that path.

Provenance and reproducibility

Property	Value
Source model	moonshotai/Kimi-K3
Source revision verified	false
Public parent revision observed 2026-08-01	9f62e4e9fffbd0a83ddd60e1c209d828994b3569 (observational only)
Structural config verified	text_config.num_experts=720; 93 hidden layers; KimiK3ForConditionalGeneration
Vision tower modified	false
MM projector modified	false

Revision caveat. The derivative config contains no embedded commit hash or source revision. Do not claim the build source is pinned to the observed public parent revision above; source_revision_verified is false. That revision is recorded only as the public parent state observed on 2026-08-01.

The pruning procedure is recorded in REAP_PRUNING.json (schema version 1, created_utc 2026-08-01), including SHA-256 hashes of the scripts and reference baselines for reproducibility:

Artifact	SHA-256
reap_prune_apply.py	c80a94dff3d4f8af6ab683661f849c96c8b22c94ac55bf0080100b69920814c2
k3_stream_saliency.py	5bb8588297893965a0ead68d531bff759be051830bdea7e4fa06ad0fe647d94c
k3_instrument_moe.py	330cce517022aae5979a0390cb2f7eda73f40dcadef989b7130d449428f66592
k3_calibration_multidomain.py	e802baf7377d0eef55b73bf8305e73708e0444fbec9f071817bb962e2a678bab
apply_k3_patches.py	5858e092cc890d21ca03c0e934ada21adfb08b91248564dc7823757687909e05
eval_reap.py	40430141e8c413dad4f11a26dff16591b2dfd5c4a465766d7d7a20633550ee3b
reap_baselines.json	fcd3338b3b1d651477d59a86e606f70e67eb22ca97be2a4e615acd828bb87f95

The published reap_tools/eval_reap.py bytes hash to the value shown above, and that value matches REAP_PRUNING.json. A later unpublished local experiment increased the vision output budget to 32,768 tokens, so it has different evaluator bytes; no result from that modified evaluator is represented as a completed benchmark in this repository.

Recovered expert keep map

The exact retained-expert map is published as:

• provenance/kimi_k3_reap720_keep_map.json — human-readable artifact with per-layer new→old and old→new mappings, layer hashes, shard names, and validation statistics.
• provenance/kimi_k3_reap720_keep_map.bin — canonical row-major little-endian uint16 map with shape [92, 720].
• provenance/keepmap_validation.json — independent structural validation summary.

Recovery method: for each of the 92 MoE layers, every retained BF16 router row in gate.weight was matched byte-exactly against the 896 parent rows at the observed public parent revision. All 66,240 retained rows had exactly one parent match, were strictly ascending, and were injective. The corresponding mapped F32 e_score_correction_bias entries were then verified byte-for-byte for all 66,240 mappings. This establishes an empirical match to public parent revision 9f62e4e9fffbd0a83ddd60e1c209d828994b3569; it does not establish build-time source attestation, so source_revision_verified remains false.

Keep-map artifact property	Value
MoE layers	92
Retained mappings	66,240
Bias entries verified	66,240
Binary SHA-256	c08751b82b0eabb4c09edfc5829510e7cd27e9978685fe9313dd76fb53293703
Canonical JSON SHA-256	d927b83a44562bad6e080bf8133a2e1a6ec053fe57c3f8f8163064b86c3774ca
Binary encoding	row-major little-endian uint16, shape [92, 720]

Intended use and limitations

This checkpoint is intended primarily for coding, tooling, native vision, and agentic workflows where local deployment, long context, and practical tool interaction matter. It preserves Kimi K3's multimodal architecture while reducing resident weight storage enough for a single 8×B200 deployment. The project's full-split GPQA-Diamond result of 155/198 = 78.28% (approximately 78.4% in the project summary) is accepted as a practical reasoning datapoint for these intended workloads.

Expert removal can change routing behavior and may degrade particular languages, domains, visual tasks, long-context behavior, or agent trajectories even when short tests look normal. Run task-specific acceptance tests before production use, especially for the coding repositories, tools, images, and agent loops that matter to your deployment.

This repository does not claim safety improvements, alignment changes, or a new training procedure. The upstream Kimi K3 model limitations continue to apply.

License and attribution

The checkpoint and copied upstream code are provided under the Kimi K3 License. This derivative retains Moonshot AI's copyright and license notice. Users are responsible for reviewing the license, including its conditions for certain large commercial products and Model-as-a-Service businesses.

Upstream project and model authors: MoonshotAI/Kimi-K3 and moonshotai on Hugging Face.