--- pipeline_tag: text-generation library_name: transformers model_name: K2-Horizon-MoVA-36B-A4B language: - en license: apache-2.0 datasets: - IFM/K2-Horizon-Pretrain-Data - IFM/K2-Horizon-Midtrain-Data tags: - k2-horizon - 36b - mova - moe - open-weights - ifm --- # K2-Horizon-MoVA-36B-A4B-GGUF > [!NOTE] > This repository contains GGUF versions of the [IFM/K2-Horizon-MoVA-36B-A4B](https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B) for use with `llama.cpp`. > > The model tensors are stored in their original BF16 precision. The GGUF files include the tokenizer metadata and a `llama.cpp`-compatible chat template. > > **Compatibility:** These models require a version of `llama.cpp` containing K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is in https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family: a Mixture-of-Experts model with Mixture-of-Values attention (MoVA) that stores 36B parameters and runs 4B per token. We have released the final checkpoint; intermediate checkpoints, along with the data and the training code, will be released.

K2-Horizon-MoVA-36B-A4B benchmark results against open MoE, dense, and closed models

## K2-Horizon-MoVA-36B-A4B Highlights - **Frontier-class results at 4B active parameters.** On agentic and reasoning benchmarks it outscores open weight dense (approximately 30B model size) and MoE models up to 15× its size; and also performs competitively against closed frontier models (see [Benchmark Results](#benchmark-results)). - **512K context.** Native 524,288-token context from the midtraining stages onward. - **Intermediate checkpoints.** Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint. - **Fully open.** Training data/recipe and the training code will be made public. ## Benchmark Results
Open-weight models
K2-Horizon-MoVA-36B-A4BNemotron 3 UltraNemotron 3 SuperG9v3-39A5BQwen3.6-35B-A3BMuse Glimmer-30BGemma 4 31B-it
# Params36B550B120B39B35B30B31B
# Activated params4B55B12B5B3B30B31B
ArchitectureMoEMoEMoEMoEMoEDenseDense
Agents
tau3-Banking
Agentic tool use
26.814.210.322.19.323.514.8
Coding
Terminal-Bench 2.1
Agentic terminal use
58.653.938.632.644.951.743.4
SciCode
Scientific coding
38.939.936.034.035.843.643.4
Scientific Reasoning
Humanity's Last Exam (without tools)
Expert-level reasoning
25.228.420.817.522.222.023.6
GPQA Diamond
Graduate-level science QA
80.886.780.080.584.183.585.7
CritPt
Frontier physics reasoning
2.13.13.10.30.32.61.4
General
AA-LCR
Long-context reasoning
66.371.060.362.066.780.068.3
AA-Omniscience Accuracy
Factual accuracy
18.822.624.314.918.827.020.0
AA-Omniscience Non-Hallucination
Non-hallucination rate
69.270.313.087.049.518.115.0

Scores in %. Bold marks the best score in each row. Sections follow the Artificial Analysis Intelligence Index categories. Baseline scores are from Artificial Analysis; Muse Glimmer-30B at high reasoning effort, all other open models in their reasoning mode.

## Quickstart ### Serving vLLM, recipe at [recipes.vllm.ai/IFM](https://recipes.vllm.ai/IFM): ```shell vllm serve IFM/K2-Horizon-MoVA-36B-A4B \ --revision main \ --tensor-parallel-size 2 \ --enable-expert-parallel \ --trust-remote-code \ --dtype bfloat16 \ --max-model-len 131072 \ --reasoning-parser k2_horizon \ --tool-call-parser k2_horizon \ --enable-auto-tool-choice ``` SGLang recipe validated on 2× H200 in the [SGLang K2 Horizon cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon): ```shell python3 -m sglang.launch_server \ --model-path IFM/K2-Horizon-MoVA-36B-A4B \ --revision main \ --tp 2 \ --ep 2 \ --dtype bfloat16 \ --attention-backend fa3 \ --json-model-override-args '{"xllm_source_router_gemm_partitions":2}' \ --reasoning-parser k2_horizon \ --tool-call-parser k2_horizon \ --host 0.0.0.0 --port 30000 ``` ### API Usage > [!Tip] > Recommended settings: `reasoning_effort="high"`, `temperature=1.0`, `top_p=0.95`. > Reasoning depth is selected per request through `chat_template_kwargs`. Thinking is returned in `reasoning_content` and the answer in `content`. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY") response = client.chat.completions.create( model="IFM/K2-Horizon-MoVA-36B-A4B", messages=[{"role": "user", "content": "Explain the result step by step."}], temperature=1.0, top_p=0.95, max_tokens=32768, extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}}, ) message = response.choices[0].message print("Reasoning:", getattr(message, "reasoning_content", None)) print("Answer:", message.content) ``` Our model supports multiple tool calls formats, which can be changed with `chat_template_kwargs`. The supported values are `json`, `xml`, and `xml_typed` . The default is `xml`. Keep `--tool-call-parser k2_horizon` enabled to parse the selected format. ### Transformers Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0. ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "IFM/K2-Horizon-MoVA-36B-A4B" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True ) inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device) inputs.pop("token_type_ids", None) outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` ## Best Practices 1. **Reasoning effort: always `high`.** All reported results use high reasoning effort. Pass `{"chat_template_kwargs": {"reasoning_effort": "high"}}` on every request. 2. **Sampling parameters.** `temperature=1.0`, `top_p=0.95`. 3. **Serving.** Use the validated SGLang recipe above: BF16, TP=2, FlashAttention-3, and the `xllm_source_router_gemm_partitions` override, which preserves the checkpoint's router numerics. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon) and the [vLLM recipe](https://recipes.vllm.ai/IFM). 4. **Parsers.** Enable the `k2_horizon` reasoning parser for chat, and add the `k2_horizon` tool-call parser for agent use. Leave both off for plain completion-style generation. ## Citation ```bibtex @misc{k2horizon2026, title = {Introducing K2 Horizon: Frontier Performance, Radically Open}, author = {{IFM Team}}, year = {2026}, url = {https://ifm.ai/blog/k2/}, } ```