K2-Horizon-375B-A23B

K2-Horizon-375B-A23B is the flagship of the K2-Horizon family: a sparse Mixture-of-Experts model that stores 375B parameters and runs 23B per token, with a 512K context window. We have released the final checkpoint; intermediate checkpoints, along with the data and the training code, will be released.

K2-Horizon-375B-A23B Highlights

• Frontier-class agentic performance. On agentic tool use, terminal, and long-horizon workflow benchmarks it matches or beats open-weight MoE models up to 2.6× its size and is competitive with closed frontier models (see Benchmark Results).
• 512K context. Native 524,288-token context from the midtraining stages onward.
• Intermediate checkpoints. Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
• Fully open. Training data/recipe and the training code will be made public.

Benchmark Results

		Open-weight models				Closed models
	K2-Horizon-375B-A23B	Nemotron 3 Ultra	Inkling (xhigh)	MiniMax-M3	GLM 5.2 (max)	GPT 5.6 Luna (max)	GPT 5.6 Terra (high)	Claude Sonnet5 (max)
# Params	375B	550B	975B	428B	753B	--	--	--
# Activated params	23B	55B	41B	23B	40B	--	--	--
Architecture	MoE	MoE	MoE	MoE	MoE	Closed	Closed	Closed
Agents
GDPVal-AA Real-world professional tasks (Elo)	1,441	1,162	1,234	1,380	1,498	1,569	1,503	1,584
tau3-Banking Agentic tool use	34.0	14.2	29.1	15.3	34.6	31.1	28.7	37.3
Toolathlon Verified Agentic tool use	65.3	34.3	45.5	53.7	59.9	67.5	64.8	71.6
Automation Bench Public Workflow automation	25.3	8.0	12.8	20.5	26.2	33.5	28.0	34.7
Apex-Agents (pass@1) Long-horizon professional workflows	24.8	9.0	19.0	23.8	26.9	28.6	25.4	31.7
MCPMark MCP tool use	67.7	45.7	51.2	48.8	72.4	66.9	74.0	65.3
BrowseComp Deep web research	72.8	44.4	77.1	83.5	--	83.3	--	84.7
WildClawBench In-the-wild agentic tasks	50.9	34.2	52.3	56.4	55.0	50.4	60.0	--
Coding
Terminal-Bench 2.1 Agentic terminal use	70.2	53.9	55.1	65.2	77.9	80.9	75.7	80.5
SciCode Scientific coding	42.7	39.9	46.1	45.4	50.5	52.5	50.1	53.6
SWE-Atlas-QnA (strict) Repo-level code Q&A	48.4	--	25.5	42.3	46.4	--	--	--
SWE Bench Pro (strict) Software engineering	42.6	38.7	43.1	43.8	46.7	48.8	--	--
Scientific Reasoning
Humanity's Last Exam (without tools) Expert-level reasoning	32.0	28.4	31.9	39.0	41.1	39.5	38.5	41.3
GPQA Diamond Graduate-level science QA	87.3	86.7	87.2	92.9	89.5	91.1	89.6	91.1
CritPt Frontier physics reasoning	8.6	3.1	5.4	3.7	20.9	21.0	22.9	16.9
General
AA-LCR Long-context reasoning	76.0	71.0	73.3	80.3	76.7	78.3	73.3	77.0
AA-Omniscience Accuracy Factual accuracy	23.0	23.0	42.0	17.0	24.0	43.0	45.0	40.0
AA-Omniscience Non-Hallucination Non-hallucination rate	74.7	70.0	32.0	82.0	74.0	7.0	10.0	61.0

Scores in %. SWE-Atlas-QnA and SWE Bench Pro: strict = no internet. BrowseComp: different models use different harness, we use the Discard-all@95k context length proposed in DeepSeek-V3.2 technical report. WildClawBench: we use a subset of the English text-only-modality tasks. Apex-Agents: we use a subset of text-only-modality tasks. GDPVal-AA is the Elo rating.

Quickstart

Serving

vLLM, recipe at recipes.vllm.ai/IFM:

vllm serve IFM/K2-Horizon-375B-A23B \
  --revision main \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --trust-remote-code \
  --dtype bfloat16 \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --enable-auto-tool-choice

Use an exact branch name from the inventory with vLLM's --revision option. For example, --revision pretrain_ph1_211000 selects the final checkpoint of Pretraining Phase 1, at step 211,000.

SGLang recipe validated on 8× H200 in the SGLang K2 Horizon cookbook:

python3 -m sglang.launch_server \
  --model-path IFM/K2-Horizon-375B-A23B \
  --revision main \
  --tp 8 \
  --ep 8 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --model-loader-extra-config '{"enable_multithread_load":false}' \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --host 0.0.0.0 --port 30000

API Usage

Recommended settings: reasoning_effort="high", temperature=1.0, top_p=0.95. Reasoning depth is selected per request through chat_template_kwargs. Thinking is returned in reasoning_content and the answer in content.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-375B-A23B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Our model supports multiple tool calls formats, which can be changed with chat_template_kwargs. The supported values are json, xml, and xml_typed . The default is xml. Keep --tool-call-parser k2_horizon enabled to parse the selected format.

Transformers

Validated with Transformers 4.57.6, PyTorch 2.13.0, Safetensors 0.8.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-375B-A23B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Overview

The table below lists the training stages in order and the purpose of each stage.

Training steps are counted within each stage or phase. Token budgets cover only the additional training in that stage or phase. For example, the 8T tokens listed for Pretraining Phase 2 are additional to the 7T tokens in Phase 1, bringing the cumulative budget to 15T tokens by the end of Phase 2. Here, B and T denote billion and trillion tokens, respectively.

Each stage or phase continues from the final checkpoint of the preceding stage or phase. During RL, training branches into five expert models, which are then merged, as described below.

Some stages, such as SFT, have multiple phases with slight changes to the data mix while retaining the same overall purpose.

Training stage	Training steps	Training tokens	Sequence length	Purpose
Pretraining — Phase 1	211000	7T	8K	Pretraining.
Pretraining — Phase 2	240000	8T	8K	Continued pretraining from Phase 1, with newer and better data.
Midtraining — Stage 1	32500	1T	32K	Context extension.
Midtraining — Stage 2	15000	500B	128K	Context extension.
Midtraining — Stage 3	3500	120B	512K	Context extension.
Midtraining — Stage 4	6000	200B	512K	Continued context extension from Stage 3, with the data mix shifted toward agentic and reasoning SFT data.
RL	To be updated	To be updated	512K	We trained five expert models from the final checkpoint of Midtraining Stage 4: knowledge work, IF, search, tool use, and reasoning. We then merged the expert models.
SFT — Phase 1	2400	80B	512K	SFT for better domain coverage, starting from the merged RL checkpoint.
SFT — Phase 2	6000	200B	512K	Continued from Phase 1 with nearly the same data mix.
SFT — Phase 3	1500	50B	512K	SFT on a high-quality subset of the data used in Phases 1 and 2, with learning rate decay.

RL training steps and token counts are not reported here, so a cumulative token total that includes RL is not provided.

Release Artifacts

The tables below list the release artifacts for K2-Horizon-375B-A23B, their availability, and the expected release dates for remaining items.

Last updated: 2026-09-10

Status:

• Available — fully released for the scope listed;
• Partial — some items are available, with remaining items listed in the notes;
• In Progress — being prepared for release but not yet available.

Artifact Index

Artifact	Link	Status	Remaining items / expected availability
Model card	Hugging Face	Available	N/A
Training logs	W&B	Available	N/A
Blog post	Blog post	Available	N/A
Checkpoints	Checkpoint inventory	Partial	See details below
Technical report	Not yet available	In Progress	End of September 2026
Code repository	GitHub	In Progress	End of September 2026

Checkpoint Inventory

Model repository: IFM/K2-Horizon-375B-A23B

Branch names below refer to this repository. Patterns containing * group branches by training stage or phase. The * is a placeholder for a training-step number, not a literal branch name. Intermediate checkpoint groups exclude the final checkpoint listed separately; a pattern does not imply that a checkpoint is available at every step.

For example, pretrain_ph1_211000 is the checkpoint saved at training step 211,000 within Pretraining Phase 1, and is the final checkpoint of that phase. The numeric suffix is the step within the named stage or phase, not the cumulative step across all training. Thus, pretrain_ph2_240000 refers to step 240,000 within Pretraining Phase 2.

For a partially released group, the available checkpoints and the remaining checkpoints are listed in the notes.

Checkpoint	Branch / repository	Status	Remaining items / expected availability
Pretrain Phase 1 Intermediate Checkpoints	pretrain_ph1_*	Available	N/A
Pretrain Phase 1 Final Checkpoint	pretrain_ph1_211000	Available	N/A
Pretrain Phase 2 Intermediate Checkpoints	pretrain_ph2_*	Available	N/A
Pretrain Phase 2 Final Checkpoint	pretrain_ph2_240000	Available	N/A
Midtrain Stage 1 Intermediate Checkpoints	mid_1_*	Available	N/A
Midtrain Stage 1 Final Checkpoint	mid_1_32500	Available	N/A
Midtrain Stage 2 Intermediate Checkpoints	mid_2_*	Available	N/A
Midtrain Stage 2 Final Checkpoint	mid_2_15000	Available	N/A
Midtrain Stage 3 Intermediate Checkpoints	mid_3_*	Available	N/A
Midtrain Stage 3 Final Checkpoint	mid_3_3500	Available	N/A
Midtrain Stage 4 Intermediate Checkpoints	mid_4_*	Available	N/A
Midtrain Stage 4 Final Checkpoint	mid_4_6000	Available	N/A
RL Knowledge Work Expert Checkpoint	rl_knowledge_work	In Progress	Mid-September 2026
RL IF Expert Checkpoint	rl_if	In Progress	Mid-September 2026
RL Search Expert Checkpoint	rl_search	In Progress	Mid-September 2026
RL Tool Use Expert Checkpoint	rl_tool_use	In Progress	Mid-September 2026
RL Reasoning Expert Checkpoint	rl_reasoning	In Progress	Mid-September 2026
RL Merged Final Checkpoint	rl_merged	Available	N/A
SFT Phase 1 Intermediate Checkpoints	sft_1_*	Available	N/A
SFT Phase 1 Final Checkpoint	sft_1_2400	Available	N/A
SFT Phase 2 Intermediate Checkpoints	sft_2_*	Available	N/A
SFT Phase 2 Final Checkpoint	sft_2_6000	Available	N/A
SFT Phase 3 Intermediate Checkpoints	sft_3_*	Available	N/A
SFT Phase 3 Final Checkpoint	sft_3_1500	Available	N/A

Best Practices

1. Reasoning effort: always high. All reported results use high reasoning effort. Pass {"chat_template_kwargs": {"reasoning_effort": "high"}} on every request.
2. Sampling parameters. temperature=1.0, top_p=0.95.
3. Serving. Use the validated SGLang recipe above: BF16, TP=8 on one 8× H200 node, FlashAttention-3, with multithreaded weight loading disabled. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the SGLang cookbook and the vLLM recipe.
4. Parsers. Enable the k2_horizon reasoning parser for chat, and add the k2_horizon tool-call parser for agent use. Leave both off for plain completion-style generation.

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}