Hugging Face's logo

IFM
/
K2-Horizon-375B-A23B-FP8

Text Generation
Transformers
Safetensors
English
k2_horizon
k2-horizon
375b
Mixture of Experts
open-weights
ifm
conversational
custom_code
compressed-tensors
Model card Files
xet
 Community
1
LiqunMa commited on
Commit
4068e80
·
verified ·
1 Parent(s): c358c11

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -471
README.md CHANGED
@@ -34,518 +34,117 @@ tags:
34
  > **Serving note**: the routed experts' intermediate size (1792) is not splittable into whole 128-wide quantization blocks at the usual tensor-parallel sizes (TP=4, TP=8), so expert parallelism is required.
35
 
36
 
 
37
 
38
- K2-Horizon-375B-A23B is a flagship Mixture-of-Experts base model in the IFM K2-Horizon family.
 
 
39
 
40
- K2-Horizon is an open-weight IFM model family built for transparent foundation-model research, staged checkpoint analysis, and practical deployment. The series spans compact dense models for local experimentation, dense mid-size and large models for high-quality research workloads, and sparse expert models for higher-capacity serving.
41
 
42
- All model cards in this family intentionally share the same structure and family-level reference material. Each repository focuses on one model size, but the card also describes the full release series so users can compare architectures, context length, serving requirements, and benchmark behavior before selecting a model.
 
 
 
43
 
44
- The repositories provide base/pretraining and staged midtraining revisions, making it possible to study capability changes across training phases instead of evaluating only one checkpoint. The `main` branch contains the default checkpoint, while stable stage tags such as `base_final`, `mid_1_final`, and `mid_4_final` identify specific model states for reproducible use.
45
 
 
 
 
46
 
47
- ## Model Series Overview
48
 
49
- K2-Horizon is released as a family of checkpoints that share the same post-training recipe, release process, and user-facing deployment guidance while covering dense, MoE, and MoVA configurations. The table below summarizes the public family view; individual repositories contain the files and branches for one model size.
50
 
51
- | Model | Architecture | Stored parameters | Active parameters/token | Context length | Vocab size | Intended use |
52
- | --- | --- | --- | --- | --- | --- | --- |
53
- | K2-Horizon-0.9B | K2HorizonForCausalLM; dense decoder-only | 0.9B | 0.9B | 8,192 | 64,256 | Lightweight experimentation, local development, evaluation dry runs |
54
- | K2-Horizon-3.7B | XllmForCausalLM; dense decoder-only | 5.06B | 5.06B | 524,288 | 250,624 | Efficient research, evaluation, and single-node serving |
55
- | K2-Horizon-7B | XllmForCausalLM; dense decoder-only | 9.00B | 9.00B | 524,288 | 250,624 | General research, fine-tuning, and cost-conscious deployment |
56
- | K2-Horizon-32B | K2HorizonForCausalLM; dense decoder-only | 34.78B | 34.78B | 524,288 | 250,624 | Stronger long-context and reasoning experiments |
57
- | K2-Horizon-36B | K2HorizonForCausalLM; MoVA + MoE | 37.44B | 5.95B | 524,288 | 250,624 | Multistage research and production-style serving experiments |
58
- | K2-Horizon-375B-A23B | XllmForCausalLM; sparse MoE | 379.17B | 23B | 524,288 | 250,624 | Frontier-scale open research, long-context evaluation, and high-capacity serving |
59
 
60
- Tokenizer files are distributed with each model repository. The compact K2-Horizon-0.9B model comes from the IFM Mobile line and uses a 64,256-token vocabulary. The 3.7B, 7B, 32B, 36B, and 375B models use a 250,624-token vocabulary.
61
 
62
- The native 0.9B `main`, 32B `mid_4`, and 36B `main`/`mid_4` revisions publish `K2HorizonForCausalLM`, `model_type: k2_horizon`, and matching `configuration_k2_horizon.py` and `modeling_k2_horizon.py` modules. The deployment preflights below validate that public contract before loading weights.
63
-
64
- ## This Repository
65
-
66
- | Field | Value |
67
- | --- | --- |
68
- | Model name | K2-Horizon-375B-A23B |
69
- | Repository | IFM/K2-Horizon-375B-A23B |
70
- | Model family | K2-Horizon |
71
- | Architecture class | XllmForCausalLM |
72
- | Classification | MoE |
73
- | Stored parameters | 376.09B core; 379.17B including embeddings |
74
- | Active parameters per token | 23.59B core; 23B including embeddings |
75
- | Context length | 512K tokens |
76
- | Transformer layers | 61 |
77
- | Hidden size | 6,144 |
78
- | Dense / expert intermediate size | 16,384 / 1,792 |
79
- | Attention heads / KV heads | 48 / 8 |
80
- | Dense attention / sparse attention blocks | 61 / 0 |
81
- | Dense FFN / sparse FFN blocks | 3 / 58 |
82
- | MoE experts / active experts | 192 / 8 |
83
- | MoVA count / active values | 0 / 0 |
84
- | Vocab size | 250,624 |
85
- | Tokenizer metadata | Hugging Face tokenizer files; vocab size 250,624 |
86
- | Checkpoint storage dtype | float32 (FP32) |
87
- | Checkpoint config dtype metadata | float32 |
88
- | Evaluation and sample dtype | bfloat16 (BF16) |
89
- | Shard/index summary | 61 shards |
90
-
91
- ## Checkpoint Revisions
92
-
93
- Each repository uses branches for model stages and stable tags for specific checkpoints. Pin a revision tag when reproducibility matters; use `main` for the default model checkpoint.
94
-
95
- | Stage | Branch | Revision tag | Max context | Description |
96
- | --- | --- | --- | --- | --- |
97
- | Base / pretraining | pretrain | base_final | 8K tokens | Base model checkpoint |
98
- | Midtraining 1 | mid_1 | mid_1_final | 32K tokens | Midtraining 1 checkpoint |
99
- | Midtraining 2 | mid_2 | mid_2_final | 128K tokens | Midtraining 2 checkpoint |
100
- | Midtraining 3 | mid_3 | mid_3_final | 512K tokens | Midtraining 3 checkpoint |
101
- | Midtraining 4 | mid_4 | mid_4_final | 512K tokens | Midtraining 4 checkpoint |
102
- | Reinforcement Learning and Model Merging | rl_and_merge | rl_and_merge_final | 512K tokens | Reinforcement learning and model merging checkpoint |
103
- | Supervised Fine-tuning | sft | sft_final | 512K tokens | SFT checkpoint |
104
-
105
- Example revision pinning:
106
-
107
- <details>
108
- <summary>Show revision pinning example</summary>
109
-
110
- ```python
111
- # Validated with Python 3.14,
112
- # Transformers 4.57.6, and
113
- # Safetensors 0.8.0.
114
- from transformers import AutoModelForCausalLM, AutoTokenizer
115
-
116
- model_id = "IFM/K2-Horizon-375B-A23B"
117
- revision = "mid_4_final"
118
-
119
- tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision, trust_remote_code=True)
120
- model = AutoModelForCausalLM.from_pretrained(
121
- model_id,
122
- revision=revision,
123
- device_map="auto",
124
- dtype="bfloat16",
125
- low_cpu_mem_usage=True,
126
- trust_remote_code=True,
127
- )
128
- ```
129
-
130
- </details>
131
-
132
-
133
-
134
- ## Training Provenance
135
-
136
- Each row identifies the last W&B process record associated with the selected checkpoint, together with its Slurm job and exact xLLM source revision. A training stage can span multiple W&B process records when checkpointed training is resumed.
137
-
138
- | Stage | Revision | Checkpoint context | xLLM source | W&B last run | Slurm job | Training nodes |
139
- | --- | --- | --- | --- | --- | --- | --- |
140
- | Base / pretraining | `base_final` | 8K tokens | [`ad6dac16`](https://github.com/LLM360/xllm/commit/ad6dac16031ae74d3e02c10bf3f37384815503a2) | [`k2-moe-pretrain/atini4yn`](https://wandb.ai/mbzuai-llm/k2-moe-pretrain/runs/atini4yn) | `1455201` | 256 |
141
- | Midtraining 1 | `mid_1_final` | 32K tokens | [`b69a6b2e`](https://github.com/LLM360/xllm/commit/b69a6b2ed17091dfee8af68a570f78816f765bad) | [`k2-moe-midtrain/rmd115xs`](https://wandb.ai/mbzuai-llm/k2-moe-midtrain/runs/rmd115xs) | `1704859` | 256 |
142
- | Midtraining 2 | `mid_2_final` | 128K tokens | [`9ab31836`](https://github.com/LLM360/xllm/commit/9ab3183694cbf36cf77a5c60761576435f550433) | [`k2-moe-midtrain/wtibunwh`](https://wandb.ai/mbzuai-llm/k2-moe-midtrain/runs/wtibunwh) | `1745961` | 256 |
143
- | Midtraining 3 | `mid_3_final` | 512K tokens | [`9ab31836`](https://github.com/LLM360/xllm/commit/9ab3183694cbf36cf77a5c60761576435f550433) | [`k2-moe-midtrain/smuaryfc`](https://wandb.ai/mbzuai-llm/k2-moe-midtrain/runs/smuaryfc) | `1757765` | 256 |
144
- | Midtraining 4 | `mid_4_final` | 512K tokens | [`5494c84c`](https://github.com/LLM360/xllm/commit/5494c84cc7a2a66e5d6439bf9aa07454097ce9be) | [`k2-moe-midtrain/9t42kr7w`](https://wandb.ai/mbzuai-llm/k2-moe-midtrain/runs/9t42kr7w) | `1826968` | 256 |
145
-
146
-
147
-
148
- ## Training Loss
149
-
150
- The chart reconstructs the release checkpoint lineage for Pretraining, Midtraining 1, Midtraining 2, Midtraining 3, Midtraining 4. Training loss is the next-token prediction objective; lower values indicate that the model assigns greater probability to the observed next token. Training loss is not a validation score and should not be compared across different tokenizers or data mixtures without additional controls.
151
-
152
- ![K2-Horizon-375B-A23B training loss versus cumulative training tokens](./assets/k2-horizon-375b-training-loss-vs-tokens.png)
153
-
154
- Each panel combines the W&B process records that contribute to the release checkpoint. Faint lines show sampled raw training loss; bold lines show an unpadded centered moving average with a raised-cosine taper over the final half-window. The final sample retains 20% of its base weight, limiting right-edge bias without hiding the raw tail. Checkpoint restart overlap is replaced by the later process record.
155
-
156
-
157
- ## Deployment Guide
158
-
159
- These are base checkpoints. They can be served for research, evaluation, retrieval-augmented generation experiments, tool-use research, and downstream adaptation. They are not instruction-aligned assistants by default, so production deployments should add application-level safety filters, system prompting, and task-specific validation.
160
-
161
- All reported K2-Horizon evaluation used BF16. Some released checkpoint shards and configuration metadata remain FP32; the examples below explicitly load or serve them as BF16 to match the evaluation runtime and reduce accelerator memory usage.
162
-
163
- ### Validated Independent Dependency Versions
164
-
165
- Use a separate Python environment for every workflow. Direct Transformers loading, xLLM conversion, vLLM, and the pinned SGLang production revision have independent dependency contracts. Do not install one row into another row's environment. Versions intentionally differ when a framework or remote-model contract requires them.
166
-
167
- | Workflow | Python | PyTorch | CUDA | Transformers | Safetensors | Attention kernel | Framework revision |
168
- | --- | --- | --- | --- | --- | --- | --- | --- |
169
- | Direct Transformers loading | 3.14 | 2.13.0 | 12.6 | 4.57.6 | 0.8.0 | PyTorch SDPA 2.13.0 | Transformers 4.57.6 remote code |
170
- | xLLM checkpoint conversion | 3.12 | 2.8.0 | 12.8 | 4.57.3 | 0.8.0 | FlashAttention 2.8.3.post1 | xLLM `7cfa3b2c` |
171
- | vLLM serving | 3.12 | 2.13.0 | 12.9 | 5.16.1 | 0.8.0 | FlashInfer 0.6.17 | vLLM 0.26.1rc1.dev1212 source `d9fd5f11` (PR #53806) |
172
- | SGLang serving | 3.12 | 2.9.1 | 12.8 (CUDA Python 12.9) | 5.3.0 | 0.8.0 | FA3 (sglang-kernel 0.4.1); FlashInfer 0.6.7.post2 available | SGLang `ffb41d77` (stable-only metadata) |
173
-
174
- ### Inference Parameter Starting Points
175
-
176
- K2-Horizon does not use one family-wide sampling preset. The values below are operational baselines: temperature 0 is the validated deterministic serving comparison, while sampled settings are conservative starting points that should be tuned on application prompts. Benchmark reproduction must use the decoding configuration attached to that benchmark.
177
-
178
- | Use case | Temperature | Top-p | Max new tokens | Notes |
179
- | --- | --- | --- | --- | --- |
180
- | Checkpoint smoke test and deterministic evaluation | 0.0 | 1.0 | 512-4,096 | Greedy decoding is the reproducible baseline used by serving comparisons. |
181
- | Base-model continuation | 0.6 | 0.95 | 1,024-4,096 | A conservative sampling baseline; tune on a held-out prompt set. |
182
- | Long-context continuation or summarization | 0.2 | 0.9 | 2,048-8,192 | Start with one request and increase context only after KV-cache capacity checks. |
183
- | Reasoning and tool-call validation | 0.0 | 1.0 | 4,096 | Use a compatible chat template and the K2Horizon parser stack below. |
184
-
185
- ### Transformers
186
-
187
- The selected default checkpoint is loaded with Transformers 4.57.6 and PyTorch 2.13.0 on CUDA 12.6. Select the corresponding PyTorch build for your accelerator when that CUDA runtime is unavailable. The selected checkpoint uses the self-contained legacy `XllmForCausalLM` remote implementation. Its saved Transformers 4.57.3 contract is compatible with stable 4.57.6, PyTorch SDPA, and no external xLLM or FlashAttention package. xLLM conversion remains isolated below because its native build has a different runtime contract.
188
-
189
- <details>
190
- <summary>Show Transformers installation commands</summary>
191
-
192
- ```bash
193
- python -m pip install "torch==2.13.0" \
194
- --index-url "https://download.pytorch.org/whl/cu126"
195
- python -m pip install \
196
- "transformers==4.57.6" \
197
- "safetensors==0.8.0" \
198
- "accelerate==1.14.0"
199
- ```
200
-
201
- </details>
202
-
203
- <details>
204
- <summary>Show Transformers inference example</summary>
205
-
206
- ```python
207
- from transformers import AutoModelForCausalLM, AutoTokenizer
208
-
209
- MODEL_ID = "IFM/K2-Horizon-375B-A23B"
210
- REVISION = "mid_4"
211
-
212
- tokenizer = AutoTokenizer.from_pretrained(
213
- MODEL_ID,
214
- revision=REVISION,
215
- trust_remote_code=True,
216
- )
217
- model = AutoModelForCausalLM.from_pretrained(
218
- MODEL_ID,
219
- revision=REVISION,
220
- device_map="auto",
221
- dtype="bfloat16",
222
- low_cpu_mem_usage=True,
223
- trust_remote_code=True,
224
- )
225
-
226
- prompt = "Explain why long-context evaluation is difficult."
227
- inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
228
- inputs.pop("token_type_ids", None)
229
- outputs = model.generate(
230
- **inputs,
231
- max_new_tokens=512,
232
- temperature=0.7,
233
- top_p=0.9,
234
- do_sample=True,
235
- )
236
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
237
- ```
238
-
239
- </details>
240
-
241
- ### xLLM
242
-
243
- xLLM provides the training framework, checkpoint converter, serving bridges, and runtime kernels used by the XLLM-based K2-Horizon repositories. For checkpoint conversion, parity work, or loading a legacy 36B `XllmForCausalLM` stage that imports xLLM MoVA kernels, use the isolated legacy contract and reviewed bridge revision [`7cfa3b2c`](https://github.com/LLM360/xllm/commit/7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f):
244
-
245
- <details>
246
- <summary>Show xLLM setup and conversion commands</summary>
247
-
248
- ```bash
249
- git clone https://github.com/LLM360/xllm.git
250
- cd xllm
251
- git checkout 7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f
252
-
253
- python -m pip install "torch==2.8.0" \
254
- --index-url "https://download.pytorch.org/whl/cu128"
255
- export XLLM_BUILD_ROOT="${XLLM_BUILD_ROOT:-$PWD/.xllm-build}"
256
- export TMPDIR="$XLLM_BUILD_ROOT/tmp"
257
- export PIP_CACHE_DIR="$XLLM_BUILD_ROOT/pip-cache"
258
- mkdir -p "$TMPDIR" "$PIP_CACHE_DIR"
259
- python -m pip install packaging ninja
260
- python -m pip install "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post1/flash_attn-2.8.3.post1+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl"
261
- python -m pip install -r requirements.txt
262
- python -m pip install -e . --no-build-isolation \
263
- --config-settings editable_mode=compat
264
- python -m pip install \
265
- "transformers==4.57.3" \
266
- "safetensors==0.8.0" \
267
- accelerate fire
268
-
269
- python xllm_bridges/huggingface/xllm_to_hf_main.py --help
270
- ```
271
-
272
- </details>
273
-
274
- The converter requires an original xLLM distributed checkpoint and tokenizer. Released Hugging Face revisions are already converted and use the pure direct Transformers/SDPA contract above. Install xLLM only for conversion, parity work, or a model-specific native-kernel dependency. Pinning a revision does not make runtime contracts interchangeable.
275
-
276
- ### vLLM
277
-
278
- The example pins [vLLM PR #53806](https://github.com/vllm-project/vllm/pull/53806) at [`d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c). That source registers `K2HorizonForCausalLM` plus the built-in `k2_horizon` reasoning and tool parsers. The command uses vLLM's `transformers` model implementation for this checkpoint contract.
279
-
280
- <details>
281
- <summary>Show vLLM setup and serving commands</summary>
282
-
283
- ```bash
284
- git clone --filter=blob:none --no-checkout \
285
- https://github.com/vllm-project/vllm.git
286
- cd vllm
287
- git fetch origin \
288
- pull/53806/head:refs/remotes/origin/pr-53806
289
- git checkout --detach d9fd5f11423a1a5628fe29e7296ceb9de91aac3c
290
- test "$(git rev-parse HEAD)" = "d9fd5f11423a1a5628fe29e7296ceb9de91aac3c"
291
-
292
- python3.12 -m venv .venv
293
- source .venv/bin/activate
294
- python -m pip install --upgrade pip uv
295
- export UV_LINK_MODE=copy
296
- VLLM_USE_PRECOMPILED=1 uv pip install --upgrade --editable . \
297
- --torch-backend=auto
298
- uv pip install \
299
- "transformers==5.16.1" \
300
- "safetensors==0.8.0"
301
- python -m pip check
302
-
303
- python - <<'PY'
304
- from vllm import ModelRegistry
305
- from vllm.reasoning import ReasoningParserManager
306
- from vllm.tool_parsers import ToolParserManager
307
-
308
- assert "K2HorizonForCausalLM" in ModelRegistry.get_supported_archs()
309
- assert ReasoningParserManager.get_reasoning_parser("k2_horizon") is not None
310
- assert ToolParserManager.get_tool_parser("k2_horizon") is not None
311
- PY
312
-
313
- export MODEL_ID="IFM/K2-Horizon-375B-A23B"
314
- export MODEL_REVISION="mid_4"
315
- export TENSOR_PARALLEL_SIZE=8
316
- export MAX_MODEL_LEN=131072
317
-
318
- vllm serve "$MODEL_ID" \
319
- --revision "$MODEL_REVISION" \
320
  --model-impl transformers \
321
- --tensor-parallel-size "$TENSOR_PARALLEL_SIZE" \
322
  --trust-remote-code \
323
  --dtype bfloat16 \
324
- --served-model-name "$MODEL_ID" \
325
- --max-model-len "$MAX_MODEL_LEN" \
326
  --reasoning-parser k2_horizon \
327
  --enable-auto-tool-choice \
328
  --tool-call-parser k2_horizon
329
  ```
330
 
331
- </details>
332
-
333
- The source checkout is immutable at `d9fd5f11423a1a5628fe29e7296ceb9de91aac3c`. The command serves `mid_4`; the repository's longest checkpoint configuration reports 524,288 tokens, while this example starts at 131,072 tokens. Increase context and concurrency only after measuring KV-cache capacity. The preflight fails immediately if the native architecture or either Horizon parser registration is absent.
334
-
335
- ### SGLang
336
-
337
- Native K2 Horizon support is provided by
338
- [sgl-project/sglang#37654](https://github.com/sgl-project/sglang/pull/37654).
339
- Use a `lmsysorg/sglang:dev` image built after that PR is merged. Once support is
340
- included in a tagged SGLang release, use the corresponding versioned image.
341
 
342
- <details>
343
- <summary>Show the SGLang serving command</summary>
344
-
345
- ```bash
346
- docker run --gpus all \
347
- --shm-size 32g \
348
- -p 30000:30000 \
349
- -v ~/.cache/huggingface:/root/.cache/huggingface \
350
- --ipc=host \
351
- lmsysorg/sglang:dev \
352
- python3 -m sglang.launch_server \
353
- --model-path "IFM/K2-Horizon-375B-A23B" \
354
- --revision main \
355
- --tp 8 \
356
- --dtype bfloat16 \
357
- --context-length 131072 \
358
- --attention-backend fa3 \
359
- --reasoning-parser k2_horizon \
360
- --tool-call-parser k2_horizon \
361
- --model-loader-extra-config '{"enable_multithread_load":false}' \
362
- --mem-fraction-static 0.85 \
363
- --host 0.0.0.0 \
364
- --port 30000
365
  ```
366
 
367
- </details>
368
-
369
- This uses SGLang's native `K2HorizonForCausalLM` implementation; no
370
- `--trust-remote-code`, source patch, or external parser plugin is required. The
371
- 131,072-token limit is a conservative starting point; increase it only after
372
- measuring KV-cache capacity. The 375B command disables multithreaded weight loading, matching the validated loading path.
373
 
374
- ### Reasoning and Tool Parsers
375
-
376
- Parser configuration is part of the deployment contract. The canonical K2-Horizon format uses `<ifm|...>` reasoning and tool tokens; test streaming and non-streaming responses before exposing an OpenAI-compatible endpoint.
377
-
378
- | Goal | vLLM guidance | SGLang guidance | When to use |
379
- | --- | --- | --- | --- |
380
- | Plain text generation | Do not enable a reasoning or tool parser. | Do not enable a reasoning or tool parser. | Base-model evaluation, completion-style generation, and checkpoint smoke tests. |
381
- | Canonical K2-Horizon reasoning | Use the built-in `--reasoning-parser k2_horizon`. | Use `--reasoning-parser k2_horizon`. | Checkpoints and chat templates using `<ifm|think>`, `<ifm|think_fast>`, or `<ifm|think_faster>`. |
382
- | Canonical K2-Horizon tool calls | Use `--enable-auto-tool-choice --tool-call-parser k2_horizon`. | Use `--tool-call-parser k2_horizon`. | XML, typed-XML, or JSON calls wrapped in `<ifm|tool_call>`. |
383
- | Legacy K2 reasoning traces | Use the legacy parser only in a compatibility image that registers it. | Use `--reasoning-parser k2_horizon_legacy`. | Older traces using `<think>`, `<think_fast>`, or `<think_faster>` instead of canonical IFM tokens. |
384
- | Mixed tool-call corpora | Use `multi_format` only in a build that includes and tests that parser. | Use `--tool-call-parser multi_format`. | Evaluation corpora that intentionally mix K2, GLM, or JSON tool-call dialects. |
385
-
386
- PR #53806 registers both Horizon parsers in-tree; no external parser plugin is required.
387
-
388
- For OpenAI-compatible requests, select reasoning effort through chat-template kwargs:
389
-
390
- <details>
391
- <summary>Show OpenAI-compatible request example</summary>
392
 
393
  ```python
394
  from openai import OpenAI
395
 
396
- client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
397
  response = client.chat.completions.create(
398
  model="IFM/K2-Horizon-375B-A23B",
399
  messages=[{"role": "user", "content": "Explain the result step by step."}],
400
- temperature=0,
401
- max_tokens=4096,
 
402
  extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
403
  )
404
- print(response.choices[0].message)
 
 
405
  ```
406
 
407
- </details>
408
-
409
- ### Other Serving Frameworks
410
-
411
- Text Generation Inference, TensorRT-LLM, llama.cpp, Ollama, and LM Studio may require model-specific conversion, quantization, or custom architecture support. Validate tokenizer behavior, remote-code requirements, attention implementation, and long-context behavior before publishing service results.
412
-
413
-
414
- ## Hardware Planning
415
-
416
- The table below gives capacity-planning starting points for self-hosted inference. Weight memory uses the parameter count including input and output embeddings. On-disk FP32 shards and dtype conversion can require substantially more host RAM during loading than the final BF16 GPU footprint. KV-cache estimates use BF16 keys and values for one request at the smaller of the model maximum and 128K tokens; batching, allocator overhead, activations, CUDA graphs, and framework workspaces require additional memory.
417
-
418
- | Model | Stored params | Active params/token | BF16 weight memory | FP32 weight memory | BF16 KV cache / request | Starting TP | Recommended starting point |
419
- | --- | --- | --- | --- | --- | --- | --- | --- |
420
- | K2-Horizon-0.9B | 0.9B | 0.9B | 2.0 GiB | 4.0 GiB | 0.4 GiB @ 8K tokens | 1 | 1 GPU with 16 GiB is a practical starting point for the model's 8K context; allow additional memory for batching. |
421
- | K2-Horizon-3.7B | 5.06B | 5.06B | 9.4 GiB | 18.8 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 24 GiB can serve short contexts in BF16; use 48-80 GiB for larger KV caches and 128K-class requests. |
422
- | K2-Horizon-7B | 9.00B | 9.00B | 16.8 GiB | 33.5 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 40-48 GiB is suitable for short-context BF16 serving; 80 GiB is the practical starting point for 128K-class requests. |
423
- | K2-Horizon-32B | 34.78B | 34.78B | 64.8 GiB | 129.6 GiB | 32.0 GiB @ 128K tokens | 2-4 | 1x 80 GiB fits BF16 weights only at short context; start with 2x 80 GiB for 128K requests and 4x 80 GiB for larger context or throughput. |
424
- | K2-Horizon-36B | 37.44B | 5.95B | 69.7 GiB | 139.5 GiB | 24.0 GiB @ 128K tokens | 2-4 | Use at least 256 GiB of host RAM to load the current 139.5 GiB FP32 shard set. Start with 2x 80 GiB GPUs for BF16 weight sharding, or 1x H200 141 GiB for short single-request validation; use 4x 80 GiB for 128K-class requests. |
425
- | K2-Horizon-375B-A23B | 379.17B | 23B | 706.3 GiB | 1.38 TiB | 30.5 GiB @ 128K tokens | 8 H200 or 16x 80 GiB | The full 512K profile is validated on 8x H200 141 GiB GPUs at TP=8 and 0.92 memory utilization; use at least 16x 80 GiB GPUs on lower-memory hardware. |
426
-
427
- ## Evaluation
428
-
429
- Benchmark values are reported as percentages unless otherwise noted. A dash means the metric was not reported for that model or revision in the release evaluation records. The family tables below use each model's latest listed midtraining release checkpoint.
430
-
431
- ### Family Evaluation: Knowledge and Instruction Following
432
-
433
- | Model | Revision | IFEval | MMLU-Pro | MMLU | BBH |
434
- | --- | --- | --- | --- | --- | --- |
435
- | K2-Horizon-0.9B | `mid_2_final` | 43.1 | 40.3 | 44.4 | 43.8 |
436
- | K2-Horizon-3.7B | `mid_4_final` | 85.7 | 73.0 | 80.7 | 84.4 |
437
- | K2-Horizon-7B | `mid_4_final` | 89.9 | 75.7 | 83.3 | 87.8 |
438
- | K2-Horizon-32B | `mid_4_final` | 88.0 | 79.5 | 87.5 | 89.1 |
439
- | K2-Horizon-36B | `mid_4_final` | 89.6 | 79.2 | 87.3 | 88.6 |
440
- | K2-Horizon-375B-A23B | `mid_4_final` | 92.1 | 82.6 | 89.5 | 87.5 |
441
-
442
- ### Family Evaluation: STEM, Reasoning, and Coding
443
-
444
- | Model | Revision | AIME26 | GPQA-Diamond | GSM8K | MBPP |
445
- | --- | --- | --- | --- | --- | --- |
446
- | K2-Horizon-0.9B | `mid_2_final` | 12.3 | 27.8 | 53.1 | 53.2 |
447
- | K2-Horizon-3.7B | `mid_4_final` | 88.2 | 73.3 | 92.3 | 86.4 |
448
- | K2-Horizon-7B | `mid_4_final` | 87.6 | 73.9 | 95.1 | 91.0 |
449
- | K2-Horizon-32B | `mid_4_final` | 91.7 | 75.4 | 95.5 | 87.8 |
450
- | K2-Horizon-36B | `mid_4_final` | 92.7 | 75.4 | 95.2 | 87.2 |
451
- | K2-Horizon-375B-A23B | `mid_4_final` | 92.5 | 84.9 | 95.1 | 93.8 |
452
-
453
- ### Family Evaluation: Agentic and Long-Context Tasks
454
-
455
- | Model | Revision | SWE-Bench Verified (avg@3) | Terminal-Bench 2.1 (avg@3) | GDPval (acc@75%) | Tau3 Banking | BrowseComp+ (acc@1) | Apex Agents (pass@1) | WildClawBench | LCR |
456
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
457
- | K2-Horizon-7B | `mid_4_final` | 57.5 | 28.1 | 34.5 | 10.4 | 52.2 | 6.0 | 23.9 | - |
458
- | K2-Horizon-375B-A23B | `mid_4_final` | 67.8 | 38.2 | 39.6 | 11.4 | 58.1 | 13.8 | 34.8 | 56.0 |
459
-
460
- Agentic results are joined only to exact selected release checkpoints. SWE-Bench Verified and Terminal-Bench 2.1 report three-run averages where available; GDPval reports accuracy at 75 percent; BrowseComp+ reports accuracy at one; Apex Agents reports pass at one. Fractional workbook values are converted to percentages. When a source cell contains multiple reported aggregates, the first aggregate is shown unless the cell contains only three slash-separated trial values, in which case their mean is reported. A dash means the exact release checkpoint has no reported result for that benchmark.
461
-
462
- ### Family Evaluation: Advanced Research Benchmarks
463
-
464
- | Model | Revision | MCP Atlas | HLE | SciCode | OmniScience Accuracy | OmniScience Non-Hallucination | CritPT |
465
- | --- | --- | --- | --- | --- | --- | --- | --- |
466
- | K2-Horizon-7B | `mid_4_final` | 53.6 | 13.8 | - | - | 43.7 | - |
467
- | K2-Horizon-375B-A23B | `mid_4_final` | 59.0 | 21.2 | 40.3 | 20.6 | 49.2 | 5.1 |
468
-
469
- These results use exact final-stage checkpoint records. Benchmark availability differs by model, so unreported cells remain `-`.
470
-
471
- ### This Repository: Base / Pretraining Snapshot
472
-
473
- | Revision | ArabicMMLU | ARC-C | BBH | GPQA-Diamond-CoT | GSM8K | MMLU | MMLU-Pro | HumanEval | MBPP | IFEval |
474
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
475
- | `base_final` | 73.3 | 71.8 | 87.5 | 32.3 | 90.8 | 84.1 | 66.1 | 72.0 | 76.4 | 27.9 |
476
-
477
- ### This Repository: Midtraining Snapshot
478
-
479
- | Revision | IFEval avg | MMLU-Pro | GSM8K | MMLU | BBH | MBPP | AIME26 | GPQA-Diamond | LongBench-v2 | LCR | SWE-Bench Verified | Terminal Bench 2 |
480
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
481
- | `mid_1_final` | 85.2 | 80.9 | 95.7 | 89.1 | 87.3 | 85.8 | 81.2 | 70.7 | - | - | - | - |
482
- | `mid_2_final` | 90.0 | 81.3 | 95.5 | 89.1 | 87.5 | 92.0 | 89.2 | 78.0 | - | - | - | - |
483
- | `mid_3_final` | 88.4 | 81.7 | 95.3 | 89.3 | 87.9 | 87.6 | 91.9 | 78.1 | - | 52.0 | - | - |
484
- | `mid_4_final` | 92.1 | 82.6 | 95.1 | 89.5 | 87.5 | 93.8 | 92.5 | 84.9 | 48.3 | 56.0 | - | - |
485
-
486
- ### This Repository: Agentic and Long-Context Results by Stage
487
-
488
- | Revision | SWE-Bench Verified | Terminal-Bench | GDPval (acc@75%) | Tau3 Banking | BrowseComp+ | Toolathlon | Apex Agents | LongBench-v2 | LCR |
489
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
490
- | `mid_2_final` | 56.8 | - | 34.5 | - | 52.1 | - | - | - | - |
491
- | `mid_3_final` | 58.0 | 40.1 | 34.5 | - | 56.1 | 25.9 | 9.0 | - | 52.0 |
492
- | `mid_4_final` | 67.8 | 38.2 | 39.6 | 11.4 | 58.1 | - | 13.8 | 48.3 | 56.0 |
493
-
494
-
495
-
496
- ### This Repository: RULER Long-Context Results
497
-
498
- | Revision | 4K | 8K | 16K | 32K | 64K | 128K |
499
- | --- | --- | --- | --- | --- | --- | --- |
500
- | `mid_3_final` | 98.4 (13) | 97.5 (13) | 97.0 (13) | 96.5 (13) | 94.2 (13) | 90.6 (13) |
501
-
502
- RULER scores are macro-averages over the non-empty task columns in the exact checkpoint row. The number of contributing tasks is shown in parentheses.
503
-
504
-
505
- ## Datasets and Mixtures
506
-
507
- K2-Horizon training is organized around transparent data releases:
508
-
509
- - Pretraining data: [IFM/K2-Horizon-Pretrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Pretrain-Data), derived from the TxT360 lineage.
510
- - Midtraining data: [IFM/K2-Horizon-Midtrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Midtrain-Data), following the TxT360-Midas reasoning and long-context release pattern.
511
- - The public data-preparation lineage is maintained in [LLM360/TxT360](https://github.com/LLM360/TxT360).
512
 
513
- ## Source Code
514
 
515
- - [LLM360/xllm](https://github.com/LLM360/xllm) - Training framework and Hugging Face/vLLM bridges. Per-stage training commits are linked in the provenance table.
516
- - [LLM360/Eval360-V2 at `f5081bfb`](https://github.com/LLM360/Eval360-V2/commit/f5081bfba9ef8e7b380f5bd85ed1cf61a91f90b0) - Evaluation harness revision reviewed against the release benchmark workbooks.
517
- - [LLM360/agent-evals at `f3788399`](https://github.com/LLM360/agent-evals/commit/f3788399a974e49121984f11556138f23f907cee) - Agentic benchmark harness, Slurm launchers, and serving profiles used to reconcile the agentic release results.
518
- - [vLLM PR #53806 at `d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c) - Native K2-Horizon architecture plus built-in reasoning and tool parsers used by the serving example.
519
- - [LLM360/Inference360 at `a7ee94f9`](https://github.com/LLM360/Inference360/commit/a7ee94f9ad9a561e65a67ede22d4e7a2c8428b57) - Serving-image and deployment-validation references.
520
- - [LLM360/sglang at `ffb41d77`](https://github.com/LLM360/sglang/commit/ffb41d77d8b96b8a953351cb4d66d6bf4ed061a7) - SGLang serving base used by the hash-guarded K2-Horizon namespace migration.
521
- - [LLM360/TxT360](https://github.com/LLM360/TxT360) - Public data-preparation and source-data lineage reference.
522
 
523
- ## Intended Use
 
 
 
 
524
 
525
- K2-Horizon-375B-A23B is intended for open research on foundation models, transparent evaluation, staged checkpoint analysis, downstream fine-tuning, and reproducibility studies. It is suitable for researchers and developers who want to inspect training-stage behavior, serve a model privately, compare model sizes, or build downstream adaptations.
 
 
 
 
526
 
527
- ## Limitations and Safety
528
 
529
- - This is a base checkpoint family and may produce inaccurate, biased, unsafe, or hallucinated content.
530
- - The model is not optimized for moderation, refusal behavior, or production assistant safety.
531
- - Long-context behavior depends on serving stack, prompt construction, retrieval quality, and KV-cache configuration.
532
- - Tool-use and agentic results depend heavily on parser format, prompt template, tool schema, and execution harness.
533
- - The model family is documented primarily with English-language evaluations; additional multilingual evaluation should be performed before broad multilingual claims.
534
- - Users are responsible for complying with the model license and for validating deployments in their own risk domain.
535
 
536
  ## Citation
537
 
538
- Please cite this model as:
539
-
540
  ```bibtex
541
- @misc{ifmk2horizon375b2026,
542
- title = {K2-Horizon Model Family: K2-Horizon-375B-A23B},
543
- author = {IFM Team},
544
- year = {2026},
545
- url = {https://huggingface.co/IFM/K2-Horizon-375B-A23B},
546
  }
547
- ```
548
-
549
- ## Contact
550
-
551
- For questions about this release, contact the IFM / K2-Horizon team through the model repository or the IFM organization.
 
34
  > **Serving note**: the routed experts' intermediate size (1792) is not splittable into whole 128-wide quantization blocks at the usual tensor-parallel sizes (TP=4, TP=8), so expert parallelism is required.
35
 
36
 
37
+ K2-Horizon-375B-A23B is the flagship of the K2-Horizon family: a sparse Mixture-of-Experts model that stores 375B parameters and runs 23B per token, with a 512K context window. We have released the final checkpoint; intermediate checkpoints, along with the data and the training code, will be released.
38
 
39
+ <p align="center">
40
+ <img src="assets/k2-horizon-375b-a23b-benchmarks.png" alt="K2-Horizon-375B-A23B benchmark results against open MoE, dense, and closed models" width="100%">
41
+ </p>
42
 
43
+ ## K2-Horizon-375B-A23B Highlights
44
 
45
+ - **Frontier-class agentic performance.** On agentic tool use, terminal, and long-horizon workflow benchmarks it matches or beats open-weight MoE models up to 2.6× its size and is competitive with closed frontier models (see [Benchmark Results](#benchmark-results)).
46
+ - **512K context.** Native 524,288-token context from the midtraining stages onward.
47
+ - **Intermediate checkpoints.** Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
48
+ - **Fully open.** Training data/recipe and the training code will be made public.
49
 
50
+ ## Benchmark Results
51
 
52
+ <!-- TABLE:START -->
53
+ <div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;margin:0 auto;padding:8px 0 16px;overflow-x:auto"><table style="display:table;width:100%;table-layout:fixed;border-collapse:collapse;font-size:12px;margin:0"><thead><tr><th style="width:22%;border-bottom:none"></th><th colspan="1" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)"></th><th colspan="4" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)">Open-weight models</th><th colspan="3" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)">Closed models</th></tr><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2450D6"></th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;background:rgba(36,80,214,0.08);">K2-Horizon-375B-A23B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">Nemotron 3 Ultra</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">Inkling (xhigh)</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">MiniMax-M3</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">GLM 5.2 (max)</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">GPT 5.6 Luna (max)</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">GPT 5.6 Terra (high)</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:9.75%;overflow-wrap:anywhere;">Claude Sonnet5 (max)</th></tr></thead><tbody><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">375B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">550B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">975B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">428B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">753B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Activated params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">23B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">55B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">41B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">23B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">40B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600">Architecture</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">MoE</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">MoE</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">MoE</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">MoE</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">MoE</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Closed</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Closed</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Closed</td></tr><tr><td colspan="9" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Agents</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">GDPVal-AA</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Real-world professional tasks (Elo)</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">1,441</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,162</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,234</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,380</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,498</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,569</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1,503</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>1,584</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">tau3-Banking</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic tool use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">34.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">14.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">29.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">15.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">34.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">31.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">28.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>37.3</strong></td></tr><tr><td colspan="9" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Coding</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Terminal-Bench 2.1</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic terminal use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">70.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">53.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">55.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">65.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">77.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>80.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">75.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">80.5</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">SciCode</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Scientific coding</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">42.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">39.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">46.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">45.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">50.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">52.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">50.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>53.6</strong></td></tr><tr><td colspan="9" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Scientific Reasoning</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Humanity's Last Exam (without tools)</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Expert-level reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">32.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">28.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">31.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">39.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">41.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">39.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">38.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>41.3</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">GPQA Diamond</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Graduate-level science QA</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">87.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">86.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">87.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>92.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">89.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">91.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">89.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">91.1</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">CritPt</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Frontier physics reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">8.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">5.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">20.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">21.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>22.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">16.9</td></tr><tr><td colspan="9" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">General</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-LCR</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Long-context reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">76.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">71.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">73.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>80.3</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">76.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">78.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">73.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">77.0</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-Omniscience Accuracy</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Factual accuracy</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">23.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">23.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">42.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">17.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">24.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">43.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>45.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">40.0</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-Omniscience Non-Hallucination</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Non-hallucination rate</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">74.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">70.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">32.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>82.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">74.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">7.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">10.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">61.0</td></tr><tr><td colspan="9" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Agentic Evaluations</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Toolathlon Verified</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic tool use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">65.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">34.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">45.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">53.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">59.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">67.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">64.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>71.6</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Automation Bench Public</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Workflow automation</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">25.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">8.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">12.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">20.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">26.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">33.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">28.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>34.7</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Apex-Agents (pass@1)</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Long-horizon professional workflows</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">24.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">9.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">19.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">23.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">26.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">28.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">25.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>31.7</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">MCPMark</div><div style="margin-top:2px;font-size:10px;opacity:0.65">MCP tool use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">67.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">45.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">51.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">48.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">72.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">66.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>74.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">65.3</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">BrowseComp</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Deep web research</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">72.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">44.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">77.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">83.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">83.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>84.7</strong></td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">WildClawBench</div><div style="margin-top:2px;font-size:10px;opacity:0.65">In-the-wild agentic tasks</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">50.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">34.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">52.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">56.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">55.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">50.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>60.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">SWE-Atlas-QnA</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Repo-level code Q&A (strict)</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>48.4</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">25.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">42.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">46.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">SWE Bench Pro</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Software engineering (strict)</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">42.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">38.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">43.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">43.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">46.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>48.8</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">--</td></tr></tbody></table></div>
54
+ <!-- TABLE:END -->
55
 
56
+ <p style="font-size:1.25em;line-height:1.6">Scores in %, except GDPVal-AA, which is an Elo rating. Bold marks the best score in each row. The first four sections follow the <a href="https://artificialanalysis.ai/methodology/intelligence-benchmarking">Artificial Analysis Intelligence Index</a> categories. Baseline scores are from Artificial Analysis where available, otherwise from the IFM evaluation harness. SWE-Atlas-QnA and SWE Bench Pro are run without internet access; BrowseComp uses the Discard-all@95k context setting from the DeepSeek-V3.2 technical report; WildClawBench and Apex-Agents use the English text-only subsets.</p>
57
 
58
+ ## Quickstart
59
 
60
+ ### Serving
 
 
 
 
 
 
 
61
 
62
+ vLLM, recipe at [recipes.vllm.ai/IFM](https://recipes.vllm.ai/IFM):
63
 
64
+ ```shell
65
+ vllm serve IFM/K2-Horizon-375B-A23B \
66
+ --revision main \
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  --model-impl transformers \
68
+ --tensor-parallel-size 8 \
69
  --trust-remote-code \
70
  --dtype bfloat16 \
71
+ --max-model-len 131072 \
 
72
  --reasoning-parser k2_horizon \
73
  --enable-auto-tool-choice \
74
  --tool-call-parser k2_horizon
75
  ```
76
 
77
+ SGLang recipe validated on 8× H200 in the [SGLang K2 Horizon cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon):
 
 
 
 
 
 
 
 
 
78
 
79
+ ```shell
80
+ python3 -m sglang.launch_server \
81
+ --model-path IFM/K2-Horizon-375B-A23B \
82
+ --revision main \
83
+ --tp 8 \
84
+ --dtype bfloat16 \
85
+ --attention-backend fa3 \
86
+ --model-loader-extra-config '{"enable_multithread_load":false}' \
87
+ --reasoning-parser k2_horizon \
88
+ --tool-call-parser k2_horizon \
89
+ --host 0.0.0.0 --port 30000
 
 
 
 
 
 
 
 
 
 
 
 
90
  ```
91
 
92
+ ### API Usage
 
 
 
 
 
93
 
94
+ > [!Tip]
95
+ > Recommended settings: `reasoning_effort="high"`, `temperature=1.0`, `top_p=0.95`, and at least 32,768 output tokens.
96
+ > Reasoning depth is selected per request through `chat_template_kwargs`. Thinking is returned in `reasoning_content` and the answer in `content`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
97
 
98
  ```python
99
  from openai import OpenAI
100
 
101
+ client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
102
  response = client.chat.completions.create(
103
  model="IFM/K2-Horizon-375B-A23B",
104
  messages=[{"role": "user", "content": "Explain the result step by step."}],
105
+ temperature=1.0,
106
+ top_p=0.95,
107
+ max_tokens=32768,
108
  extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
109
  )
110
+ message = response.choices[0].message
111
+ print("Reasoning:", getattr(message, "reasoning_content", None))
112
+ print("Answer:", message.content)
113
  ```
114
 
115
+ ### Transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
+ Validated with Transformers 4.57.6, PyTorch 2.13.0, Safetensors 0.8.0.
118
 
119
+ ```python
120
+ from transformers import AutoModelForCausalLM, AutoTokenizer
 
 
 
 
 
121
 
122
+ model_id = "IFM/K2-Horizon-375B-A23B"
123
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
124
+ model = AutoModelForCausalLM.from_pretrained(
125
+ model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
126
+ )
127
 
128
+ inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
129
+ inputs.pop("token_type_ids", None)
130
+ outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
131
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
132
+ ```
133
 
134
+ ## Best Practices
135
 
136
+ 1. **Reasoning effort: always `high`.** All reported results use high reasoning effort. Pass `{"chat_template_kwargs": {"reasoning_effort": "high"}}` on every request.
137
+ 2. **Sampling parameters.** `temperature=1.0`, `top_p=0.95`.
138
+ 3. **Serving.** Use the validated SGLang recipe above: BF16, TP=8 on one 8× H200 node, FlashAttention-3, with multithreaded weight loading disabled. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon) and the [vLLM recipe](https://recipes.vllm.ai/IFM).
139
+ 4. **Parsers.** Enable the `k2_horizon` reasoning parser for chat, and add the `k2_horizon` tool-call parser for agent use. Leave both off for plain completion-style generation.
 
 
140
 
141
  ## Citation
142
 
 
 
143
  ```bibtex
144
+ @misc{k2horizon2026,
145
+ title = {Introducing K2 Horizon: Frontier Performance, Radically Open},
146
+ author = {{IFM Team}},
147
+ year = {2026},
148
+ url = {https://ifm.ai/blog/k2/},
149
  }
150
+ ```