hermitdave/K2-Horizon-32B-MLX-6bit

K2-Horizon-32B MLX 6-bit

Uniform 6-bit quantization of IFM/K2-Horizon-32B, a 32B dense decoder-only model with 512K context. Converts to ~24 GB, fitting on 64 GB Apple Silicon with room to spare.

Upstream model: IFM/K2-Horizon-32B by Institute of Foundation Models, released under Apache 2.0.

Conversion: Quantized to MLX format using Hermes Agent with mlx-lm.

Architecture

K2-Horizon-32B is the dense member of the K2-Horizon family — not MoE, not MoVA. A straightforward 32B decoder with 64 layers, hidden dim 5120, and 64 attention heads (8 KV heads).

Spec	Value
Total params	32B (~35B with embeddings)
Architecture	Dense
Layers	64
Hidden dim	5120
Attention heads	64 (8 KV)
Context length	512K native
Reasoning effort	Always high
BF16 size	~64 GB
Quantized size	~24 GB

Note: This is the Stage 1 checkpoint. A final checkpoint with improved results is expected.

Quickstart

pip install -U mlx-lm

python3 -m mlx_lm.generate \
  --model hermitdave/K2-Horizon-32B-MLX-6bit \
  --prompt "Explain why long-context evaluation is difficult." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

Reasoning

K2-Horizon-32B is a reasoning model. Always use reasoning_effort="high":

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="hermitdave/K2-Horizon-32B-MLX-6bit",
    messages=[{"role": "user", "content": "Explain step by step."}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
print("Reasoning:", getattr(response.choices[0].message, "reasoning_content", None))
print("Answer:", response.choices[0].message.content)

oMLX Patch

K2-Horizon requires oMLX v0.6.4+ with the K2-Horizon support patch:

• Registers k2_horizon model type
• Recognizes IFM thinking tags (<ifm|think>, <ifm|think_fast>, <ifm|think_faster>)
• Parses IFM tool call format (<ifm|tool_call>, <ifm|arg_key>, <ifm|arg_value>)
• Strips tags in streaming and non-streaming paths

Benchmark Results

Benchmark	Score
tau3 Banking Agentic	22.5
Terminal-Bench 2.1	36.6
SciCode	30.2
Humanity's Last Exam	22.8
GPQA Diamond	82.3
AA-LCR (long-context)	65.3

See the model card for full results.

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}

License

Apache 2.0 (same as upstream).