--- license: apache-2.0 base_model: IFM/K2-Horizon-0.9B base_model_relation: quantized library_name: llama.cpp pipeline_tag: text-generation tags: - gguf - llama.cpp - k2-horizon - quantized - text-generation --- # K2-Horizon-0.9B GGUF Community GGUF quantizations of [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B).
☕ If you find this GGUF useful, please consider
buying me a coffee.
Your support helps cover the GPU costs of future releases.
Thank you for supporting this work.
## About K2-Horizon-0.9B [K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) is IFM's compact 0.9B-class dense reasoning model. It has a 128K-token context window with YaRN RoPE scaling and is intended for English and Chinese conversational, reasoning, coding, science, and tool-use workloads. See the [official model card](https://huggingface.co/IFM/K2-Horizon-0.9B) for the supported serving stack, prompt conventions, and full evaluation protocol. [![K2-Horizon-0.9B benchmark results](assets/k2-horizon-0.9b-benchmarks.png)](https://huggingface.co/IFM/K2-Horizon-0.9B) *Upstream K2-Horizon-0.9B benchmark results; image and results are from the [official model card](https://huggingface.co/IFM/K2-Horizon-0.9B).* This is a quantization-only release. No training, fine-tuning, merging, or weight modification other than GGUF conversion and quantization was performed. ## Files | Quantization | File size (GiB) | A10M generation token/s | Validation | Recommendation / Notes | |---|---:|---:|---|---| | Q8_0 | 1.07 | 281.05 | Load/generate pass | Highest-fidelity option in this release. | | Q6_K | 0.83 | 310.87 | Load/generate pass | High-fidelity local inference. | | Q5_K_M | 0.72 | 316.41 | Load/generate pass | Balanced quality and memory. | | Q4_K_M | 0.62 | 352.89 | Load/generate pass | Recommended starting point for general use. | | Q3_K_M | 0.51 | 261.54 | Load/generate pass | Lower-memory profile; validate your workload. | | Q2_K | 0.41 | 326.72 | Load/generate pass | Aggressive low-memory option. | | IQ2_XS | 0.34 | 328.10 | Load/generate pass | Experimental low-memory profile. | | IQ1_M | 0.29 | 343.17 | Load/generate pass | Experimental; instruction following and reasoning may degrade. | | Q1_0 | 0.21 | 446.95 | Load/generate pass | Experimental / legacy minimum-memory option. | Every file above was loaded and generated non-empty text in the K2 pre-release llama.cpp runtime on an NVIDIA A10M. The A10M measurements were taken with single-stream `llama-bench`, prompt 256, generation 64, batch/ubatch 512, 64 CPU threads, full GPU offload, and three repetitions on an otherwise idle GPU. Low-bit files—especially Q2, IQ2, IQ1, and Q1—can materially reduce reasoning, instruction following, and tool-call reliability; test the selected file with the prompts that matter to you. K2-Horizon support is pre-release in llama.cpp. The upstream full tool-aware Jinja template uses constructs not yet parsed by the runtime revision used for this release; the reproducibility folder includes the compatible single-turn chat wrapper used for smoke testing. Tool calling is therefore not certified by this GGUF release. ## License and attribution The upstream model is licensed under Apache License 2.0. Preserve upstream attribution and the license when redistributing these derivative artifacts. These are community GGUF quantizations, not an official IFM release or endorsement. The compact benchmark summary and reproduction script are in [`reproducibility/a10m-benchmark.tsv`](reproducibility/a10m-benchmark.tsv) and [`reproducibility/benchmark_gpu1.sh`](reproducibility/benchmark_gpu1.sh). Checksums for published artifacts and reproducibility inputs are in [`SHA256SUMS.txt`](SHA256SUMS.txt).