--- base_model: openbmb/MiniCPM5-1B library_name: gguf pipeline_tag: text-generation tags: - gguf - llama.cpp - quantized - imatrix --- # MiniCPM5-1B GGUF GGUF quantizations of [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B), covering 30 files (18.1 GB total). ## Files | File | Quant | Size | Notes | |---|---|---:|---| | [MiniCPM5-1B-BF16.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-BF16.gguf) | `BF16` | 2.02 GB | Full precision source. Every quant below is cut from this file. | | [MiniCPM5-1B-Q8_0.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q8_0.gguf) | `Q8_0` | 1.07 GB | Effectively lossless. Use when disk and RAM are not the constraint. | | [MiniCPM5-1B-Q6_K.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q6_K.gguf) | `Q6_K` | 0.83 GB | Near-lossless; the last stop before quality becomes measurable. | | [MiniCPM5-1B-Q5_K_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q5_K_M.gguf) | `Q5_K_M` | 0.73 GB | Very good quality, noticeably smaller than Q6_K. | | [MiniCPM5-1B-Q5_K_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q5_K_S.gguf) | `Q5_K_S` | 0.72 GB | Slightly smaller than Q5_K_M for a slight quality cost. | | [MiniCPM5-1B-Q5_1.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q5_1.gguf) | `Q5_1` | 0.77 GB | Legacy. Prefer Q5_K_M. | | [MiniCPM5-1B-Q5_0.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q5_0.gguf) | `Q5_0` | 0.72 GB | Legacy. Prefer Q5_K_M. | | [MiniCPM5-1B-Q4_K_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q4_K_M.gguf) | `Q4_K_M` | 0.64 GB | The usual default. Best quality-per-byte for most people. | | [MiniCPM5-1B-Q4_K_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q4_K_S.gguf) | `Q4_K_S` | 0.62 GB | A little smaller than Q4_K_M, a little worse. | | [MiniCPM5-1B-IQ4_NL.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ4_NL.gguf) | `IQ4_NL` | 0.62 GB | Non-linear 4-bit; good on hardware without fast K-quant kernels. | | [MiniCPM5-1B-IQ4_XS.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ4_XS.gguf) | `IQ4_XS` | 0.60 GB | Best sub-4.5bpw option; usually beats Q4_K_S at a smaller size. | | [MiniCPM5-1B-Q4_1.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q4_1.gguf) | `Q4_1` | 0.67 GB | Legacy. Prefer Q4_K_M. | | [MiniCPM5-1B-Q4_0.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q4_0.gguf) | `Q4_0` | 0.62 GB | Legacy round-to-nearest. Prefer Q4_K_M unless a runtime needs this. | | [MiniCPM5-1B-Q3_K_L.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q3_K_L.gguf) | `Q3_K_L` | 0.57 GB | Small, with real quality loss. Usable when RAM is tight. | | [MiniCPM5-1B-Q3_K_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q3_K_M.gguf) | `Q3_K_M` | 0.54 GB | Smaller again; noticeable degradation. | | [MiniCPM5-1B-IQ3_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ3_M.gguf) | `IQ3_M` | 0.52 GB | Strong at ~3.7bpw, clearly better than Q3_K_M. | | [MiniCPM5-1B-IQ3_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ3_S.gguf) | `IQ3_S` | 0.51 GB | Slightly smaller than IQ3_M. | | [MiniCPM5-1B-Q3_K_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q3_K_S.gguf) | `Q3_K_S` | 0.51 GB | Aggressive. Prefer IQ3_M at a similar size. | | [MiniCPM5-1B-IQ3_XS.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ3_XS.gguf) | `IQ3_XS` | 0.50 GB | Aggressive but coherent. | | [MiniCPM5-1B-IQ3_XXS.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ3_XXS.gguf) | `IQ3_XXS` | 0.46 GB | Very aggressive; imatrix carries it. | | [MiniCPM5-1B-Q2_K.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q2_K.gguf) | `Q2_K` | 0.45 GB | Very small, heavily degraded. For experimentation. | | [MiniCPM5-1B-IQ2_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ2_M.gguf) | `IQ2_M` | 0.43 GB | The smallest size most people find usable. | | [MiniCPM5-1B-Q2_K_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q2_K_S.gguf) | `Q2_K_S` | 0.43 GB | Smaller than Q2_K, requires the imatrix. | | [MiniCPM5-1B-IQ2_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ2_S.gguf) | `IQ2_S` | 0.41 GB | Below the usual usability line. | | [MiniCPM5-1B-IQ2_XS.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ2_XS.gguf) | `IQ2_XS` | 0.38 GB | Experimental. | | [MiniCPM5-1B-IQ2_XXS.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ2_XXS.gguf) | `IQ2_XXS` | 0.36 GB | Experimental. | | [MiniCPM5-1B-Q2_0.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q2_0.gguf) | `Q2_0` | 0.44 GB | Extreme, group-64. Included for completeness. | | [MiniCPM5-1B-IQ1_M.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ1_M.gguf) | `IQ1_M` | 0.34 GB | Extreme. Expect substantial degradation. | | [MiniCPM5-1B-IQ1_S.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-IQ1_S.gguf) | `IQ1_S` | 0.33 GB | Extreme. Expect substantial degradation. | | [MiniCPM5-1B-Q1_0.gguf](https://huggingface.co/NANI-Nithin/MiniCPM5-1B-GGUF/blob/main/MiniCPM5-1B-Q1_0.gguf) | `Q1_0` | 0.27 GB | Extreme. Included for completeness. | ## Which one should I download? Pick the largest file that leaves a couple of gigabytes of headroom on the device you will run it on — the model has to fit in RAM (or VRAM, if you are offloading) alongside the KV cache and the OS. - Plenty of memory: **Q6_K** or **Q8_0**. - The usual choice: **Q4_K_M**. - Tight on memory: **IQ4_XS**, then **IQ3_M**, then **IQ2_M**. - The `IQ*` files are imatrix-guided and generally beat a `Q*` file of similar size, at the cost of slightly slower inference on some hardware. ## Quantization details - Importance matrix computed with `llama-imatrix` over 500 rows of [Salesforce/wikitext](https://huggingface.co/datasets/Salesforce/wikitext) (`wikitext-2-raw-v1`). - The matrix was computed on the **BF16** weights. - K-quants below 6 bit and the whole `IQ` set are imatrix-guided. `Q4_0`/`Q4_1`/`Q5_0`/`Q5_1` are legacy round-to-nearest and ignore it; `Q6_K`/`Q8_0` are near-lossless and do not need it. - All files are cut from the same BF16 GGUF, so differences between them are quantization only. ## Usage ```bash llama-cli -hf NANI-Nithin/MiniCPM5-1B-GGUF:Q4_K_M -p "Hello" ``` Or download one file and point at it directly: ```bash huggingface-cli download NANI-Nithin/MiniCPM5-1B-GGUF MiniCPM5-1B-Q4_K_M.gguf --local-dir . llama-cli -m MiniCPM5-1B-Q4_K_M.gguf -p "Hello" ``` --- Quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp) by AgentQuantix on 2026-09-04. Licensing follows the base model.