---
language:
- en
license: apache-2.0
library_name: llama.cpp
tags:
- gguf
- llama.cpp
- minicpm
- minicpm5
- quantized
- text-generation
- chat
- reasoning
- tool-calling
- instruct
base_model:
- openbmb/MiniCPM5-1B
pipeline_tag: text-generation
---

# MiniCPM5-1B-GGUF

GGUF quantizations of **openbmb/MiniCPM5-1B** for use with **llama.cpp**, **Ollama**, **LM Studio**, **Jan**, **OpenWebUI**, **llama-cpp-python**, and other GGUF-compatible runtimes.

## Base Model

**Base Model:** [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B)

MiniCPM5-1B is a compact 1B-parameter language model from OpenBMB focused on:

- General chat
- Reasoning
- Coding assistance
- Tool calling
- Long-context workloads
- Edge and local deployment

This repository contains community-generated GGUF conversions and quantizations derived from the original BF16 checkpoint.

---

# Available Files

## Full Precision

| File | Description |
|--------|--------|
| MiniCPM5-1B-BF16.gguf | BF16 GGUF conversion of the original checkpoint |

## Standard Quantizations

| Quant | Recommended Usage |
|---------|---------|
| Q2_K | Smallest footprint |
| Q3_K_S | Extreme memory savings |
| Q3_K_M | Low memory systems |
| Q3_K_L | Better quality than Q3_K_M |
| Q4_0 | Legacy quant |
| Q4_1 | Legacy quant |
| Q4_K_S | Balanced |
| Q4_K_M | Recommended general-purpose quant |
| Q5_K_S | Higher quality |
| Q5_K_M | Near-lossless experience |
| Q6_K | Very high quality |
| Q8_0 | Maximum quality quantized version |

## Importance Matrix (IQ) Quantizations

Generated using llama.cpp importance matrix quantization.

| Quant | Recommended Usage |
|---------|---------|
| IQ2_M | Smallest IQ variant |
| IQ3_XXS | Ultra compact |
| IQ3_XS | Compact |
| IQ3_M | Strong quality-to-size ratio |
| IQ4_XS | High quality compact quant |
| IQ4_NL | Best IQ variant in this repository |

---

# Recommended Quant

If you are unsure which file to choose:

### Most Users

```text
MiniCPM5-1B-Q4_K_M.gguf
```

Best balance between:

- Quality
- Speed
- Memory usage
- Compatibility

### Higher Quality

```text
MiniCPM5-1B-Q5_K_M.gguf
```

or

```text
MiniCPM5-1B-Q6_K.gguf
```

### Maximum Quality

```text
MiniCPM5-1B-Q8_0.gguf
```

### Best IQ Quant

```text
MiniCPM5-1B-IQ4_NL.gguf
```

---

# Example Usage

## llama.cpp

```bash
llama-cli \
    -m MiniCPM5-1B-Q4_K_M.gguf \
    -ngl 99 \
    -c 8192
```

## llama-server

```bash
llama-server \
    -m MiniCPM5-1B-Q4_K_M.gguf \
    -c 8192 \
    -ngl 99
```

## Ollama

Create a Modelfile:

```text
FROM MiniCPM5-1B-Q4_K_M.gguf
```

Then:

```bash
ollama create minicpm5-1b -f Modelfile
ollama run minicpm5-1b
```

## Python

```python
from llama_cpp import Llama

llm = Llama(
    model_path="MiniCPM5-1B-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,
)

print(
    llm(
        "Explain quantum computing in simple terms.",
        max_tokens=256
    )
)
```

---

# Conversion Details

Conversion pipeline:

1. Download original Hugging Face model.
2. Convert to BF16 GGUF using llama.cpp.
3. Generate importance matrix for IQ quantization.
4. Generate standard GGUF quantizations.
5. Generate IQ quantizations.
6. Upload all artifacts to Hugging Face.

Tools used:

- llama.cpp
- convert_hf_to_gguf.py
- llama-quantize
- llama-imatrix

---

# Hardware Guidance

| Quant | Approximate Use Case |
|---------|---------|
| Q2_K | Very constrained devices |
| Q3_* | Raspberry Pi / older CPUs |
| Q4_K_M | Most laptops and desktops |
| Q5_K_M | Modern systems |
| Q6_K | High-end CPUs and GPUs |
| Q8_0 | Largest RAM footprint |

---

# Disclaimer

This repository is a community conversion of the original model and is not an official OpenBMB release.

All model weights, architecture, training methodology, and evaluation results belong to the original authors of MiniCPM5-1B.

If you use this model in research or production environments, please cite and credit the original MiniCPM team.

---

# Credits

- OpenBMB for creating MiniCPM5-1B
- ggml-org for llama.cpp
- Hugging Face for model hosting
- Community contributors supporting GGUF deployment

Original model:

https://huggingface.co/openbmb/MiniCPM5-1B