Instructions to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Ollama
How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with Ollama:
```
ollama run hf.co/webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with Docker Model Runner:
```
docker model run hf.co/webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M
```

Lemonade

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Sakura-MiniCPM-V-4.7-1B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Sakura — MiniCPM-V 4.7 1B (GGUF)

GGUF quantizations of MiniCPM-V 4.7 1B by OpenBMB: a 1B vision-language model whose language part is a Qwen3.5-style hybrid (24 layers, three linear-attention layers for each full-attention layer). Independent work, not made or endorsed by OpenBMB.

The official GGUF repository openbmb/MiniCPM-V-4.7-1B-gguf already exists. We compared two of its files with ours: its Q4_K_M is byte-identical to a Q4_K_M that we build without importance matrix (same sha256), so that file was made without one. Its Q8_0 has the same tensor data as our Q8_0; ours was quantized with the importance matrix and still matches, because Q8_0 does not use one (our Q8_0 file carries the importance-matrix metadata all the same). What this repository adds is importance-matrix quantizations (Q6_K, Q5_K_M, Q4_K_M, IQ4_XS) and a measured comparison against the F16 file.

Files

File	Size	What it is
`Sakura-MiniCPM-V-4.7-1B-Q8_0-1.01GiB.gguf`	1.01 GiB	Q8_0 (the tensors equal the official Q8_0)
`Sakura-MiniCPM-V-4.7-1B-Q6_K-0.78GiB.gguf`	0.78 GiB	Q6_K with importance matrix
`Sakura-MiniCPM-V-4.7-1B-Q5_K_M-0.70GiB.gguf`	0.70 GiB	Q5_K_M with importance matrix
`Sakura-MiniCPM-V-4.7-1B-Q4_K_M-0.63GiB.gguf`	0.63 GiB	Q4_K_M with importance matrix
`Sakura-MiniCPM-V-4.7-1B-IQ4_XS-0.59GiB.gguf`	0.59 GiB	IQ4_XS with importance matrix
`mmproj-MiniCPM-V-4.7-1B-F16.gguf`	1.03 GiB	vision projector, unchanged copy of the official `mmproj-model-f16.gguf`
`mmproj-MiniCPM-V-4.7-1B-F16-4x.gguf`	0.84 GiB	vision projector, unchanged copy of the official `mmproj-model-f16-4x.gguf`

How they were made

The llama.cpp checkout we used (7 October 2026) has a converter for MiniCPM-V 4.6 but not for 4.7, so we did not convert from the safetensors. We requantized the official F16 GGUF (MiniCPM-V-4_7-1B-F16.gguf) with llama-quantize of llama.cpp b11524. The importance matrix was computed with llama-imatrix on that F16 file over 300 chunks of 512 tokens of our calib_mix_v1 text (English / German / code mix); it covers the language model only, the vision projector files are not requantized.

Run

llama-server -m Sakura-MiniCPM-V-4.7-1B-Q5_K_M-0.70GiB.gguf --jinja -ngl 99 -c 8192

The command above is text only; add --mmproj mmproj-MiniCPM-V-4.7-1B-F16.gguf for images. Image input needs a llama.cpp build that knows the MiniCPM-V 4.7 projector, first available in b11594 (PR 29416, merged on 10 October 2026); the older b11524 stops with unknown projector type: minicpmv4_7. The text part also loads with b11524. The metadata of mmproj-MiniCPM-V-4.7-1B-F16.gguf has scale factor 4 and that of the -4x file scale factor 2; according to the original model card, the 4x setting produces more image tokens for finer detail.

What we checked

Text only, llama-completion b11524, raw ChatML prompt, temperature 0, CPU: the Q4_K_M and Q8_0 files each produced coherent text for a harmless question (the 60 generated tokens were still inside the thinking block <think> that the chat template of the original starts). The log is logs/text_smoke.log in our workspace; it also records the projector load error. Image input was tested separately (next paragraph); we did not test video, long contexts or speed.

Image input, llama-server b11594 (the build with the 4.7 projector; the sha256 of the downloaded release file matches the value GitHub lists), CPU only, temperature 0, 160 tokens, thinking switched off: two images, one synthetic (the text SAKURA TEST 4217, a red circle and a blue square) and one frame of a generated fox video. The official Q8_0 and our Q5_K_M, the latter with the mmproj-MiniCPM-V-4.7-1B-F16.gguf projector and with the -4x projector, read the text correctly, named both shapes and their colors and described the fox scene correctly (a red fox in a snowy forest). That is two images and four configurations; we did not test OCR quality, many images, video, other quantizations or the GPU backends. The answers are in logs/vision_test.log in our workspace.

Quality (what we measured)

KL divergence of every file against the official F16 file on the next-token distribution, with llama-perplexity (b11524, Vulkan, same texts, 16 chunks of 2048 tokens, one stored set of reference logits for all files). The texts are separate German (de), English (en) and neutral web (web) evaluation mixes. A line check found a few prompt-template lines in de (24 of 328 lines with at least 80 characters) and en (3 of 401) that also occur in the importance-matrix text; web shares none. A lower KLD and a higher same-top-token rate are better.

File	Size	KLD de	KLD en	KLD web	same top token de / en / web
Q8_0	1.01 GiB	0.0035	0.0030	0.0029	97.4 / 97.4 / 96.2 %
Q6_K	0.78 GiB	0.0115	0.0095	0.0082	95.5 / 95.6 / 94.4 %
Q5_K_M	0.70 GiB	0.0272	0.0261	0.0190	93.4 / 93.4 / 92.1 %
Q4_K_M	0.63 GiB	0.0984	0.0813	0.0605	88.0 / 88.8 / 86.1 %
IQ4_XS	0.59 GiB	0.1242	0.1078	0.0786	86.7 / 87.3 / 84.6 %
Q4_K_M without importance matrix (equals the official Q4_K_M; not in this repository)	0.63 GiB	0.1371	0.1140	0.0841	85.9 / 86.3 / 84.3 %

The importance matrix lowers the Q4_K_M KLD by about 28 % (de), 29 % (en) and 28 % (web). The largest standard error of the KLD values is 0.0027 (IQ4_XS, de). The model reacts strongly to 4-bit quantization: from Q5_K_M to Q4_K_M the KLD rises by a factor of 3.1 to 3.6, so we would take Q5_K_M or larger when memory allows. The reference is the official F16 file; we did not measure how far that file is from the BF16 original. As a task check we ran zero-shot MMLU (all 14,042 test questions of cais/mmlu; the model picks among the answer letters A to D by token probability after a plain Answer: prompt, no chat template; llama-server b11524, temperature 0): Q8_0 50.4 % (95 % interval 49.6 to 51.2), Q4_K_M 49.3 % (48.4 to 50.1); the Q4_K_M is 1.12 percentage points below the Q8_0 (paired 95 % interval 0.51 to 1.73). Absolute values depend on this prompt format and are not comparable to published MMLU numbers; we ran no other benchmark.

License

Apache-2.0 as the original model (LICENSE). The .gguf language-model files are quantized (modified) versions of the official F16 GGUF; the projector files are unchanged copies. See NOTICE.

Credits

Model: openbmb/MiniCPM-V-4.7-1B and its official GGUF openbmb/MiniCPM-V-4.7-1B-gguf (OpenBMB).
Quantization and measurement: ggml-org/llama.cpp (MIT).

中文说明 · 樱花 (Simplified Chinese)

English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura — MiniCPM-V 4.7 1B(GGUF)

OpenBMB 的 MiniCPM-V 4.7 1B 的 GGUF 量化版本:这是一个 1B 的视觉语言模型,其语言部分是 Qwen3.5 风格的混合结构(24 层,每一层完整注意力对应三层线性注意力)。属于独立工作,并非 OpenBMB 制作或认可。

官方的 GGUF 仓库 openbmb/MiniCPM-V-4.7-1B-gguf 已经存在。我们将其中两个文件与我们的进行了比较:它的 Q4_K_M 与我们不带重要性矩阵构建的 Q4_K_M 逐字节相同(sha256 相同),所以该文件是在没有重要性矩阵的情况下制作的。它的 Q8_0 与我们的 Q8_0 张量数据相同;我们的是用重要性矩阵量化的,结果仍然相同,因为 Q8_0 不使用重要性矩阵(我们的 Q8_0 文件仍带有重要性矩阵的元数据)。本仓库新增的是 带重要性矩阵的量化版本(Q6_K、Q5_K_M、Q4_K_M、IQ4_XS)以及与 F16 文件的实测对比。

文件

文件	大小	说明
`Sakura-MiniCPM-V-4.7-1B-Q8_0-1.01GiB.gguf`	1.01 GiB	Q8_0(张量与官方 Q8_0 相同)
`Sakura-MiniCPM-V-4.7-1B-Q6_K-0.78GiB.gguf`	0.78 GiB	带重要性矩阵的 Q6_K
`Sakura-MiniCPM-V-4.7-1B-Q5_K_M-0.70GiB.gguf`	0.70 GiB	带重要性矩阵的 Q5_K_M
`Sakura-MiniCPM-V-4.7-1B-Q4_K_M-0.63GiB.gguf`	0.63 GiB	带重要性矩阵的 Q4_K_M
`Sakura-MiniCPM-V-4.7-1B-IQ4_XS-0.59GiB.gguf`	0.59 GiB	带重要性矩阵的 IQ4_XS
`mmproj-MiniCPM-V-4.7-1B-F16.gguf`	1.03 GiB	视觉投影器,官方 `mmproj-model-f16.gguf` 的未改动副本
`mmproj-MiniCPM-V-4.7-1B-F16-4x.gguf`	0.84 GiB	视觉投影器,官方 `mmproj-model-f16-4x.gguf` 的未改动副本

制作方式

我们使用的 llama.cpp 检出版本(2026 年 10 月 7 日)有 MiniCPM-V 4.6 的转换器,但没有 4.7 的,因此我们没有从 safetensors 转换。我们用 llama.cpp b11524 的 llama-quantize 对 官方 F16 GGUF(MiniCPM-V-4_7-1B-F16.gguf)重新量化。重要性矩阵用 llama-imatrix 在该 F16 文件上、对我们的 calib_mix_v1 文本(英文 / 德文 / 代码混合)的 300 个、每个 512 词元的块计算;它只覆盖语言模型,视觉投影器文件没有重新量化。

运行

llama-server -m Sakura-MiniCPM-V-4.7-1B-Q5_K_M-0.70GiB.gguf --jinja -ngl 99 -c 8192

上面的命令是仅文本命令;要输入图像,请加上 --mmproj mmproj-MiniCPM-V-4.7-1B-F16.gguf。图像输入需要认识 MiniCPM-V 4.7 投影器的 llama.cpp 版本,最早为 b11594(PR 29416,2026 年 10 月 10 日合并);较旧的 b11524 会以 unknown projector type: minicpmv4_7 停止。语言部分在 b11524 上同样可以加载。mmproj-MiniCPM-V-4.7-1B-F16.gguf 的元数据中缩放因子为 4,-4x 文件的缩放因子为 2;根据原模型卡,4x 设置会产生更多图像词元以获得更精细的细节。

我们检查了什么

仅文本,llama-completion b11524,原始 ChatML 提示词,温度 0,CPU:Q4_K_M 和 Q8_0 文件各自对一个无害问题生成了连贯的文本(生成的 60 个词元仍在原模型聊天模板开启的思考块 <think> 之内)。日志为我们工作区中的 logs/text_smoke.log,其中也记录了投影器的加载错误。图像输入另行测试(见下一段);我们没有测试视频、长上下文或速度。

图像输入,b11594 的 llama-server(带有 4.7 投影器的版本;下载的发布文件的 sha256 与 GitHub 列出的值一致),仅 CPU,温度 0;160 个词元,关闭思考:两张图像,一张合成图(文字 SAKURA TEST 4217、一个红色圆形和一个蓝色方形),一张是生成的狐狸视频中的一帧。官方 Q8_0 以及我们的 Q5_K_M(后者分别使用 mmproj-MiniCPM-V-4.7-1B-F16.gguf 投影器和 -4x 投影器)都正确读出了文字,说出了两个形状及其颜色,并正确描述了狐狸场景(雪林中的一只红狐狸)。这是两张图像、四种配置;我们没有测试 OCR 质量、大量图像、视频、其他量化版本或 GPU 后端。回答记录在我们工作区的 logs/vision_test.log 中。

质量(我们测量了什么)

每个文件相对于 官方 F16 文件 在下一个词元分布上的 KL 散度,使用 llama-perplexity(b11524,Vulkan,相同文本,16 个、每个 2048 词元的块,所有文件使用同一组存储的参考 logits)。文本是独立的德文(de)、英文(en)和中性网页(web)评估文本。逐行检查发现 de(328 行中 24 行,至少 80 个字符)和 en(401 行中 3 行)中有少量提示模板行也出现在重要性矩阵文本中;web 没有重合。KLD 越低、相同首选词元比例越高越好。

文件	大小	KLD de	KLD en	KLD web	相同首选词元 de / en / web
Q8_0	1.01 GiB	0.0035	0.0030	0.0029	97.4 / 97.4 / 96.2 %
Q6_K	0.78 GiB	0.0115	0.0095	0.0082	95.5 / 95.6 / 94.4 %
Q5_K_M	0.70 GiB	0.0272	0.0261	0.0190	93.4 / 93.4 / 92.1 %
Q4_K_M	0.63 GiB	0.0984	0.0813	0.0605	88.0 / 88.8 / 86.1 %
IQ4_XS	0.59 GiB	0.1242	0.1078	0.0786	86.7 / 87.3 / 84.6 %
不带重要性矩阵的 Q4_K_M(等同于官方 Q4_K_M;不在本仓库中)	0.63 GiB	0.1371	0.1140	0.0841	85.9 / 86.3 / 84.3 %

重要性矩阵使 Q4_K_M 的 KLD 降低约 28 %(de)、29 %(en)和 28 %(web)。KLD 值的最大标准误差为 0.0027(IQ4_XS,de)。该模型对 4 比特量化反应强烈:从 Q5_K_M 到 Q4_K_M,KLD 上升到 3.1 至 3.6 倍,因此在内存允许时我们会选 Q5_K_M 或更大的文件。参考是官方 F16 文件;我们没有测量该文件与 BF16 原版相差多少。作为任务检查,我们运行了零样本 MMLU(cais/mmlu 的全部 14,042 道测试题;模型在简单的 Answer: 提示词之后按词元概率在答案字母 A 到 D 之间选择,不使用聊天模板;b11524 的 llama-server,温度 0):Q8_0 为 50.4 %(95 % 区间 49.6 到 51.2),Q4_K_M 为 49.3 %(48.4 到 50.1);Q4_K_M 比 Q8_0 低 1.12 个百分点(配对 95 % 区间 0.51 到 1.73)。绝对数值取决于这种提示词格式,不能与已发布的 MMLU 数字比较;我们没有运行其他基准测试。

许可

与原模型相同,为 Apache-2.0(LICENSE)。.gguf 语言模型文件是官方 F16 GGUF 的量化(修改)版本;投影器文件是未改动的副本。见 NOTICE。

致谢

模型:openbmb/MiniCPM-V-4.7-1B 及其官方 GGUF openbmb/MiniCPM-V-4.7-1B-gguf(OpenBMB)。
量化与测量:ggml-org/llama.cpp(MIT)。

Downloads last month: 660

GGUF

Model size

1B params

Architecture

qwen35

Hardware compatibility

4-bit

5-bit

6-bit

8-bit

Model tree for webmp3/Sakura-MiniCPM-V-4.7-1B-GGUF

Base model

openbmb/MiniCPM-V-4.7-1B

Quantized

(47)

this model