Instructions to use openbmb/MiniCPM-V-4.7-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM-V-4.7-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="openbmb/MiniCPM-V-4.7-1B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("openbmb/MiniCPM-V-4.7-1B")
model = AutoModelForMultimodalLM.from_pretrained("openbmb/MiniCPM-V-4.7-1B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openbmb/MiniCPM-V-4.7-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM-V-4.7-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4.7-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM-V-4.7-1B

SGLang

How to use openbmb/MiniCPM-V-4.7-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM-V-4.7-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4.7-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM-V-4.7-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4.7-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM-V-4.7-1B with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM-V-4.7-1B
```

tc-mb commited on 4 days ago

Commit

76bf512

verified ·

1 Parent(s): 39b8f6d

update model card

Browse files

Files changed (6) hide show

.gitattributes +1 -0
README.md +15 -7
minicpm_v4dot7_assets/minicpmv4.7/moe_speed_comparison_en.gif +2 -2
minicpm_v4dot7_assets/minicpmv4.7/performance_efficiency_frontier.png +2 -2
minicpm_v4dot7_assets/minicpmv4.7/v47_1b_iphone_en.gif +2 -2
minicpm_v4dot7_assets/minicpmv4.7/{v47_35ba3b_iphone_en.gif → v47_35ba3b_iphone2_en.gif} +2 -2

.gitattributes CHANGED Viewed

@@ -42,3 +42,4 @@ minicpm_v4dot7_assets/minicpmv4.7/performance_efficiency_frontier.png filter=lfs
 minicpm_v4dot7_assets/minicpmv4.7/v47_1b_iphone_en.gif filter=lfs diff=lfs merge=lfs -text
 minicpm_v4dot7_assets/minicpmv4.7/v47_35ba3b_ipad_en.gif filter=lfs diff=lfs merge=lfs -text
 minicpm_v4dot7_assets/minicpmv4.7/v47_35ba3b_iphone_en.gif filter=lfs diff=lfs merge=lfs -text

 minicpm_v4dot7_assets/minicpmv4.7/v47_1b_iphone_en.gif filter=lfs diff=lfs merge=lfs -text
 minicpm_v4dot7_assets/minicpmv4.7/v47_35ba3b_ipad_en.gif filter=lfs diff=lfs merge=lfs -text
 minicpm_v4dot7_assets/minicpmv4.7/v47_35ba3b_iphone_en.gif filter=lfs diff=lfs merge=lfs -text
+minicpm_v4dot7_assets/minicpmv4.7/v47_35ba3b_iphone2_en.gif filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -1,8 +1,8 @@
-[← Home](../README_en.md) · [中文](./minicpm_v4dot7_zh.md)
 # MiniCPM-V 4.7
-[Evaluation](#evaluation) · [Examples](#examples) · [Usage](#usages) · [Model downloads](#model-downloads)
 **MiniCPM-V 4.7** is the latest model in the MiniCPM-V series, available in a **1B** variant. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include:
@@ -11,9 +11,9 @@
 - 🚀 **Efficient Architecture.**
   Based on the intra-ViT early compression technique from [LLaVA-UHD v4](https://github.com/THUMAI-Lab/LLaVA-UHD-v4), MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports **mixed 4x/16x visual token compression**, allowing developers to balance visual detail and inference cost across tasks.
 - 💻 **Broad Platform and Chip Support.**
-  MiniCPM-V 4.7 retains MiniCPM-V 4.6's support for **iOS, Android, and HarmonyOS**, together with open-source on-device deployment support. It also provides **day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm**, extending deployment options across mobile and local computing devices.
 - 🛠️ **Developer Friendly.**
-  MiniCPM-V 4.7 supports [inference frameworks](./deployment_en.md#supported-inference-and-training-frameworks-and-api) including SGLang, vLLM, llama.cpp, and Ollama, along with [fine-tuning frameworks](./deployment_en.md#supported-inference-and-training-frameworks-and-api) including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds **FP8 quantization support**. We also release **Self-Replay Fine-Tuning**, a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier.
 ### Evaluation
@@ -37,7 +37,7 @@ MiniCPM-V 4.7 can run on phones and tablets. The clips below are unedited screen
 <table align="center">
   <tr>
-    <td align="center"><b>1B</b><br><sub>iPhone</sub></td>
   </tr>
   <tr>
     <td align="center"><img src="./minicpm_v4dot7_assets/minicpmv4.7/v47_1b_iphone_en.gif" height="520"/></td>
@@ -231,6 +231,14 @@ curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/
 </details>
 #### Inference with OpenVINO™
 [OpenVINO™](https://docs.openvino.ai) is an open-source toolkit for optimizing and deploying AI inference on Intel CPU/GPU/NPU platforms. OpenVINO provides day-0 support for MiniCPM-V-4.7 on Intel platforms with high-performance inference. See the [OpenVINO™ Toolkit](https://github.com/openvinotoolkit) to get started. You can install and run the example with this notebook (link TBD).
@@ -241,7 +249,7 @@ MiniCPM-V 4.7 supports deployment on iOS, Android, and HarmonyOS, with day-0 sup
 #### Inference and Fine-Tuning
-For supported inference and training frameworks, see [Supported Inference and Training Frameworks and API](./deployment_en.md#supported-inference-and-training-frameworks-and-api). The [MiniCPM-V 4.7 Cookbook](https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/docs/pages/v4.7/overview.md) covers deployment, fine-tuning, and quantization workflows.
 **Task guides**
@@ -275,7 +283,7 @@ We would like to thank the following projects:
 </details>
-## Model Downloads
 MiniCPM-V 4.7 is available in the following variants:

+[← Home](https://github.com/yiranyyu/MiniCPM-o-private/blob/main/README.md) · [中文](https://github.com/yiranyyu/MiniCPM-o-private/blob/main/docs/minicpm_v4dot7_zh.md)
 # MiniCPM-V 4.7
+[Evaluation](#evaluation) · [Examples](#examples) · [Usage](#usages) · [Model weights](#model-weights)
 **MiniCPM-V 4.7** is the latest model in the MiniCPM-V series, available in a **1B** variant. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include:
 - 🚀 **Efficient Architecture.**
   Based on the intra-ViT early compression technique from [LLaVA-UHD v4](https://github.com/THUMAI-Lab/LLaVA-UHD-v4), MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports **mixed 4x/16x visual token compression**, allowing developers to balance visual detail and inference cost across tasks.
 - 💻 **Broad Platform and Chip Support.**
+  MiniCPM-V 4.7 supports **iOS, Android, and HarmonyOS**, together with open-source on-device deployment support. It also provides **day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm**, extending deployment options across mobile and local computing devices.
 - 🛠️ **Developer Friendly.**
+  MiniCPM-V 4.7 supports [inference frameworks](https://github.com/yiranyyu/MiniCPM-o-private/blob/main/docs/deployment_en.md#supported-inference-and-training-frameworks-and-api) including SGLang, vLLM, llama.cpp, and Ollama, along with [fine-tuning frameworks](https://github.com/yiranyyu/MiniCPM-o-private/blob/main/docs/deployment_en.md#supported-inference-and-training-frameworks-and-api) including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds **FP8 quantization support**. We also release **Self-Replay Fine-Tuning**, a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier.
 ### Evaluation
 <table align="center">
   <tr>
+    <td align="center"><b>1B</b><br><sub>iPhone 17 Pro Max</sub></td>
   </tr>
   <tr>
     <td align="center"><img src="./minicpm_v4dot7_assets/minicpmv4.7/v47_1b_iphone_en.gif" height="520"/></td>
 </details>
+#### Speculative decoding (DSpark) <!-- omit in toc -->
+<details>
+<summary>Click to show speculative decoding. </summary>
+Speculative decoding (DSpark): we also release [MiniCPM-V-4.7-1B-DSpark](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark), a DSpark draft model trained for [MiniCPM-V-4.7-1B](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B).
+</details>
 #### Inference with OpenVINO™
 [OpenVINO™](https://docs.openvino.ai) is an open-source toolkit for optimizing and deploying AI inference on Intel CPU/GPU/NPU platforms. OpenVINO provides day-0 support for MiniCPM-V-4.7 on Intel platforms with high-performance inference. See the [OpenVINO™ Toolkit](https://github.com/openvinotoolkit) to get started. You can install and run the example with this notebook (link TBD).
 #### Inference and Fine-Tuning
+For supported inference and training frameworks, see [Supported Inference and Training Frameworks and API](https://github.com/yiranyyu/MiniCPM-o-private/blob/main/docs/deployment_en.md#supported-inference-and-training-frameworks-and-api). The [MiniCPM-V 4.7 Cookbook](https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/docs/pages/v4.7/overview.md) covers deployment, fine-tuning, and quantization workflows.
 **Task guides**
 </details>
+## Model Weights
 MiniCPM-V 4.7 is available in the following variants: