Instructions to use openbmb/MiniCPM5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM5-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openbmb/MiniCPM5-2B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B")
model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openbmb/MiniCPM5-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM5-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM5-2B

SGLang

How to use openbmb/MiniCPM5-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM5-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM5-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-2B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM5-2B with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM5-2B
```

## Question about LongBench v2 evaluation with the 128K context limit

by pengwenzhi - opened 4 days ago

Discussion

pengwenzhi

4 days ago

Hi MiniCPM team,

Thanks for releasing MiniCPM5-2B!

I have a question about the LongBench v2 result reported in the model card.

The model card specifies that MiniCPM5-2B has a native context length of 131,072 tokens (128K), while the reported LongBench v2 score is 43.7.

However, LongBench v2 contains examples with very long contexts, ranging from roughly 8K to 2M words, and some prompts can exceed 128K tokens by a large margin, with the longest examples potentially reaching well beyond 1M tokens depending on the tokenizer.

Could you please clarify the evaluation protocol used for the reported LongBench v2 score?

Specifically:

Was the full LongBench v2 test set of 503 examples evaluated?
For examples whose tokenized prompt length exceeded 128K, how were they handled?
- Were those examples filtered out?
- Was the context truncated to fit within 128K?
- If truncated, what truncation strategy was used (e.g. middle truncation, head/tail retention, etc.)?
- Or was MiniCPM5-2B evaluated with a context window extended beyond its stated 128K context length?
If only examples within the 128K context limit were evaluated, how many LongBench v2 examples remained, and was the reported 43.7 calculated only over that subset?
Could you share the evaluation script/configuration or the exact LongBench v2 reproduction settings used to obtain the 43.7 score?

We are trying to reproduce the LongBench v2 evaluation under the same setting, so having the exact handling of samples exceeding the native context window would be very helpful.

Thanks!

funmaker

OpenBMB org 2 days ago

Hi, thanks for digging into this. Here's exactly what was run.

Short version: all 503 examples were evaluated and 43.7 is the full-set score. Over-long prompts were middle-truncated, consistent with the official LongBench v2 handling.

1. Full 503-example set?
Yes — 503/503 scored, no length-based filtering anywhere.

2. Prompts over the context limit?
Neither filtered nor run with an extended window — the model was served at its native 128K. Prompts are truncated to 120,000 tokens, the cap the official LongBench v2 evaluation applies to 128K-context models.

3. Truncation strategy?
Middle truncation, matching the official approach: keep the first and last half of the budget, drop the middle (60K head + 60K tail). Template, question, and all four choices are always retained in full.

4. Exact settings
Two-stage evaluation identical in structure to the official pipeline: stage 1 follows the official CoT prompt with the full context; stage 2 re-asks the model for the final answer using the official follow-up prompt, which injects
the stage-1 reasoning and replaces the long context with an omission placeholder. Answer extraction follows the official The correct answer is (X) format.

Prompt cap 120,000 tokens, middle truncation, chat template applied, tokenizer = the evaluated model's own
No dataset-side output-length cap on either stage; limits come from the serving config
Sampling: temperature 1.0, top_p 0.95, thinking mode enabled
Scoring: accuracy over all 503

Thanks again for your interest in MiniCPM5. We hope this clarification helps.

pengwenzhi

2 days ago

Get it! Thanks for the clarification! This is very helpful and answers my question.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment