How to Run MiniCPM-V 4.7 35B-A3B Without Refusals Locally

One refusal direction projected out of OpenBMB's original weights, shipped two ways: three baked GGUF builds, and a 106 MB LoRA adapter for the MiniCPM-V 4.7 quants you already have. The measurements behind this card are public.

On held-out prompts, refusals fall from 96.3% to 0% in English and from 70% to 0% in Russian, and from 13 of 30 to none with thinking on. MMLU moves by 0.25 points, inside the noise.
The 20.1 GB build matches the original's token choice 94.5% of the time; our quant of the unmodified model at the same size does 94.9%.
Already running our MiniCPM-V 4.7 quants? Add the adapter with --lora-scaled instead of downloading a new model.

The builds

Build	On disk	Mean KLD	Same top-1	Refused: EN / RU / XSTest
`AD-Q4_K-IQ4_XS`	20.1 GB	0.0232	94.47%	0% / 0% / 0%
`AD-Q5_K-Q4_K`	22.1 GB	0.0191	94.84%	0% / 0% / 0%
`AD-Q6_K`	29.1 GB	0.0109	96.15%	0% / 0% / 0%

AD-Q4_K-IQ4_XS is the one for a 24 GB card, with room for a long context; take AD-Q6_K if it fits.

KLD and top-1 are against the original, unmodified BF16 model, so they hold the ablation and the quantization together. For scale: the ablation alone, on BF16 weights, is 0.0090; our plain Q8_0 of the original is 0.0051. The same layouts of the unmodified model, from our quants:

Layout	On disk	Unmodified: KLD / top-1	This repo: KLD / top-1
`AD-Q4_K-IQ4_XS`	20.1 GB	0.0197 / 94.94%	0.0232 / 94.47%
`AD-Q5_K-Q4_K`	22.1 GB	0.0150 / 95.47%	0.0191 / 94.84%
`AD-Q6_K`	29.1 GB	0.0061 / 97.18%	0.0109 / 96.15%

The names follow our original quants: AD-<ffn_down>-<ffn_up> is what the routed experts got; attention, the shared expert, the embedding and the output head stay at q8_0.

What changed, and what did not

Original against ablated, both BF16, on prompts never used to pick anything:

	Original	Ablated
JBB harmful behaviours (EN, 81), refused	96.3%	0%
Aya red-teaming (RU, 100), refused	70.0%	0%
XSTest safe prompts (250), refused	7.6%	0%
Thinking on (30 prompts), refused	43%	0%
Thinking on, traces left unclosed at 3072 tokens	16 of 30	0 of 30
MMLU, 2000 questions	84.25%	84.50% (+0.25, McNemar p = 0.52)
Tool calls (20)	20/20 valid	20/20 valid
Needle in a long context (3 depths)	3/3	3/3
Mean KLD to the original, neutral / code	-	0.0090 / 0.0095

Refusal is counted by the opening of the reply, so it is indicative, not a judge; empty or degenerate replies count as damage, and there were none. The baked builds were checked the same way on the same test prompts.

The adapter

lora/MiniCPM-V-4.7-35B-A3B-Uncensored-LoRA.gguf: rank 1, 106 MB, 120 sites. It applies to any GGUF of the original MiniCPM-V-4.7-35B-A3B weights, including our published quants. Use scale 1.0:

llama-server -m MiniCPM-V-4.7-35B-A3B-AD-Q4_K-IQ4_XS.gguf -ngl 99 --jinja \
  --lora-scaled lora/MiniCPM-V-4.7-35B-A3B-Uncensored-LoRA.gguf:1.0

The adapter was measured on the BF16 weights (the table above); on a quant it adds the same edit on top of that quant's own error.

Get started

llama.cpp at or after 69f201a (MiniCPM-V 4.7 support, ggml-org/llama.cpp#29416). Download one build and point -m at its first shard:

hf download AtomicChat/MiniCPM-V-4.7-35B-A3B-Uncensored-GGUF \
  --include "MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS/*" --local-dir .

llama-server \
  -m MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS/MiniCPM-V-4.7-35B-A3B-Uncensored-AD-Q4_K-IQ4_XS-00001-of-00011.gguf \
  --mmproj mmproj-MiniCPM-V-4.7-35B-A3B-Uncensored-F16.gguf \
  -ngl 99 -c 32768 --jinja

With --jinja the model's own chat template starts every answer in thinking mode; turn it off per request with "chat_template_kwargs": {"enable_thinking": false}.

How it was made

Direction. Difference of means of the residual stream at the last prompt token, chat template applied, thinking off: 416 harmful against 416 harmless English prompts, taken entering block 28 of 40, with the harmless-mean component removed.
Edit. W' = W - r rᵀW on the matrices that write into the residual: 40 attention and linear-attention outputs, the down projections of all 256 experts in all 40 MoE blocks, and 40 shared experts. The token embedding is left alone: the vision projector copies rows of it.
Choice. Directions from four depths at four strengths, then narrower writer sets, one direction per block, English only or English + Russian, and a random direction as the control, all on validation prompts; the numbers on this card come from held-out test prompts. The screen's own pick (one direction per block, scale 0.75) drifted a little less on the first token but still refused 5 of 30 prompts with thinking on, which this template does by default, so this release uses the English direction at scale 1.0, which refused none.
Bake. The same edit applied to the BF16 weights in f32 and rounded once; every one of the 120 writers checked to keep none of its component along r, within 7.6e-3 after bf16 rounding. Then quantized with the importance matrix of our original quants, which a 300-chunk matrix of the ablated model agrees with (cosine 0.999 or more).

Everything was measured against one reference, one corpus, one machine: the original BF16 model's own logits over a held-out neutral set, 87 chunks at 4096 context, llama.cpp 8bb3cf0d, 2x RTX PRO 6000 (sm_120). The unmodified quants above were measured on H100s against their own reference from the same weights; our Q8_0 scores 0.0051 on both.

Limitations

The refusal count reads the opening of each reply; a soft refusal phrased as an answer would be missed.
Vision was not tested after the edit. The mmproj files here are the original's projector, unmodified.
Not run yet on a Mac, in LM Studio or in Ollama.
The direction comes from English prompts at one token position. Russian refusals went with it; other languages were not measured.

Responsible use

This model has a learned refusal direction removed, so it will answer requests the original declines. Nothing about it makes the output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

Credits

Direction estimation, adapter build, quantization and evaluation by AtomicChat. Base model MiniCPM-V 4.7 35B-A3B by OpenBMB, under Apache-2.0. The method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).