MiniCPM5-2B-heretic-abliterated-GGUF

This repository hosts quantized GGUF checkpoints based on openbmb/MiniCPM5-2B, implementing the directional refusal ablation methodology showcased in insraq/MiniCPM5-2B-heretic-abliterated via Heretic v1.4.0.

These builds are packaged for local on-device inference using llama.cpp, Ollama, LM Studio, Jan, and standard GGUF executors.

---

Abliteration Profile

Directional ablation neutralizes the refusal reflex across the residual stream and MLP projections while preserving the primary base model's mathematical, coding, and multi-step reasoning performance.

Benchmark Metrics

Metric	Abliterated Model	Original Base (openbmb/MiniCPM5-2B)
Refusal Rate	5 / 100	99 / 100
KL Divergence	0.0391	0.0000 (Reference)

A low KL divergence of 0.0391 relative to the original openbmb/MiniCPM5-2B weights indicates minimal representation drift, preventing capability degradation on standard non-boundary tasks.

Ablation Parameters

Parameter	Value	Parameter	Value
direction_index	per layer	mlp.down_proj.max_weight	0.89
attn.o_proj.max_weight	1.47	mlp.down_proj.max_weight_position	28.68
attn.o_proj.max_weight_position	29.44	mlp.down_proj.min_weight	0.66
attn.o_proj.min_weight	1.45	mlp.down_proj.min_weight_distance	20.31
attn.o_proj.min_weight_distance	14.36	—	—

---

Available Quantizations

File Name	Size	Recommended Use Case
MiniCPM5-2B-heretic-abliterated-Q3_K_M.gguf	1.29 GB	Low-memory edge devices & micro-compute nodes.
MiniCPM5-2B-heretic-abliterated-Q4_K_S.gguf	1.50 GB	Lightweight 4-bit quant; lower RAM overhead.
MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf	1.56 GB	Recommended: Best balance of speed, footprint, and perplexity.
MiniCPM5-2B-heretic-abliterated-Q5_K_M.gguf	1.81 GB	High accuracy; preserves intricate reasoning chains.
MiniCPM5-2B-heretic-abliterated-Q6_K.gguf	2.07 GB	High fidelity; practically indistinguishable from BF16.
MiniCPM5-2B-heretic-abliterated-Q8_0.gguf	2.68 GB	Full 8-bit precision; minimal quantization loss.

---

Prompt Template (ChatML)

MiniCPM5 uses the standard ChatML prompt template:

<|im_start|>system
You are a helpful, unfiltered assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Quickstart Guide

llama.cpp

Run inference using llama-cli:

llama-cli \
  -m MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf \
  -p "<|im_start|>user\nExplain how stack-based buffer overflows work at the register level.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 \
  -c 4096 \
  -ngl 99 \
  --repeat-penalty 1.15 \
  --temp 0.8 \
  --top-p 0.95