Instructions to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
# Run inference directly in the terminal:
llama cli -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
# Run inference directly in the terminal:
llama cli -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
# Run inference directly in the terminal:
./llama-cli -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Use Docker

docker model run hf.co/SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

LM Studio
Jan

vLLM

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SC117/Spark-X2.5-4B-abliterated-FIT-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Spark-X2.5-4B-abliterated-FIT-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Ollama
How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with Ollama:
```
ollama run hf.co/SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
```
Unsloth Desktop

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with Docker Model Runner:
```
docker model run hf.co/SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S
```

Lemonade

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Run and chat with the model

lemonade run user.Spark-X2.5-4B-abliterated-FIT-GGUF-Q4_K_S

List all available models

lemonade list

Hermes Agent

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SC117/Spark-X2.5-4B-abliterated-FIT-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SC117/Spark-X2.5-4B-abliterated-FIT-GGUF:Q4_K_S" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

FIT-GGUF v0.2.0GATE-VERIFIED TIERS4 TIERS · 2.26–3.15 GiBABLITERATED (T615)MEASURED KL + SAME-TOPAPACHE-2.0

Spark-X2.5-4B-abliterated · FIT-GGUF

Four fidelity tiers of a 1M-context hybrid-attention 4B — every shipped file re-verified against its own BF16, byte-exact and reproducible.

2.26 GiBverified minimum at each fidelity3.15 GiB

English · 简体中文 📖

🧭 About FIT-GGUF — the tool behind these files

Every file in this repository was planned, executed and verified by FIT-GGUF, an open-source, deterministic tensor-level planning layer on top of llama.cpp quantization. Standard GGUF quantization asks you to pick one of a handful of presets; FIT-GGUF instead asks what quality do you want, then finds and verifies the smallest GGUF that demonstrably meets it.

Traditional GGUF gives you presets. FIT gives you a fidelity contract: macro KL ≤ tier anchor ∧ same-top ≥ model-calibrated floor.

Deterministic size prediction & byte-exact delivery	✅ Validated (G2 gate, delta = 0)
Universally optimal tensor allocation	⚠️ Not established — FIT claims verified fidelity contracts, not a universal quality optimum

The method, the preregistered research record and the fit CLI are open source: github.com/Scorp1o117/FIT-GGUF

⚠️ Safety notice / 安全提示

The source model is an abliterated, refusal-removed model with no meaningful built-in guardrails, and may comply with harmful, illegal or unsafe requests. Use it only where you can provide appropriate moderation, access control and legal review. Do not deploy it to end users without your own safety layer.

源模型经过拒答方向消融，不具备可靠的内置安全护栏；请仅在合法、受控、具备审核与滥用防护的环境中使用，使用者自行承担部署责任。

🧬 Abliteration — trial T615, documented

The refusal-direction ablation was performed locally with the abliterix pipeline: mean-LoRA steering — rank-8 full-norm LoRA adapters built from per-layer mean refusal directions, with projected abliteration, a gaussian decay kernel over depth, and per-layer vector scope. The exported weights are Optuna trial #615 of study spark_x25_4b_lora_v30, selected from the measured Pareto front.

Metric	Original	T615
Keyword refusals (100 prompts, held-out slice)	100	7
3-token full-distribution KL vs original (nats/token)	—	0.1468
Validation KL (mean over held-out prompts)	—	0.148

Honest selection trade-off: the pre-registered "ship bar" was refusals ≤ 10 ∧ KL ≤ 0.05. T615 clears the refusal bar with margin but deliberately trades weight distance for openness (KL 0.147 > 0.05) — the previous export T580 (KL 0.093) left sexual-content soft-refusals and was replaced by user choice. This is an openness/faithfulness Pareto decision, reported as measured.

Trial	Refusals /100	KL (nats/token)	Note
615	7	0.1468	this export
389	10	0.1433	Pareto neighbor
221	3	0.2299	most open
580	20	0.0929	previous export (leftover refusals)

All fidelity-tier measurements in this repository are taken against the T615 BF16 weights themselves — the ablation delta is baked into the reference, so the tier numbers below quantify quantization loss only.

📦 Pick a tier

File size ≠ RAM/VRAM usage. KV cache, compute buffers and runtime overhead are separate. The native context is 1M tokens; the hybrid attention (3 sliding-window : 1 full) keeps most depth cheap, but the 9 full-attention layers still need ~34 GiB of f16 KV at 1M tokens — pick a sane -c. Naming: Spark-X2.5-4B-abliterated-FIT-<tier>-<size>-<dominant>.gguf

Tier	GiB	Dominant	Macro KL ↓	Same-top ↑	Gates (KL ≤ / top ≥)	Measured positioning
⭐ QUALITY	3.147	Q6_K	0.0289	93.50%	0.05 / 93.16%	Max verified quality; the Q6_K preset is itself the minimum verified PASS
BALANCED	2.680	Q5_K_S	0.0739	89.15%	0.10 / 89.14%	Best size/quality trade-off; nothing smaller passes both gates
COMPACT	2.432	Q4_K	0.1458	84.88%	0.15 / 84.88%	FIT tensor-level recipe — fills the preset gap (Q5_K_S 0.074 → Q4_K_M 0.170)
MINI	2.260	Q4_K_S	0.1834	82.61%	0.20 / 82.61%	Smallest verified PASS

Quick picks

🏆 BALANCED 2.68G — the sweet spot

Q5_K_S-class quality at 2.68 GiB; the verified minimum at the 0.10 KL tier.

🎯 QUALITY 3.15G — max verified quality

93.50% same-top, KL 0.0289 — within 0.47 GiB of the Q8_0 preset with clearly stronger economics.

🧩 COMPACT 2.43G — where FIT earns its keep

No preset exists between KL 0.074 and 0.170; this tensor-level recipe lands at 0.146 and passes both gates.

⚠️ Below ~2.2 GiB

Steep low-bit cliff: 2-bit classes collapse (macro KL 0.8–3.9) and are deliberately not offered.

All numbers are protocol-scoped observations (five fixed 64 KiB domains vs this model's aligned BF16), not an application benchmark or a universal ranking. Honest scope: on this model the preset ladder is strong — Quality/Balanced/Mini minimum verified PASS are the native presets themselves; FIT contributes the verification and the Compact gap-fill. All four tiers are same-top-bound: the guard floors, not KL, decide the sizes.

📈 Measured quality

KL and Same-top curves with presets, calibration probes and shipped tiers labeled

Quality improves monotonically across the healthy preset ladder, and every tier above is same-top-bound — the model-calibrated floors, not KL, are what separate PASS from FAIL here. The 2-bit region is reported as measured: Q2_K/IQ2_* collapse (macro KL 0.8–3.9). The COMPACT tier fills the gap between Q5_K_S (0.074) and Q4_K_M (0.170) where no native preset exists.

Full-size EN · 中文大图

🚀 Run it — runtime requirement first

These GGUFs use the spark2_5 architecture. You need a llama.cpp build with Spark2_5 support — PR #27868 (open at release time, 2026-09) or any later release that includes it. Upstream master without this PR cannot load these files.

llama.cpp

./llama-server \ -m Spark-X2.5-4B-abliterated-FIT-BALANCED-2.68GiB-Q5_K_S.gguf \ -ngl 99 -c 32768

./llama-cli
-m Spark-X2.5-4B-abliterated-FIT-QUALITY-3.15GiB-Q6_K.gguf
-ngl 99 -c 8192 -p "Hello" -n 256

The chat template is embedded in every file. Once Spark2_5 support merges, any llama.cpp-based runner (llama-cli, llama-server, LM Studio, KoboldCpp, Jan, …) loads these files directly. This is a dense 4B text model — no MTP head, no vision projector.

🔬 Evaluation protocol & honest scope

Runtime	llama.cpp @ PR #27868 head (ae320b1) · Linux x86_64 · ROCm (gfx1151) — perplexity/KL code byte-identical to the pinned eval-v1 build
Command shape	`llama-perplexity -ngl 99 -t 16 -c 512 -b 512 --kl-divergence …`
Reference	This model's own BF16 logits (T615 weights)
Domains	wiki_test · wiki_valid · Chinese · code · agent_chat (five fixed 64 KiB slices, macro mean)

Tier verification: a tier ships only if macro KL ≤ its anchor and same-top ≥ a model-calibrated guard floor (15-point preset ladder + 2 gap probes; floors are exact-model-scoped and are not transferred across models). Every shipped file was re-evaluated as the exact shipped bytes; the COMPACT recipe is reproducible byte-for-byte.

Allocator scope: the balanced v0.1 policy was used as-is (no model-specific refine profile exists for spark2_5). This release claims deterministic size planning and measured, gate-verified fidelity for these specific artifacts — it does not claim a universally optimal allocation.

🧩 Included — and not included

✅ 4 gate-verified tier GGUFs · ✅ the BF16 reference GGUF (the exact weights every measurement above is taken against) · ✅ chat template embedded in every file · ✅ checksums (SHA256SUMS.txt) · ✅ labeled quality curves (results/) · ✅ tier manifest (FIT-TIERS.md)

❌ 2-bit quantization classes — measured collapse on this model (macro KL 0.8–3.9), excluded by design · ❌ no vision projector / no MTP head (dense text model)

The abliteration (T615) was performed locally from XHToken/Spark-X2.5-4B-Base; this repository contributes the abliterated weights and the FIT quantization plans and artifacts.

🔍 Verify & reproduce

sha256sum -c SHA256SUMS.txt

The evaluation slices are public in the FIT-GGUF repository; the calibration ladder, gap probes, search audit and guard profile for this release are retained in the FIT-GGUF experiment records (2026-09-06-spark-x25-4tier). Exact-size behavior is scoped to the recorded source metadata and the recorded llama.cpp build; changing the converter, runtime, source layout or metadata requires revalidation.

📄 License & credits

Apache-2.0, inherited from the base model — follow the upstream license and model-card requirements.

XHToken / SparkLLM — the Spark-X2.5 model family · llama.cpp PR #27868 (KnightYao) — Spark2_5 support · abliterix — the local ablation pipeline (trial T615) · FIT-GGUF — verified size-exact quantization. FIT-GGUF is an independent project, not affiliated with XHToken, SparkLLM or llama.cpp.

⭐ FIT-GGUF on GitHub — the tool, the method, the full research record

English · 简体中文