Instructions to use stornic56/Spark-X2.5-1.7B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use stornic56/Spark-X2.5-1.7B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use stornic56/Spark-X2.5-1.7B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "stornic56/Spark-X2.5-1.7B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stornic56/Spark-X2.5-1.7B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Ollama
How to use stornic56/Spark-X2.5-1.7B-GGUF with Ollama:
```
ollama run hf.co/stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use stornic56/Spark-X2.5-1.7B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use stornic56/Spark-X2.5-1.7B-GGUF with Docker Model Runner:
```
docker model run hf.co/stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M
```

Lemonade

How to use stornic56/Spark-X2.5-1.7B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Spark-X2.5-1.7B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use stornic56/Spark-X2.5-1.7B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use stornic56/Spark-X2.5-1.7B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "stornic56/Spark-X2.5-1.7B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Llamacpp imatrix Quantizations of Spark-X2.5-1.7B by XHToken

Using XHToken/llama.cpp fork commit b10514-4a3635c32 (build tag b10514-4a3635c32) for quantization.

Original model: https://huggingface.co/XHToken/Spark-X2.5-1.7B

Companion repo: stornic56/Spark-X2.5-4B-GGUF — same pipeline, same calibration corpus, same fork commit. The family quality floor study below compares both.

Model details:

Parameter count: 1.7B
Input support: text
imatrix: yes - details

How to run

⚠️ Important: the spark2_5 architecture is not supported by mainline llama.cpp. All files in this repo require the XHToken/llama.cpp fork at the commit linked above or newer - see How to run.

Prompt format

The chat template is embedded in every GGUF; run with --jinja and it is applied automatically. The rendered format is:

<｜start▁of▁sentence｜><|System|>
{system_prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>
{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|>
<think>

The model is a thinking model by default: reasoning is emitted inside <think>...</think> before the final answer. The embedded template also supports tool calling, matching the upstream chat_template.jinja.

Don't know which to choose? Grab Spark-X2.5-1.7B-Q4_K_M.gguf (1.03GB) - usually a good mix of size and performance. Download instructions available here.

Available files:

Filename	Quant type	File Size	Description
Spark-X2.5-1.7B-bf16.gguf	bf16	3.2GB	Full BF16 weights, converted directly from upstream safetensors.
Spark-X2.5-1.7B-Q8_0.gguf	Q8_0	1.7GB	Extremely high quality, generally unneeded but max available quant.
Spark-X2.5-1.7B-Q6_K.gguf	Q6_K	1.3GB	Very high quality, near perfect.
Spark-X2.5-1.7B-Q5_K_M.gguf	Q5_K_M	1.2GB	High quality.
Spark-X2.5-1.7B-Q4_K_M.gguf	Q4_K_M	1.0GB	Recommended default for most use cases.
Spark-X2.5-1.7B-IQ4_NL.gguf	IQ4_NL	1.0GB	Similar quality to Q4_K_M in a smaller file.
Spark-X2.5-1.7B-IQ3_M.gguf	IQ3_M	842MB	Quality floor — see family study below.

Q3_K_M is intentionally absent: it failed the quality gate on this model (see Family quality floor study below). IQ3_M, which passed, is the smallest recommended quant.

Downloading using the Hugging Face CLI

Click to view download instructions

First, make sure you have the Hugging Face CLI installed:

pip install -U "huggingface_hub[cli]"

Download a specific file:

hf download stornic56/Spark-X2.5-1.7B-GGUF --include "Spark-X2.5-1.7B-Q4_K_M.gguf" --local-dir ./

How to run

These quants require the XHToken llama.cpp fork. Mainline llama.cpp does not implement spark2_5 and will refuse to load these files.

git clone https://github.com/XHToken/llama.cpp.git && cd llama.cpp
# Intel/AMD GPU via Vulkan: add -DGGML_VULKAN=ON
# NVIDIA GPU:               add -DGGML_CUDA=ON
# CPU only:                 plain build works
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON
cmake --build build --config Release -j 8

Interactive chat (template is embedded, --jinja applies it):

./build/bin/llama-cli -m Spark-X2.5-1.7B-Q4_K_M.gguf -ngl 99 -c 8192 --jinja -cnv

OpenAI-compatible server:

./build/bin/llama-server -m Spark-X2.5-1.7B-Q4_K_M.gguf -ngl 99 -c 16384 --jinja

Recommended sampling from the upstream model card: temp 1.0, top_p 0.95, top_k -1. Use --temp 0 for deterministic output.

Other runtimes: Ollama and LM Studio can run these files, but only with builds backed by the same fork - see the "Ollama" and "LM Studio" sections of the official model card for the build steps.

Flash Attention

The fork enables FlashAttention automatically for the sliding-window layers of spark2_5. No extra flag is needed.

imatrix

All quants below bf16 were made using the imatrix option, computed with llama-imatrix from this fork. The calibration corpus is the wikitext-2-raw-v1 train split (Salesforce/wikitext), written to disk verbatim and in order by reproducibility/calibration.py. The corpus file is included in this repo: calibration_data.txt.

The imatrix is available here: Spark-X2.5-1.7B-imatrix.gguf.

Calibration details

{
  "dataset": "Salesforce/wikitext",
  "config": "wikitext-2-raw-v1",
  "split": "train",
  "rendering": "raw text, no chat template, no special tokens",
  "chunk_size": 512,
  "chunks": 128,
  "tokens_consumed": 65536,
  "consumption_order": "sequential from file start (deterministic)",
  "threads": 8,
  "final_perplexity": "141.2760 +/- 3.43585",
  "imatrix_format": "GGUF (fork default)",
  "notes": "corpus is plain prose; a future revision may render tool-calling and reasoning conversations through the chat template, as done by other maintainers"
}

Single clean imatrix run (determinism across runs was demonstrated in the 4B project with this exact pipeline and corpus; see the companion 4B repo). Run log: reproducibility/imatrix-run.log.

Which file should I choose?

Click here for details

The first thing to figure out is how big a model you can run. If you want the model running as fast as possible, fit the whole thing in VRAM: aim for a file 1-2GB smaller than your GPU's total VRAM, leaving room for the KV cache (this model's hybrid attention keeps the cache small, which helps). If you want maximum quality and can tolerate partial CPU offload, add your system RAM and VRAM together and pick a file 1-2GB smaller than that total.

Next, K-quant or I-quant? On NVIDIA (CUDA) and Apple silicon, I-quants (IQX_X) generally offer better quality per bit below Q4. On Intel Arc via Vulkan (Mesa), our measurements show the opposite for generation speed: K-quants are faster at equal bit-width, while I-quants win on file size - see the benchmarks below. Either way, both families were validated for coherence in this release.

Quality Gate (deterministic test, temp 0, seed 1)

All published quants produce a complete, logically correct solution to a Python is_prime coding prompt. Excluded quants and their failure modes:

Q3_K_M — paragraph-level repetition loop in the reasoning trace, never reached a final answer even with 2× token budget.
Q2_K — repetition loop + broken logic (labels 27 and 33 as prime).
IQ2_M — hard repetition loop + hallucinated terminology.

Raw outputs: reproducibility/iq2_m_q2_k_family_study.log

Family quality floor study: 4B vs 1.7B

Same calibration data, same protocol, same fork:

Quant	Spark-X2.5-4B	Spark-X2.5-1.7B (this repo)
IQ2_M	✅ passes clean	❌ repetition loop
Q2_K	❌ repetition loops	❌ loops + broken logic
Q3_K_M	✅ passes (fastest tg)	❌ reasoning repetition loop
IQ3_M	✅ passes	✅ passes

Conclusion: the usable quality floor scales up as models shrink. At 4B the real floor sits between 2 and 3 bits (IQ2_M viable); at 1.7B it is exactly 3-bit and specifically the imatrix-quantized IQ3_M — the K-quant Q3_K_M at similar bpw fails while IQ3_M passes, so --imatrix quantization is not optional for small models in this family.

Benchmarks

Measured with llama-bench from the same fork commit, Intel Arc B580 (Battlemage, Mesa Vulkan driver), -ngl 99 -t 4, 3 runs. Raw log: reproducibility/benchmarks_gpu.txt.

Vulkan (Intel Arc B580 12GB)

Quant	pp512 t/s	tg128 t/s
Q8_0	4345.9	135.6
Q6_K	3962.1	141.8
Q5_K_M	4168.5	153.4
Q4_K_M	4249.1	174.1
IQ4_NL	4354.4	136.9
IQ3_M	4280.6	157.7

On Vulkan, K-quants generate faster than IQ quants (as with the 4B).
IQ3_M is an interesting exception: it is faster than Q6_K, Q8_0 and IQ4_NL in generation on this GPU despite the lowest bpw.

CPU (i3-12100F 4C/8T, `-t 8`, r=3)

Quant	pp512 t/s	tg128 t/s
Q8_0	129.1	21.0
Q6_K	92.7	25.9
Q5_K_M	76.0	28.8
Q4_K_M	155.4	31.5
IQ4_NL	147.1	33.7
IQ3_M	30.9	20.2

Backend-dependent trade-off: IQ3_M is fast on Vulkan but slow on CPU (generic dequant kernels). If you run CPU-only, prefer Q4_K_M / IQ4_NL.

Validation

test-llama-archs -a spark2_5 from the fork: OK on CPU (NMSE 0.00e+00), Vulkan (NMSE 9.23e-08) and meta buffers. Roundtrip: SKIP is expected for this architecture. Raw log: reproducibility/test-llama-archs.log
Every published quant passed a deterministic coding coherence test (--temp 0 --seed 1, 700-token budget): complete, logically correct final answer, no repetition. The excluded quants (Q3_K_M, Q2_K, IQ2_M) and the 1500-token Q3_K_M re-run are documented with raw outputs in reproducibility/iq2_m_q2_k_family_study.log.
bf16 sanity check passed before quantization (coherent reasoning + answer on CPU)

Reproducibility

Everything needed to rebuild these files bit-for-bit is in the repo:

File	Content
SHA256SUMS.txt	Checksums of all GGUFs, imatrix and calibration corpus
Spark-X2.5-1.7B-imatrix.gguf	The importance matrix itself
calibration_data.txt	Calibration corpus, verbatim
reproducibility/calibration.py	Exact corpus generation script
reproducibility/fork_commit.txt	Fork commit used for conversion, imatrix and quantization
reproducibility/base_model_revision.txt	Upstream safetensors revision
reproducibility/imatrix-run.log	Imatrix run output (output of the single clean imatrix run)
reproducibility/benchmarks_gpu.txt	Raw llama-bench output
reproducibility/benchmarks_cpu.txt	Full llama-bench CPU output
reproducibility/iq2_m_q2_k_family_study.log	Raw quality-gate outputs for all excluded quants + Q3_K_M re-run

Exact commands:

python convert_hf_to_gguf.py Spark-X2.5-1.7B --outfile Spark-X2.5-1.7B-bf16.gguf --outtype bf16

./build/bin/llama-imatrix \
  -m Spark-X2.5-1.7B-bf16.gguf \
  -f calibration_data.txt \
  -o Spark-X2.5-1.7B-imatrix.gguf \
  -c 512 --chunks 128 -t 8

./build/bin/llama-quantize --imatrix Spark-X2.5-1.7B-imatrix.gguf \
  Spark-X2.5-1.7B-bf16.gguf Spark-X2.5-1.7B-Q4_K_M.gguf Q4_K_M

ARM/AVX information

llama.cpp automatically repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines, covering Q4_0, IQ4_NL and most K-quants. No special quant choice is needed for CPU inference.

Credits

Thanks to the XHToken/SparkLLM team for releasing the model and maintaining the llama.cpp fork, and to bartowski for the quantization card format this repo follows.

License & attribution

The upstream model is released under Apache License 2.0; these derivative quantizations inherit it. Preserve upstream attribution when redistributing. This is an unofficial community release, not endorsed by XHToken/SparkLLM.

Downloads last month: 47

GGUF

Model size

2B params

Architecture

spark2_5

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

View +1 variant

Model tree for stornic56/Spark-X2.5-1.7B-GGUF

Base model

XHToken/Spark-X2.5-1.7B-Base

Finetuned

XHToken/Spark-X2.5-1.7B

Quantized

(7)

this model

Llamacpp imatrix Quantizations of Spark-X2.5-1.7B by XHToken

Prompt format

Available files:

Downloading using the Hugging Face CLI

How to run

Flash Attention

imatrix

Which file should I choose?

Quality Gate (deterministic test, temp 0, seed 1)

Family quality floor study: 4B vs 1.7B

Benchmarks

Vulkan (Intel Arc B580 12GB)

CPU (i3-12100F 4C/8T, -t 8, r=3)

Validation

Reproducibility

ARM/AVX information

Credits

License & attribution

Model tree for stornic56/Spark-X2.5-1.7B-GGUF

CPU (i3-12100F 4C/8T, `-t 8`, r=3)