Instructions to use gasschina/Spark-X2.5-4B-build-cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use gasschina/Spark-X2.5-4B-build-cpp with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Use Docker

docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0

LM Studio
Jan
Ollama
How to use gasschina/Spark-X2.5-4B-build-cpp with Ollama:
```
ollama run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
```
Unsloth Desktop

How to use gasschina/Spark-X2.5-4B-build-cpp with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "gasschina/Spark-X2.5-4B-build-cpp:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use gasschina/Spark-X2.5-4B-build-cpp with Docker Model Runner:
```
docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
```

Lemonade

How to use gasschina/Spark-X2.5-4B-build-cpp with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Run and chat with the model

lemonade run user.Spark-X2.5-4B-build-cpp-Q8_0

List all available models

lemonade list

Hermes Agent

How to use gasschina/Spark-X2.5-4B-build-cpp with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use gasschina/Spark-X2.5-4B-build-cpp with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "gasschina/Spark-X2.5-4B-build-cpp:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Spark-X2.5-4B-build-cpp / scripts /oneclick_start.sh

gasschina

Upload scripts/oneclick_start.sh with huggingface_hub

f538b5b verified 3 days ago

Raw

History Blame Contribute Delete

7 kB

	#!/usr/bin/env bash
	# ============================================================================
	# oneclick_start.sh — Spark-X2.5-4B Colab T4 一键启动（实战验证版 v2）
	# 位置: /content/drive/MyDrive/spark-t4/oneclick_start.sh (Drive 持久化)
	#
	# 用法（Colab 任意 cell）:
	# !bash /content/drive/MyDrive/spark-t4/oneclick_start.sh
	#
	# 默认配置 v6（2026-09-06, 改回 Q8_0+草稿; 4bit+草稿实测反优化 27 tok/s 后回退）:
	# Q8_0 权重 + 1.7B 草稿投机解码 + CTX=262144(256k) + KV q4_0 + fa on + 别名 spark-x2.5-4b + 思考默认关
	# 实测: 显存 11.8GB/15GB, 生成 51-52 tok/s, 草稿接受率 0.89, 质量最佳
	# 4bit 档(QUANT=Q4_K_M): 自动关草稿裸跑 48.75 tok/s, 显存 6.6GB — 4bit 噪声→接受率 0.37-0.61, 挂草稿反而 27 tok/s
	#
	# 换配置（示例）:
	# 高质量+草稿: QUANT=Q8_0 !bash .../oneclick_start.sh (草稿@256k, 52.3 tok/s, 11.8GB)
	# 4bit 长上下文: CTX=500000 !bash .../oneclick_start.sh (实测 51.7 tok/s, 10.1GB, 草稿同样自动关)
	# 极限长上下文: QUANT=Q4_K_M CTX=786432 !bash .../oneclick_start.sh (14.3GB)
	# 强制开草稿(不建议 4bit): SPEC_TYPE=draft-simple !bash .../oneclick_start.sh
	# （T4 上 1M 上下文物理放不下: 1M KV@q4_0 也需 13GB+，加权重必爆）
	# （草稿模式下 CTX>300k 会 OOM: 草稿 KV/计算缓冲继承主上下文）
	# （思考模式默认关: aitun 隧道非流式请求 120s 硬超时, 开思考会 504; THINKING=on 恢复）
	# ============================================================================
	set -euo pipefail

	BASE_DIR="/content/drive/MyDrive/spark-t4"
	MODEL_DIR="/content/spark-t4-models" # 模型永不进 Drive（会话重置后自动重下）
	export BASE_DIR MODEL_DIR
	export QUANT="${QUANT:-Q8_0}" # 默认 Q8_0: 质量最佳+草稿+41%≈52tok/s(11.8GB); 4bit: QUANT=Q4_K_M(6.6GB,自动关草稿)
	export CTX="${CTX:-262144}" # 256k: 草稿投机解码最优档; 无草稿可设 500000
	export PORT="${PORT:-8100}" # 8080 被 Colab 环境占用
	export NP="${NP:-1}"
	export KVQ="${KVQ:-q4_0}"
	export BATCH="${BATCH:-512}"
	export UBATCH="${UBATCH:-512}"
	export EXTRA_ARGS="${EXTRA_ARGS:--np 1 -b 512 -ub 512 -fa on --alias spark-x2.5-4b}"
	export SPEC_TYPE_DEFAULT=1
	# 草稿策略(按量化档自动): Q8_0 挂草稿(接受率0.89,+41%); 4bit 裸跑(接受率0.37-0.61,草稿反而慢一半)
	if [[ "$QUANT" == Q8* ]]; then
	export SPEC_TYPE="${SPEC_TYPE:-draft-simple}"
	else
	export SPEC_TYPE="${SPEC_TYPE:-none}"
	fi
	export DRAFT_MODEL="${DRAFT_MODEL:-/content/spark-t4-models/Spark-X2.5-1.7B-Q8_0.gguf}"
	export DRAFT_NMAX="${DRAFT_NMAX:-8}"
	export THINKING="${THINKING:-off}" # 思考默认关: aitun 隧道非流式 120s 硬超时→504防护; THINKING=on 恢复
	export BUILD_JOBS="${BUILD_JOBS:-2}"
	TUNNEL="${TUNNEL:-1}" # 1=自动开公网隧道

	GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RED='\033[0;31m'; CYAN='\033[0;36m'; NC='\033[0m'
	info() { echo -e "${GREEN}[一键启动]${NC} $*"; }
	warn() { echo -e "${YELLOW}[等待]${NC} $*"; }
	fail() { echo -e "${RED}[失败]${NC} $*"; exit 1; }

	[[ -d /content/drive/MyDrive ]] \|\| fail "Drive 未挂载，请先在 Colab 挂载 Google Drive"

	# 1. 编译产物检查（在 Drive，跨会话持久）
	BIN="$BASE_DIR/llama.cpp-spark/build/bin/llama-server"
	if [[ -x "$BIN" ]]; then
	info "检测到已编译的 llama-server，跳过编译"
	else
	warn "未检测到编译产物，开始完整编译（首次约 40~60 分钟）..."
	bash "$BASE_DIR/deploy_spark_t4.sh" build
	fi

	# 2. 模型检查（在本地盘，会话重置后自动重下，约 1~3 分钟）
	if [[ -f "$MODEL_DIR/Spark-X2.5-4B-${QUANT}.gguf" ]]; then
	info "检测到模型: ${QUANT}"
	else
	warn "模型不在本地盘（新会话正常现象），开始下载 ${QUANT} ..."
	mkdir -p "$MODEL_DIR"
	bash "$BASE_DIR/deploy_spark_t4.sh" download
	fi

	# 2.5 草稿模型检查（投机解码默认开启；缺失自动从 HF 公开仓补下）
	if [[ "$SPEC_TYPE" != "none" && -n "$DRAFT_MODEL" && ! -f "$DRAFT_MODEL" ]]; then
	warn "草稿模型不在本地盘，从 HF 公开仓补下 Spark-X2.5-1.7B-Q8_0 (1.82GB)..."
	command -v hf >/dev/null 2>&1 \|\| pip install -q -U huggingface_hub >/dev/null 2>&1 \|\| true
	mkdir -p "$MODEL_DIR"
	hf download gasschina/Spark-X2.5-4B-build-cpp Spark-X2.5-1.7B-Q8_0.gguf \
	--local-dir "$MODEL_DIR" >/dev/null 2>&1 \|\| true
	fi
	if [[ "$SPEC_TYPE" != "none" && -n "$DRAFT_MODEL" && -f "$DRAFT_MODEL" ]]; then
	info "草稿模型就绪: $DRAFT_MODEL（投机解码实测 +41%）"
	elif [[ "$SPEC_TYPE" != "none" && -n "$DRAFT_MODEL" ]]; then
	warn "草稿模型下载失败，本次无草稿启动（稍后手动补下后重启即可）"
	export DRAFT_MODEL=""
	fi

	# 3. 起 API 服务（等待加载完成）
	info "启动 API (ctx=$CTX port=$PORT quant=$QUANT kv=$KVQ)..."
	bash "$BASE_DIR/spark_api.sh" start

	# 4. 公网隧道
	if [[ "$TUNNEL" == "1" ]]; then
	PORT="$PORT" bash "$BASE_DIR/tunnel_api.sh" start
	fi

	# 5. 快速自检
	KEY="$(cat "$BASE_DIR/.api_key" 2>/dev/null \|\| echo "")"
	AUTH=(); [[ -n "$KEY" ]] && AUTH=(-H "Authorization: Bearer $KEY")
	if curl -s --max-time 60 "http://127.0.0.1:$PORT/v1/chat/completions" \
	"${AUTH[@]}" -H "Content-Type: application/json" \
	-d '{"model":"spark","messages":[{"role":"user","content":"回复:OK"}],"max_tokens":64,"chat_template_kwargs":{"enable_thinking":false}}' \
	\| grep -q '"content"'; then
	echo -e "${CYAN}════════════════════════════════════════════════${NC}"
	echo -e "${GREEN} Spark-X2.5-4B 服务就绪 (E2E 自检通过)${NC}"
	echo -e " 本机: http://127.0.0.1:$PORT/v1/chat/completions"
	[[ -n "$KEY" ]] && echo -e " Key: $KEY"
	echo -e " 上下文: $CTX tokens \| 权重: $QUANT \| KV: ${KVQ:-f16}"
	[[ -n "$DRAFT_MODEL" && "$SPEC_TYPE" != "none" ]] && echo -e " 投机解码: 1.7B 草稿 draft-simple (Q8_0 实测 +41% ≈52 tok/s, 接受率 0.89)"
	[[ "$SPEC_TYPE" == "none" ]] && echo -e " 投机解码: 关闭(4bit 裸跑 48.75 tok/s; 草稿仅对 Q8_0 有正收益)"
	echo -e " 思考模式: $([[ "$THINKING" == "off" ]] && echo '默认关闭(网关504防护, 单请求可传 enable_thinking:true)' \|\| echo '默认开启')"
	[[ "$TUNNEL" == "1" ]] && echo -e " 公网: https://aitun.cc/$(grep -oE 'aitun\.cc/[A-Za-z0-9]+' /content/aitun-api-tunnel.log 2>/dev/null \| tail -1 \| cut -d/ -f2)/v1/chat/completions"
	echo -e " 流式: 请求体加 \"stream\": true"
	echo -e " 开思考(单请求): 请求体加 \"chat_template_kwargs\":{\"enable_thinking\":true}"
	echo -e "${CYAN}════════════════════════════════════════════════${NC}"
	else
	fail "自检未通过，查看: tail -30 $BASE_DIR/api-server.log"
	fi