GGUF
Chinese
English
spark-x2.5
llama.cpp
speculative-decoding
t4
cuda
colab
edge-deployment
conversational
Instructions to use gasschina/Spark-X2.5-4B-build-cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use gasschina/Spark-X2.5-4B-build-cpp with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0 # Run inference directly in the terminal: llama cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Use Docker
docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
- LM Studio
- Jan
- Ollama
How to use gasschina/Spark-X2.5-4B-build-cpp with Ollama:
ollama run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
- Unsloth Desktop
- Pi
How to use gasschina/Spark-X2.5-4B-build-cpp with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "gasschina/Spark-X2.5-4B-build-cpp:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use gasschina/Spark-X2.5-4B-build-cpp with Docker Model Runner:
docker model run hf.co/gasschina/Spark-X2.5-4B-build-cpp:Q8_0
- Lemonade
How to use gasschina/Spark-X2.5-4B-build-cpp with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Run and chat with the model
lemonade run user.Spark-X2.5-4B-build-cpp-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use gasschina/Spark-X2.5-4B-build-cpp with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use gasschina/Spark-X2.5-4B-build-cpp with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf gasschina/Spark-X2.5-4B-build-cpp:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "gasschina/Spark-X2.5-4B-build-cpp:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Upload scripts/oneclick_start.sh with huggingface_hub
Browse files- scripts/oneclick_start.sh +87 -0
scripts/oneclick_start.sh
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# ============================================================================
|
| 3 |
+
# oneclick_start.sh — Spark-X2.5-4B Colab T4 一键启动(实战验证版 v2)
|
| 4 |
+
# 位置: /content/drive/MyDrive/spark-t4/oneclick_start.sh (Drive 持久化)
|
| 5 |
+
#
|
| 6 |
+
# 用法(Colab 任意 cell):
|
| 7 |
+
# !bash /content/drive/MyDrive/spark-t4/oneclick_start.sh
|
| 8 |
+
#
|
| 9 |
+
# 默认配置(2026-09-06 第二次实测通过,含 zagent 实战):
|
| 10 |
+
# Q8_0 权重 + CTX=500000(500k) + KV q4_0 + fa on + 别名 spark-x2.5-4b
|
| 11 |
+
# 实测显存 11.8GB/15GB,生成 ~50 tok/s,流式/工具调用全通过
|
| 12 |
+
# zagent 实战: 84 工具调用 12 次、7 次 LLM 调用 240k prompt tokens 全部成功
|
| 13 |
+
#
|
| 14 |
+
# 换配置(示例):
|
| 15 |
+
# 极限长上下文: QUANT=Q4_K_M CTX=786432 !bash .../oneclick_start.sh (实测14.3GB)
|
| 16 |
+
# 短上下文高质量KV: QUANT=Q8_0 CTX=131072 KVQ=q8_0 !bash .../oneclick_start.sh
|
| 17 |
+
# (T4 上 1M 上下文物理放不下: 1M KV@q4_0 也需 13GB+,加权重必爆)
|
| 18 |
+
# ============================================================================
|
| 19 |
+
set -euo pipefail
|
| 20 |
+
|
| 21 |
+
BASE_DIR="/content/drive/MyDrive/spark-t4"
|
| 22 |
+
MODEL_DIR="/content/spark-t4-models" # 模型永不进 Drive(会话重置后自动重下)
|
| 23 |
+
export BASE_DIR MODEL_DIR
|
| 24 |
+
export QUANT="${QUANT:-Q8_0}"
|
| 25 |
+
export CTX="${CTX:-500000}"
|
| 26 |
+
export PORT="${PORT:-8100}" # 8080 被 Colab 环境占用
|
| 27 |
+
export NP="${NP:-1}"
|
| 28 |
+
export KVQ="${KVQ:-q4_0}"
|
| 29 |
+
export BATCH="${BATCH:-512}"
|
| 30 |
+
export UBATCH="${UBATCH:-512}"
|
| 31 |
+
export EXTRA_ARGS="${EXTRA_ARGS:--np 1 -b 512 -ub 512 -fa on --alias spark-x2.5-4b}"
|
| 32 |
+
export BUILD_JOBS="${BUILD_JOBS:-2}"
|
| 33 |
+
TUNNEL="${TUNNEL:-1}" # 1=自动开公网隧道
|
| 34 |
+
|
| 35 |
+
GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RED='\033[0;31m'; CYAN='\033[0;36m'; NC='\033[0m'
|
| 36 |
+
info() { echo -e "${GREEN}[一键启动]${NC} $*"; }
|
| 37 |
+
warn() { echo -e "${YELLOW}[等待]${NC} $*"; }
|
| 38 |
+
fail() { echo -e "${RED}[失败]${NC} $*"; exit 1; }
|
| 39 |
+
|
| 40 |
+
[[ -d /content/drive/MyDrive ]] || fail "Drive 未挂载,请先在 Colab 挂载 Google Drive"
|
| 41 |
+
|
| 42 |
+
# 1. 编译产物检查(在 Drive,跨会话持久)
|
| 43 |
+
BIN="$BASE_DIR/llama.cpp-spark/build/bin/llama-server"
|
| 44 |
+
if [[ -x "$BIN" ]]; then
|
| 45 |
+
info "检测到已编译的 llama-server,跳过编译"
|
| 46 |
+
else
|
| 47 |
+
warn "未检测到编译产物,开始完整编译(首次约 40~60 分钟)..."
|
| 48 |
+
bash "$BASE_DIR/deploy_spark_t4.sh" build
|
| 49 |
+
fi
|
| 50 |
+
|
| 51 |
+
# 2. 模型检查(在本地盘,会话重置后自动重下,约 1~3 分钟)
|
| 52 |
+
if [[ -f "$MODEL_DIR/Spark-X2.5-4B-${QUANT}.gguf" ]]; then
|
| 53 |
+
info "检测到模型: ${QUANT}"
|
| 54 |
+
else
|
| 55 |
+
warn "模型不在本地盘(新会话正常现象),开始下载 ${QUANT} ..."
|
| 56 |
+
mkdir -p "$MODEL_DIR"
|
| 57 |
+
bash "$BASE_DIR/deploy_spark_t4.sh" download
|
| 58 |
+
fi
|
| 59 |
+
|
| 60 |
+
# 3. 起 API 服务(等待加载完成)
|
| 61 |
+
info "启动 API (ctx=$CTX port=$PORT quant=$QUANT kv=$KVQ)..."
|
| 62 |
+
bash "$BASE_DIR/spark_api.sh" start
|
| 63 |
+
|
| 64 |
+
# 4. 公网隧道
|
| 65 |
+
if [[ "$TUNNEL" == "1" ]]; then
|
| 66 |
+
PORT="$PORT" bash "$BASE_DIR/tunnel_api.sh" start
|
| 67 |
+
fi
|
| 68 |
+
|
| 69 |
+
# 5. 快速自检
|
| 70 |
+
KEY="$(cat "$BASE_DIR/.api_key" 2>/dev/null || echo "")"
|
| 71 |
+
AUTH=(); [[ -n "$KEY" ]] && AUTH=(-H "Authorization: Bearer $KEY")
|
| 72 |
+
if curl -s --max-time 60 "http://127.0.0.1:$PORT/v1/chat/completions" \
|
| 73 |
+
"${AUTH[@]}" -H "Content-Type: application/json" \
|
| 74 |
+
-d '{"model":"spark","messages":[{"role":"user","content":"回复:OK"}],"max_tokens":64,"chat_template_kwargs":{"enable_thinking":false}}' \
|
| 75 |
+
| grep -q '"content"'; then
|
| 76 |
+
echo -e "${CYAN}════════════════════════════════════════════════${NC}"
|
| 77 |
+
echo -e "${GREEN} Spark-X2.5-4B 服务就绪 (E2E 自检通过)${NC}"
|
| 78 |
+
echo -e " 本机: http://127.0.0.1:$PORT/v1/chat/completions"
|
| 79 |
+
[[ -n "$KEY" ]] && echo -e " Key: $KEY"
|
| 80 |
+
echo -e " 上下文: $CTX tokens | 权重: $QUANT | KV: ${KVQ:-f16}"
|
| 81 |
+
[[ "$TUNNEL" == "1" ]] && echo -e " 公网: https://aitun.cc/$(grep -oE 'aitun\.cc/[A-Za-z0-9]+' /content/aitun-api-tunnel.log 2>/dev/null | tail -1 | cut -d/ -f2)/v1/chat/completions"
|
| 82 |
+
echo -e " 流式: 请求体加 \"stream\": true"
|
| 83 |
+
echo -e " 关思考: 请求体加 \"chat_template_kwargs\":{\"enable_thinking\":false}"
|
| 84 |
+
echo -e "${CYAN}════════════════════════════════════════════════${NC}"
|
| 85 |
+
else
|
| 86 |
+
fail "自检未通过,查看: tail -30 $BASE_DIR/api-server.log"
|
| 87 |
+
fi
|