xunlinkx/Spark-X2.6-4B-oQ4e-fp16

Spark-X2.6-4B — MLX oQ4e (FP16 quantization source)

This is an MLX quantization prepared by xunlinkx with the oMLX quantization pipeline from XHToken/Spark-X2.6-4B, the upstream SparkLLM model. This repository links back to the original model and follows its Apache-2.0 license. See the upstream model card and official Spark-X2.6 project for model details, intended use, and limitations.

This repo contains the same model architecture and tokenizer, with quantized weights for local MLX inference. The original checkpoint is BF16; it was converted to FP16 before quantization. The -fp16 suffix records that quantization input format. Quantization used oMLX enhanced calibration (128 samples, 512-token sequence length), group size 64, and affine quantization.

Model details

• Base model: XHToken/Spark-X2.6-4B
• Quantization: 4-bit enhanced mixed-precision
• Quantization layout: 4-bit base weights with selected attention gates stored at 8-bit and selected attention and MLP projections at 6-bit. The full per-layer layout is recorded in config.json.
• Architecture: spark2_5
• License: Apache-2.0, inherited from the upstream model. See the upstream license.
• Intended hardware: Apple Silicon Macs using MLX, including M1 through M6 generations. FP16 quantization source was chosen for broad compatibility across older and newer Apple Silicon.
• Use case: The smallest of these three variants, intended for Macs with tighter unified-memory budgets, including 8 GB systems. Actual memory use depends on context length and concurrent apps.

Loading

Use a recent oMLX installation with Spark spark2_5 architecture support. Select this repository as the model in oMLX. The saved config.json contains the quantization metadata needed to load the weights.

For direct generation with an MLX-LM installation that supports Spark:

mlx_lm.generate \
  --model xunlinkx/Spark-X2.6-4B-oQ4e-fp16 \
  --prompt "What is the capital of Anhui Province?" \
  --max-tokens 256 \
  --temp 1.0 \
  --top-p 0.95

The upstream model recommends temperature 1.0 and top-p 0.95; consult its card for chat-template and thinking-budget settings supported by your runtime.

Validation

A local short-prompt generation smoke check completed for this quantized checkpoint: 16 generated tokens; peak memory reported by MLX: 2.859 GB. This is a basic load-and-generate check, not a benchmark or a quality comparison. Memory requirements increase with context length and runtime overhead.

Attribution

Original model and research are by the SparkLLM team at XHToken. This repository contains a community quantization by xunlinkx; it is not an official XHToken release and has not been endorsed by the upstream authors.