bong-water-water-bong/Bonsai-1.7B-TQ2-1BP

Bonsai-1.7B — 1BP format, TQ2 (2-bit ternary)

prism-ml/Ternary-Bonsai-1.7B (Apache-2.0), a genuinely ternary-trained model (Qwen3 architecture), converted to 1BP using the TQ2 quantization type — not the default Q4NX.

Why this one's different

Every other model in this project's catalog goes through 1BP's Q4NX path: 4-bit asymmetric quantization, same scheme regardless of source precision. That's the right choice for normal float-precision models, but it wastes space on a model whose weights are already ternary — every value in a 32-element group is exactly -scale, 0, or +scale.

TQ2 stores that natively: 2 bits per weight (not 4), one BF16 scale per 32-group, no zero-point needed since ternary is symmetric around zero. Verified against the source GGUF: 100% of dequantized values match within BF16 scale-rounding precision (mean relative error rounds to 0.000000) — this is a lossless repack, not a lossy requantization. Q4NX would've stored the same ternary values in twice the space for no accuracy gain.

538 MB, vs 3.45 GB for the source F16 GGUF.

Format

1BP's tensor index and 256-byte header are unchanged; only the tile payload differs. Per 32×256 tile: 8×32 BF16 scales (one per 32-element group) + 2048 2-bit codes packed 4/byte. header.quant is set to ONEBP_TQ2 (3) so loaders know which dequant path to use — OnebpModel::get_tensor_f32() in engine/npu/src/onebp_loader.cpp dispatches on it automatically.

ONEBP_TQ1 (1.58-bit, base-3 packing) is defined in the format spec but not yet implemented — a real further win (256 isn't evenly divisible by the 5-values-per-byte base-3 scheme, so it needs more careful boundary handling than TQ2 did) left for later rather than shipped unverified.

Usage

git clone https://github.com/bong-water-water-bong/1bit-systems
cd 1bit-systems
cmake -B build/engine/npu -S engine/npu -DCMAKE_BUILD_TYPE=Release   # ONEBP_SUPPORT is on by default
cmake --build build/engine/npu --target npu_engine_universal -j8
./build/engine/npu/npu_engine_universal /path/to/Bonsai-1.7B-TQ2.1bp --model-tag bonsai_1_7b

To convert another ternary-native model the same way:

python3 tools/gguf_to_onebp.py model.gguf output.1bp --tq2

Attribution

Base model: prism-ml/Ternary-Bonsai-1.7B, Apache-2.0. This repo contains a lossless repack (not a requantization) into 1BP's TQ2 format; the original license terms apply to the underlying weights.