SolsticeAI/QuATernaryons-demo

🌌 QuATernaryons: Sub-2-Bit LLM Compression via 4D Isoclinic Clifford Rotors

QuATernaryons establishes the theoretical and computational foundation for sub-2-bit Large Language Model compression by lifting orthogonal channel mixing into the unit 3-sphere $\mathbb{S}^3$ of Hamilton quaternions.

By parameterizing orthogonal transformations through 4D left- and right-isoclinic Clifford rotors arranged in an $\mathcal{O}(d \log_4 d)$ butterfly network, QuATernaryons achieves uniform activation diffusion with strictly zero numerical drift and exact adjoint invertibility.

---

🏆 Verified Empirical Results: openbmb/MiniCPM3-4B

Evaluated across the 62 layers (496 linear projections) of openbmb/MiniCPM3-4B (4.1B parameters, DeepSeek-style MLA) on the 65,536-token zero-data teacher gate ($64 \times 1024$ sequences):

Pipeline Configuration	Group Size ($g$)	Target bpw	Top-1 Token Agreement	Student PPL	Teacher PPL	KL Divergence (nats/tok)	Vaulted Weights
Pristine Teacher (BF16)	—	16.00	100.00%	2.60	2.60	0.0000	—
Track 1 Arm A (Uniform)	32	2.119	76.45%	6.80	2.60	1.0473	arm_g32.pt (4,073 MB)
Track 1 Arm B (Uniform)	64	1.859	75.41%	8.46	2.60	1.2471	arm_g64.pt (3,965 MB)
Track 2 Dynamic Mixed-Rank	Mixed	~2.21	Distillation in Progress	Evaluating	2.60	Distillation in Progress	dynamic_weights.pt

All intermediate tensors, token distributions, and raw weight checkpoints are permanently preserved in the public offshore repository at Solstice-AI/QuATernaryons-demo-POC-alpha.

---

📦 The 6-GGUF Model Matrix Suite

Packed using native 5-trits-per-byte vector encoding ($3^5 = 243 \le 255$, 1.60 bpw discrete grid) to deliver standalone, runnable GGUFs ($\approx 0.95\text{–}1.15\text{ GB}$ each):

1. MiniCPM3-4B-QuATernaryon-Uniform-g32.gguf (~1.15 GB, 2.119 bpw) — Track 1 Arm A uniform baseline.
2. MiniCPM3-4B-QuATernaryon-Uniform-g64.gguf (~0.95 GB, 1.859 bpw) — Track 1 Arm B ultra-compact uniform baseline.
3. MiniCPM3-4B-QuATernaryon-DynamicNative-g32.gguf (~1.15 GB, 2.119 bpw) — Information-theoretic Hessian entropy allocation.
4. MiniCPM3-4B-QuATernaryon-DynamicNative-g64.gguf (~0.95 GB, 1.859 bpw) — Sub-2-bit information-theoretic Hessian allocation.
5. MiniCPM3-4B-QuATernaryon-DynamicHybrid-g32.gguf (~1.10 GB, 2.212 bpw) — Architectural sensitivity ladder + QAT polish ($r=16$ on o_proj, $r=14$ on down_proj).
6. MiniCPM3-4B-QuATernaryon-DynamicHybrid-g64.gguf (~0.95 GB, 1.950 bpw) — Deep sub-2-bit hybrid allocation + QAT polish.

---

📐 Why MiniCPM3-4B is Mathematically Superior for QuATernaryons

Structural Property	SmolLM2-1.7B	MiniCPM3-4B (openbmb/MiniCPM3-4B)	Mathematical Significance
Attention Architecture	Standard MHA	DeepSeek-style MLA	Multi-head Latent Attention with compressed KV latents.
Head Dimension ($d_h$)	$64$	$64 = 4^3$	Exactly 16 4D quaternion blocks; exactly $\log_4(64) = 3$ butterfly stages.
Hidden Size ($d$)	$2048 = 2^{11}$	$2304 = 36 \times 64$	Pure multiple of $4^3$; zero padding across all 36 attention heads.
Intermediate Size ($d_{\text{mlp}}$)	$8192 = 2^{13}$	$5760 = 90 \times 64$	Pure multiple of $4^3$; zero padding across all 90 MLP blocks.
SIMD Lane Efficiency	Non-uniform	100% Lane Utilization	Zero wasted parameters, zero dead weights, perfect SIMD-4 execution.
Activation Geometry	Uncalibrated Outliers	WSD + RMSNorm Scaled	Exceptionally well-conditioned activations prevent quantization collapse.

---

🎯 The Zero-Data Evaluation Gate (The One Metric That Matters)

External evaluation benchmarks (Wikitext PPL, MMLU) can be gamed through training set leakage or corpus selection. QuATernaryons adopts the Zero-Data Distribution-Fidelity Protocol:

1. Synthetic Teacher Generator: Exactly 65,536 tokens generated autoregressively by the pristine FP16/BF16 teacher model from uniform random seed blocks using top-$k(40)$ sampling ($T=1.0$, seed 1234, 64 sequences $\times$ 1024 sequence length).
2. Distribution Divergence: Token-wise evaluation of Kullback-Leibler divergence $\mathrm{KL}(P_{\text{teacher}} \parallel P_{\text{student}})$ in nats/token: $$\mathrm{KL} = \frac{1}{N} \sum_{i=1}^N \sum_{v \in V} P_{\text{teacher}}(v \mid x_{<i}) \ln \left( \frac{P_{\text{teacher}}(v \mid x_{<i})}{P_{\text{student}}(v \mid x_{<i})} \right)$$ (Note: Evaluated with per-token normalization to prevent PyTorch's batchmean inflation bug).
3. Top-1 Token Agreement: The fraction of sequence positions where the student's top greedy prediction strictly matches the teacher ($\operatorname{argmax}(P_s) == \operatorname{argmax}(P_t)$).
4. Autoregressive Coherence Guard: Verification across reasoning, coding, and mathematical prompts to ensure no MC-generation dissociation or rotor phase drift occurs during long-horizon generation.

---

⚡ Quantization Structure & Runtime Dequantization

Each weight projection $W \in \mathbb{R}^{\text{out} \times \text{in}}$ is transformed into the Clifford rotor basis: $$W_{\text{rot}}=W{R}^T\text{where}R=\prod _{s=1}^{{\log }_{4}d}{\mathrm{\Pi }}_s{\mathbf{B}}_s\in \mathrm{S}\mathrm{O}(d)W\_\{\backslash text\{rot\}\}\; =\; W\; R^T\; \backslash quad\; \backslash text\{where\}\; \backslash quad\; R\; =\; \backslash prod\_\{s=1\}^\{\backslash log\_4\; d\}\; \backslash Pi\_s\; \backslash mathbf\{B\}\_s\; \backslash in\; \backslash mathrm\{SO\}(d)$$

The weight is decomposed into:

1. Per-Row Asymmetric Offset: $\beta = \operatorname{mean}(W_{\text{rot}}, \text{axis}=1) \in \mathbb{R}^{\text{out} \times 1}$ (proven by Helcig et al.'s $\gamma^2$ variance law to be necessary for distribution-level fidelity).
2. Asymmetric Ternary Core: $T \in {-1, 0, +1}^{\text{out} \times \text{in}}$ packed 5-trits-per-byte ($1.60\text{ bpw}$).
3. Per-Group Scale: $\alpha \in \mathbb{R}^{\text{out} \times (\text{in}/g)}$.
4. Low-Rank SVD Correction: $U \in \mathbb{R}^{\text{out} \times r}, V \in \mathbb{R}^{\text{in} \times r}$ ($r \in {4, 8, 14, 16}$ depending on sensitivity).

$$\text{Forward Pass:}y=(x{R}^T){(\beta +\mathrm{repeat}(\alpha ,g)\odot T+U{V}^T)}^T+\text{bias}\backslash text\{Forward\; Pass:\}\; \backslash quad\; y\; =\; (x\; R^T)\; \backslash left(\; \backslash beta\; +\; \backslash operatorname\{repeat\}(\backslash alpha,\; g)\; \backslash odot\; T\; +\; U\; V^T\; \backslash right)^T\; +\; \backslash text\{bias\}$$

At deployment, ternary codes $T$ and rotors $R$ are frozen. Continuous parameters $(\alpha, \beta, U, V)$ are refined during Zero-Data QAT distillation without straight-through estimators (STE).

---

🚀 Installation & Mathematical Verification

Local Package Installation & Math Suite

git clone https://github.com/Solstice-Labs/QuATernaryons.git
cd QuATernaryons
pip install -e .

# Run 7/7 machine-precision mathematical tests
python3 -m unittest tests/test_quaternaryons_math.py

# Run 22/22 paper proof verifications
python3 verify_isoquant_math.py

Export Standalone Packed GGUF Suite

python3 quaternaryons/export_gguf.py \
    --weights-path benchmarks/arm_g32_quaternaryons_weights.pt \
    --output-path MiniCPM3-4B-QuATernaryon-Uniform-g32.gguf

---

📚 Citation

@article{gondaliya2026quaternaryons,
  title={QuATernaryons: Dual-Sided Sub-2-Bit Weight and Dynamic KV-Cache Quantization via 4D Isoclinic Clifford Rotors},
  author={Gondaliya, Shreyan D.},
  journal={CERN Zenodo},
  doi={10.5281/zenodo.23530592},
  year={2026}
}