TheDrainFlorist/Qwen3.8-27B-VQ-3.9bpw
Text Generation • 4B • Updated • 728
Qwen3.8-27B — Vector Quantized (MLX)
Data-free vector quantization of Qwen3.8-27B for Apple Silicon, stock mlx-lm. Three rungs, 3.9-4.8 bpw. Sizes include the vision tower.
Note 12.5 GiB, flat d4/K4096. KL 85.8 to bf16 — less than half the divergence of our 3-bit conversion, for 0.6 GiB more.
TheDrainFlorist/Qwen3.8-27B-VQ-4.5bpw
Text Generation • 19B • Updated • 700Note 14.5 GiB, flat d2/K256. KL 40.3 — closer to bf16 than the 4-bit conversion AND 0.5 GiB smaller.
TheDrainFlorist/Qwen3.8-27B-VQ-4.8bpw
Text Generation • 5B • Updated • 715Note 15.5 GiB, flat d2/K512. KL 32.8, 28% closer to bf16 than the 4-bit at +3.3% bytes. The one seeded, reproducible fit.