TheDrainFlorist/Qwen3.5-397B-A17B-VQ-2.2bpw
Text Generation • 31B • Updated • 1.95k
Qwen3.5-397B-A17B — Vector Quantized (MLX)
Qwen3.5-397B for Apple Silicon, from one 128 GB Mac up. Vector-quantized with learned codebooks, bit-packed sub-byte. Stock mlx-lm, no patches.
Note 101.0 GiB, flat d8/K16384. The accessibility rung: runs on a single 128 GB Mac with ~17 GiB free at 8k context.
TheDrainFlorist/Qwen3.5-397B-A17B-VQ-2.4bpw
Text Generation • 102B • Updated • 2.01kNote 111.6 GiB, flat d4/K256. The daily driver: fits one 128 GB Mac, verified to 30k context.
TheDrainFlorist/Qwen3.5-397B-A17B-VQ-2.6bpw
Text Generation • 36B • Updated • 586Note 122.3 GiB, flat d4/K512. Pound for pound: the same size class as the strongest community quant at this rate, ahead by 19.5% on prose.
TheDrainFlorist/Qwen3.5-397B-A17B-VQ-3.1bpw
Text Generation • 42B • Updated • 1.5kNote 143.7 GiB, flat d4/K2048. The quality rung: 21.9 GiB smaller than the community 3.5-bit, better on prose, tied on code.