Genrius-AI/Qwen3.8-27B-Pingala8-2bit

Text Generation • 10B • Updated 8 days ago • 1.17k

Note Ours, Pingala-8 v2: 8.31 GB, WikiText-2 PPL 6.89 (20 windows, ctx 4096). Runs fully on one 12 GB RTX 5070 Ti Laptop GPU at 37.7 tok/s (-ngl 99); calculator: 10.24 GB at ctx 4096. Needs our patched CUDA runtime; not Ollama or LM Studio.

jkim96/Qwen3.8-27B-DASHQ-Q2-GGUF

Text Generation • 27B • Updated 2 days ago • 419

Note DASH-Q IQ2_XS: 8.55 GB, PPL 6.54, the lowest we measured; better than ours on 20/20 windows. Fits 12 GB at 4k-8k context by our calculator. Stock llama.cpp; on our RTX 50 laptop its IQ1_S/IQ2_S/IQ3_S tensors ran on the CPU (CUDA NaN).

unsloth/Qwen3.8-27B-GGUF

27B • Updated Aug 20 • 6.07M • 5.62k

Note unsloth UD-IQ2_S: 8.37 GB, PPL 7.34; ours is lower but not significantly (ours better on 9/20 windows). Fits 12 GB at 4k-8k context by our calculator. Their UD-Q4_K_M (16.46 GB) does not fit 12 GB.

Note Our leaderboard: every 2-bit file above, ranked by WikiText-2 perplexity with size, windows won and runtime, DASH-Q first. Also the Qwen3.5-2B files.

Note Our eval data: summary.csv (bytes, PPL, CI, runtime, CPU/GPU placement) and windows.csv (per-window loss) behind every number in this collection. CC BY 4.0.