Download index.html from Genrius-AI/2bit-gguf-leaderboard: direct link, hf CLI and curl.
- Browser
- Download file 22.5 kB
-
https://huggingface.co/spaces/Genrius-AI/2bit-gguf-leaderboard/resolve/main/index.html
- Command line
-
hf download hf://spaces/Genrius-AI/2bit-gguf-leaderboard/index.html
-
curl -L -o index.html https://huggingface.co/spaces/Genrius-AI/2bit-gguf-leaderboard/resolve/main/index.html
22.5 kB
| <html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"> | |
| <title>2-bit GGUF Leaderboard</title> | |
| <meta name="description" content="Measured WikiText-2 perplexity and paired per-window statistics for published 2-bit GGUF files of Qwen3.8-27B and Qwen3.5-2B: DASH-Q, unsloth UD and Pingala-8. Raw windows in a CC BY 4.0 dataset."> | |
| <style> | |
| :root{--bg:#fbfbfa;--fg:#1c1d1f;--mut:#5b6068;--line:#dcdde0;--card:#ffffff;--acc:#1d4ed8;--top:#eef3ff; | |
| --dq:#1d4ed8;--ud:#047857;--pg:#c2410c;--st:#6b7280;color-scheme:light} | |
| @media (prefers-color-scheme:dark){:root:not([data-theme="light"]){--bg:#121316;--fg:#e8e9ec;--mut:#a3a8b1;--line:#30333a;--card:#1a1c20;--acc:#8ab4ff;--top:#1b2540; | |
| --dq:#7aa7ff;--ud:#34d399;--pg:#fb923c;--st:#a1a7b0;color-scheme:dark}} | |
| :root[data-theme="dark"]{--bg:#121316;--fg:#e8e9ec;--mut:#a3a8b1;--line:#30333a;--card:#1a1c20;--acc:#8ab4ff;--top:#1b2540; | |
| --dq:#7aa7ff;--ud:#34d399;--pg:#fb923c;--st:#a1a7b0;color-scheme:dark} | |
| *{box-sizing:border-box}html{-webkit-text-size-adjust:100%} | |
| body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,Helvetica,Arial,sans-serif} | |
| main{max-width:1180px;margin:0 auto;padding:20px 16px 40px} | |
| h1{font-size:1.55rem;line-height:1.25;margin:4px 0 8px}h2{font-size:1.2rem;margin:28px 0 8px}h3{font-size:1rem;margin:18px 0 6px} | |
| p{margin:6px 0}a{color:var(--acc)}.mut{color:var(--mut)} | |
| .lede{font-size:1.02rem;max-width:72ch} | |
| .wins{display:grid;grid-template-columns:repeat(auto-fit,minmax(260px,1fr));gap:10px;margin:12px 0} | |
| .win{background:var(--card);border:1px solid var(--line);border-left:4px solid var(--dq);border-radius:8px;padding:10px 12px} | |
| .win .k{font-size:.8rem;color:var(--mut);text-transform:uppercase;letter-spacing:.04em}.win .v{font-size:1.05rem;font-weight:650} | |
| .grid2{display:grid;grid-template-columns:1fr 1fr;gap:14px} | |
| @media (max-width:820px){.grid2{grid-template-columns:1fr}} | |
| .fig{background:var(--card);border:1px solid var(--line);border-radius:8px;padding:8px 8px 4px} | |
| .fig h3{margin:2px 4px 2px} | |
| svg.sc{width:100%;height:auto;display:block} | |
| svg .grid{stroke:var(--line);stroke-width:1}svg .axis{stroke:var(--mut);stroke-width:1} | |
| svg .tick{fill:var(--mut);font-size:11px}svg .lab{fill:var(--mut);font-size:12px}svg .pl{fill:var(--fg);font-size:12px} | |
| svg .pt{stroke:var(--card);stroke-width:1.5}svg .ring{fill:none;stroke-width:2} | |
| .pt.dq,.dot.dq{fill:var(--dq);background:var(--dq)}.pt.ud,.dot.ud{fill:var(--ud);background:var(--ud)} | |
| .pt.pg,.dot.pg{fill:var(--pg);background:var(--pg)}.pt.st,.dot.st{fill:var(--st);background:var(--st)} | |
| .ring.dq{stroke:var(--dq)}.ring.ud{stroke:var(--ud)}.ring.pg{stroke:var(--pg)}.ring.st{stroke:var(--st)} | |
| .dot{display:inline-block;width:10px;height:10px;border-radius:50%;margin-right:6px;vertical-align:baseline} | |
| .legend{display:flex;flex-wrap:wrap;gap:4px 14px;font-size:.85rem;color:var(--mut);margin:6px 2px} | |
| .tw{overflow-x:auto;-webkit-overflow-scrolling:touch;border:1px solid var(--line);border-radius:8px;background:var(--card);margin:8px 0} | |
| table{border-collapse:collapse;width:100%;font-size:.88rem} | |
| caption{caption-side:top;text-align:left;padding:8px 10px;color:var(--mut);font-size:.82rem} | |
| th,td{padding:7px 10px;border-top:1px solid var(--line);text-align:left;vertical-align:top} | |
| th{font-weight:600;white-space:nowrap;background:transparent}td.n,th.n{text-align:right;white-space:nowrap;font-variant-numeric:tabular-nums} | |
| tr.top td{background:var(--top)}.sub{color:var(--mut);font-size:.78rem;word-break:break-all}.sub2{color:var(--mut);font-size:.78rem;min-width:180px} | |
| table.wt td{min-width:220px} | |
| ul{padding-left:20px}li{margin:3px 0}code{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:.85em;background:var(--top);padding:1px 4px;border-radius:4px} | |
| footer{margin-top:30px;padding-top:12px;border-top:1px solid var(--line);color:var(--mut);font-size:.88rem} | |
| </style></head><body><main> | |
| <h1>2-bit GGUF leaderboard: Qwen3.8-27B and Qwen3.5-2B (measured, paired)</h1> | |
| <p class="lede">Published 2-bit GGUF files, scored by us with the same text, context and stride, and compared window by window. | |
| <b>Lowest perplexity: DASH-Q on both models.</b> Pingala-8 is our own file. On Qwen3.8-27B, Pingala-8 v2 is the smallest file within 10% of the best (8.31 GB, runs on a 12 GB GPU; ranks 2 of 5). On Qwen3.5-2B, v2 ranks 3 of 6 and beats both unsloth files.</p> | |
| <div class="wins"> | |
| <div class="win"><div class="k">#1 on Qwen3.8-27B</div><div class="v">DASH-Q IQ2_XS (jkim96): ppl 6.5431, 8.551 GB</div> | |
| <div class="mut">Our Pingala-8 v2 is 5.29% higher (paired dNLL +0.0516, wins 0/20 windows) at 8.314 GB.</div></div> | |
| <div class="win"><div class="k">#1 on Qwen3.5-2B</div><div class="v">DASH-Q IQ2_M (jkim96): ppl 13.3331, 0.786 GB</div> | |
| <div class="mut">DASH-Q IQ2_XS is also the smallest file here (0.757 GB) and within 4.27%. Our v2 is 5.43% higher than #1.</div></div> | |
| </div> | |
| <h2>Smallest file within X% of the best perplexity</h2> | |
| <div class="tw"><table class="wt"><thead><tr><th>Perplexity within</th><th>Qwen3.8-27B</th><th>Qwen3.5-2B</th></tr></thead><tbody><tr><td>1% of the best</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b> 8.551 GB, ppl 6.5431 (best)</td><td><span class="dot dq"></span><b>DASH-Q IQ2_M</b> 0.786 GB, ppl 13.3331 (best)</td></tr><tr><td>5% of the best</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b> 8.551 GB, ppl 6.5431 (best)</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b> 0.757 GB, ppl 13.9028 (+4.27%)</td></tr><tr><td>10% of the best</td><td><span class="dot pg"></span><b>Pingala-8 v2</b> 8.314 GB, ppl 6.8894 (+5.29%)</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b> 0.757 GB, ppl 13.9028 (+4.27%)</td></tr><tr><td>20% of the best</td><td><span class="dot pg"></span><b>Pingala-8 v2</b> 8.314 GB, ppl 6.8894 (+5.29%)</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b> 0.757 GB, ppl 13.9028 (+4.27%)</td></tr></tbody></table></div> | |
| <div class="grid2"> | |
| <div class="fig"><h3>Qwen3.8-27B, 20 windows</h3><svg class="sc" viewBox="0 0 560 360" role="img" aria-labelledby="t-m27"><title id="t-m27">Qwen3.8-27B: WikiText-2 perplexity vs file size, lower-left is better</title><line class="grid" x1="58" x2="544" y1="297.2" y2="297.2"/><text class="tick" x="52" y="301.2" text-anchor="end">6.5</text><line class="grid" x1="58" x2="544" y1="204.1" y2="204.1"/><text class="tick" x="52" y="208.1" text-anchor="end">7</text><line class="grid" x1="58" x2="544" y1="111.0" y2="111.0"/><text class="tick" x="52" y="115.0" text-anchor="end">7.5</text><line class="grid" x1="58" x2="544" y1="17.8" y2="17.8"/><text class="tick" x="52" y="21.8" text-anchor="end">8</text><line class="grid" y1="16" y2="314" x1="83.8" x2="83.8"/><text class="tick" y="330" x="83.8" text-anchor="middle">8.3</text><line class="grid" y1="16" y2="314" x1="235.6" x2="235.6"/><text class="tick" y="330" x="235.6" text-anchor="middle">8.4</text><line class="grid" y1="16" y2="314" x1="387.3" x2="387.3"/><text class="tick" y="330" x="387.3" text-anchor="middle">8.5</text><line class="grid" y1="16" y2="314" x1="539.0" x2="539.0"/><text class="tick" y="330" x="539.0" text-anchor="middle">8.6</text><line class="axis" x1="58" x2="544" y1="314" y2="314"/><line class="axis" x1="58" x2="58" y1="16" y2="314"/><text class="lab" x="301.0" y="352" text-anchor="middle">file size (GB, 1e9 bytes)</text><text class="lab" transform="translate(14 165.0) rotate(-90)" text-anchor="middle">perplexity (lower is better)</text><g><title>DASH-Q IQ2_XS: 6.5431 ppl, 8,551,168,768 bytes</title><circle class="pt dq" cx="464.9" cy="289.2" r="7"/><circle class="ring dq" cx="464.9" cy="289.2" r="11"/><text class="pl" x="453.9" y="293.2" text-anchor="end">#1 DASH-Q IQ2_XS 6.54</text></g><g><title>Pingala-8 v2: 6.8894 ppl, 8,314,241,600 bytes</title><circle class="pt pg" cx="105.4" cy="224.7" r="5.5"/><text class="pl" x="116.4" y="228.7" text-anchor="start">Pingala-8 v2 6.89</text></g><g><title>unsloth UD-IQ2_S: 7.3447 ppl, 8,371,970,048 bytes</title><circle class="pt ud" cx="193.0" cy="139.9" r="5.5"/><text class="pl" x="204.0" y="143.9" text-anchor="start">unsloth UD-IQ2_S 7.34</text></g><g><title>Pingala-8 v1: 7.3449 ppl, 8,314,489,600 bytes</title><circle class="pt pg" cx="105.8" cy="139.8" r="5.5"/><text class="pl" x="99" y="161" text-anchor="start">Pingala-8 v1 7.34</text></g><g><title>llama.cpp IQ2_XXS: 7.8765 ppl, 8,433,183,520 bytes</title><circle class="pt st" cx="285.9" cy="40.8" r="5.5"/><text class="pl" x="296.9" y="44.8" text-anchor="start">llama.cpp IQ2_XXS 7.88</text></g></svg></div> | |
| <div class="fig"><h3>Qwen3.5-2B, 72 windows</h3><svg class="sc" viewBox="0 0 560 360" role="img" aria-labelledby="t-m2"><title id="t-m2">Qwen3.5-2B: WikiText-2 perplexity vs file size, lower-left is better</title><line class="grid" x1="58" x2="544" y1="309.7" y2="309.7"/><text class="tick" x="52" y="313.7" text-anchor="end">12.5</text><line class="grid" x1="58" x2="544" y1="248.1" y2="248.1"/><text class="tick" x="52" y="252.1" text-anchor="end">15</text><line class="grid" x1="58" x2="544" y1="186.5" y2="186.5"/><text class="tick" x="52" y="190.5" text-anchor="end">17.5</text><line class="grid" x1="58" x2="544" y1="124.9" y2="124.9"/><text class="tick" x="52" y="128.9" text-anchor="end">20</text><line class="grid" x1="58" x2="544" y1="63.3" y2="63.3"/><text class="tick" x="52" y="67.3" text-anchor="end">22.5</text><line class="grid" y1="16" y2="314" x1="80.6" x2="80.6"/><text class="tick" y="330" x="80.6" text-anchor="middle">0.75</text><line class="grid" y1="16" y2="314" x1="255.5" x2="255.5"/><text class="tick" y="330" x="255.5" text-anchor="middle">0.80</text><line class="grid" y1="16" y2="314" x1="430.4" x2="430.4"/><text class="tick" y="330" x="430.4" text-anchor="middle">0.85</text><line class="axis" x1="58" x2="544" y1="314" y2="314"/><line class="axis" x1="58" x2="58" y1="16" y2="314"/><text class="lab" x="301.0" y="352" text-anchor="middle">file size (GB, 1e9 bytes)</text><text class="lab" transform="translate(14 165.0) rotate(-90)" text-anchor="middle">perplexity (lower is better)</text><g><title>DASH-Q IQ2_M: 13.3331 ppl, 786,259,840 bytes</title><circle class="pt dq" cx="207.5" cy="289.2" r="7"/><circle class="ring dq" cx="207.5" cy="289.2" r="11"/><text class="pl" x="218.5" y="293.2" text-anchor="start">#1 DASH-Q IQ2_M 13.33</text></g><g><title>DASH-Q IQ2_XS: 13.9028 ppl, 757,096,320 bytes</title><circle class="pt dq" cx="105.4" cy="275.1" r="5.5"/><text class="pl" x="99" y="309" text-anchor="start">DASH-Q IQ2_XS 13.90</text></g><g><title>Pingala-8 v2: 14.0573 ppl, 778,189,408 bytes</title><circle class="pt pg" cx="179.2" cy="271.3" r="5.5"/><text class="pl" x="190.2" y="275.3" text-anchor="start">Pingala-8 v2 14.06</text></g><g><title>unsloth UD-IQ2_M: 14.9555 ppl, 859,857,152 bytes</title><circle class="pt ud" cx="464.9" cy="249.2" r="5.5"/><text class="pl" x="453.9" y="253.2" text-anchor="end">unsloth UD-IQ2_M 14.96</text></g><g><title>Pingala-8 v1: 15.6193 ppl, 778,744,192 bytes</title><circle class="pt pg" cx="181.2" cy="232.8" r="5.5"/><text class="pl" x="192.2" y="236.8" text-anchor="start">Pingala-8 v1 15.62</text></g><g><title>unsloth UD-IQ2_XXS: 23.4124 ppl, 768,270,592 bytes</title><circle class="pt ud" cx="144.5" cy="40.8" r="5.5"/><text class="pl" x="155.5" y="44.8" text-anchor="start">unsloth UD-IQ2_XXS 23.41</text></g></svg></div> | |
| </div> | |
| <div class="legend"><span><span class="dot dq"></span>DASH-Q (jkim96)</span><span><span class="dot ud"></span>unsloth</span><span><span class="dot pg"></span>Pingala-8 (Genrius AI)</span><span><span class="dot st"></span>stock llama.cpp IQ2_XXS (local)</span><span>ringed = rank 1</span></div> | |
| <h2>Qwen3.8-27B</h2> | |
| <div class="tw"><table><caption>Qwen3.8-27B: WikiText-2 test, ctx 4096, 20 windows (first 20 of 72). Paired dNLL = mean per-window NLL of the file minus that of DASH-Q IQ2_XS (nats/token, positive = worse), 95% interval, windows where the file beats it.</caption><thead><tr><th>#</th><th>File</th><th>Repo</th><th class="n">Size GB</th><th class="n">Bytes</th><th class="n">PPL</th><th class="n">vs best</th><th class="n">Paired dNLL vs best</th><th class="n">Windows won</th><th>Runtime / placement</th></tr></thead><tbody><tr class="top"><td>1</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b><div class="sub">Qwen3.8-27B-DASHQ-IQ2_XS.gguf</div></td><td><a href="https://huggingface.co/jkim96/Qwen3.8-27B-DASHQ-Q2-GGUF" target="_blank" rel="noopener noreferrer">jkim96/Qwen3.8-27B-DASHQ-Q2-GGUF</a></td><td class="n">8.551</td><td class="n">8,551,168,768</td><td class="n"><b>6.5431</b></td><td class="n">best</td><td class="n">reference</td><td class="n">-</td><td class="sub2">llama.cpp upstream 889f2ec<br>GPU; IQ1_S/IQ2_S/IQ3_S tensors on CPU</td></tr><tr class=""><td>2</td><td><span class="dot pg"></span><b>Pingala-8 v2</b><div class="sub">Qwen3.8-27B-Pingala8-2bit-0000{1..9}-of-00009.gguf (9 parts)</div></td><td><a href="https://huggingface.co/Genrius-AI/Qwen3.8-27B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.8-27B-Pingala8-2bit</a></td><td class="n">8.314</td><td class="n">8,314,241,600</td><td class="n"><b>6.8894</b></td><td class="n">+5.29%</td><td class="n">+0.0516 ± 0.0117</td><td class="n">0/20</td><td class="sub2">Pingala-8 runtime (patched llama.cpp)<br>GPU</td></tr><tr class=""><td>3</td><td><span class="dot ud"></span><b>unsloth UD-IQ2_S</b><div class="sub">Qwen3.8-27B-UD-IQ2_S.gguf</div></td><td><a href="https://huggingface.co/unsloth/Qwen3.8-27B-GGUF" target="_blank" rel="noopener noreferrer">unsloth/Qwen3.8-27B-GGUF</a></td><td class="n">8.372</td><td class="n">8,371,970,048</td><td class="n"><b>7.3447</b></td><td class="n">+12.25%</td><td class="n">+0.1156 ± 0.0702</td><td class="n">2/20</td><td class="sub2">llama.cpp upstream 889f2ec<br>GPU; IQ1_S/IQ2_S/IQ3_S tensors on CPU</td></tr><tr class=""><td>4</td><td><span class="dot pg"></span><b>Pingala-8 v1</b><div class="sub">Qwen3.8-27B-Pingala8-2bit-0000{1..9}-of-00009.gguf (9 parts, earlier revision)</div></td><td><a href="https://huggingface.co/Genrius-AI/Qwen3.8-27B-Pingala8-2bit/tree/81c9a6d8fdb756eab0059768ecf34b7c23f4ea7b" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.8-27B-Pingala8-2bit</a></td><td class="n">8.314</td><td class="n">8,314,489,600</td><td class="n"><b>7.3449</b></td><td class="n">+12.25%</td><td class="n">+0.1156 ± 0.0132</td><td class="n">0/20</td><td class="sub2">Pingala-8 runtime (patched llama.cpp)<br>GPU</td></tr><tr class=""><td>5</td><td><span class="dot st"></span><b>llama.cpp IQ2_XXS</b><div class="sub">q38-27b-IQ2_XXS.gguf (not published)</div></td><td>not published</td><td class="n">8.433</td><td class="n">8,433,183,520</td><td class="n"><b>7.8765</b></td><td class="n">+20.38%</td><td class="n">+0.1855 ± 0.0303</td><td class="n">0/20</td><td class="sub2">llama.cpp upstream 889f2ec<br>GPU</td></tr></tbody></table></div> | |
| <h2>Qwen3.5-2B</h2> | |
| <div class="tw"><table><caption>Qwen3.5-2B: WikiText-2 test, ctx 4096, 72 windows (all). Paired dNLL = mean per-window NLL of the file minus that of DASH-Q IQ2_M (nats/token, positive = worse), 95% interval, windows where the file beats it.</caption><thead><tr><th>#</th><th>File</th><th>Repo</th><th class="n">Size GB</th><th class="n">Bytes</th><th class="n">PPL</th><th class="n">vs best</th><th class="n">Paired dNLL vs best</th><th class="n">Windows won</th><th>Runtime / placement</th></tr></thead><tbody><tr class="top"><td>1</td><td><span class="dot dq"></span><b>DASH-Q IQ2_M</b><div class="sub">Qwen3.5-2B-DASHQ-IQ2_M.gguf</div></td><td><a href="https://huggingface.co/jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF" target="_blank" rel="noopener noreferrer">jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF</a></td><td class="n">0.786</td><td class="n">786,259,840</td><td class="n"><b>13.3331</b></td><td class="n">best</td><td class="n">reference</td><td class="n">-</td><td class="sub2">llama.cpp upstream 889f2ec<br>CPU only (CUDA hidden), 12 threads</td></tr><tr class=""><td>2</td><td><span class="dot dq"></span><b>DASH-Q IQ2_XS</b><div class="sub">Qwen3.5-2B-DASHQ-IQ2_XS.gguf</div></td><td><a href="https://huggingface.co/jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF" target="_blank" rel="noopener noreferrer">jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF</a></td><td class="n">0.757</td><td class="n">757,096,320</td><td class="n"><b>13.9028</b></td><td class="n">+4.27%</td><td class="n">+0.0418 ± 0.0060</td><td class="n">5/72</td><td class="sub2">llama.cpp upstream 889f2ec<br>GPU; IQ1_S/IQ2_S/IQ3_S tensors on CPU</td></tr><tr class=""><td>3</td><td><span class="dot pg"></span><b>Pingala-8 v2</b><div class="sub">Qwen3.5-2B-Pingala8-2bit-v2.gguf</div></td><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.5-2B-Pingala8-2bit</a></td><td class="n">0.778</td><td class="n">778,189,408</td><td class="n"><b>14.0573</b></td><td class="n">+5.43%</td><td class="n">+0.0529 ± 0.0081</td><td class="n">3/72</td><td class="sub2">Pingala-8 runtime (patched llama.cpp)<br>GPU</td></tr><tr class=""><td>4</td><td><span class="dot ud"></span><b>unsloth UD-IQ2_M</b><div class="sub">Qwen3.5-2B-UD-IQ2_M.gguf</div></td><td><a href="https://huggingface.co/unsloth/Qwen3.5-2B-GGUF" target="_blank" rel="noopener noreferrer">unsloth/Qwen3.5-2B-GGUF</a></td><td class="n">0.860</td><td class="n">859,857,152</td><td class="n"><b>14.9555</b></td><td class="n">+12.17%</td><td class="n">+0.1148 ± 0.0087</td><td class="n">0/72</td><td class="sub2">llama.cpp upstream 889f2ec<br>CPU only (CUDA hidden), 12 threads</td></tr><tr class=""><td>5</td><td><span class="dot pg"></span><b>Pingala-8 v1</b><div class="sub">Qwen3.5-2B-Pingala8-2bit.gguf</div></td><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.5-2B-Pingala8-2bit</a></td><td class="n">0.779</td><td class="n">778,744,192</td><td class="n"><b>15.6193</b></td><td class="n">+17.15%</td><td class="n">+0.1583 ± 0.0107</td><td class="n">0/72</td><td class="sub2">Pingala-8 runtime (patched llama.cpp)<br>GPU</td></tr><tr class=""><td>6</td><td><span class="dot ud"></span><b>unsloth UD-IQ2_XXS</b><div class="sub">Qwen3.5-2B-UD-IQ2_XXS.gguf</div></td><td><a href="https://huggingface.co/unsloth/Qwen3.5-2B-GGUF" target="_blank" rel="noopener noreferrer">unsloth/Qwen3.5-2B-GGUF</a></td><td class="n">0.768</td><td class="n">768,270,592</td><td class="n"><b>23.4124</b></td><td class="n">+75.60%</td><td class="n">+0.5630 ± 0.0177</td><td class="n">0/72</td><td class="sub2">llama.cpp upstream 889f2ec<br>GPU; IQ1_S/IQ2_S/IQ3_S tensors on CPU</td></tr></tbody></table></div> | |
| <h2>Method</h2> | |
| <ul> | |
| <li>Text: WikiText-2 raw test split (Hugging Face rows joined), 297,193 tokens with the Qwen tokenizer. <code>llama-perplexity -c 2049 --ppl-stride 4095 -b 512 -ub 512 -ngl 99</code>: strided windows of 4096 tokens. Qwen3.8-27B: first 20 windows (time limit); Qwen3.5-2B: all 72.</li> | |
| <li>Rival files ran on llama.cpp upstream 889f2ec. On our RTX 5070 Ti Laptop GPU (sm_120, WSL2), the IQ1_S/IQ2_S/IQ3_S CUDA kernels failed test-backend-ops, so those tensors ran on the CPU (<code>-ot ...=CPU --no-op-offload</code>). The two Qwen3.5-2B IQ2_M runs ran fully on the CPU.</li> | |
| <li>A CPU/GPU cross-check agreed within 0.2%.</li> | |
| <li>Pingala files ran on the patched Pingala-8 runtime (a llama.cpp fork); those numbers are for that runtime.</li> | |
| <li>The paired statistic is the mean per-window difference in NLL against the best file, with a 95% interval (1.96 sd / sqrt(n)) and the count of windows won. Per-window NLL is recovered from the running perplexity: NLL_k = k ln P_k - (k-1) ln P_(k-1).</li> | |
| </ul> | |
| <h2>Caveats</h2> | |
| <ul> | |
| <li>Perplexity on one English text only; no downstream task scores here. Qwen3.8-27B uses 20 windows (about 82k tokens), so its intervals are wider.</li> | |
| <li>The log prints running perplexity to 4 decimals, so derived per-window NLL carries a small rounding error that grows with the window index (at most about 3e-4 nats at window 72).</li> | |
| <li>Rival files and our files ran on different builds and, for some tensors, different devices; see the placement column.</li> | |
| <li>The Qwen3.8-27B IQ2_XXS row is our own stock llama-quantize file with the imatrix used for Pingala-8; it is not published. Pingala-8 v1 for Qwen3.8-27B is an earlier revision of the same repo, replaced by v2.</li> | |
| <li>Genrius AI makes Pingala-8. We publish every window so anyone can check the ranking.</li> | |
| </ul> | |
| <h2>Data and links</h2> | |
| <ul> | |
| <li>Raw per-window measurements (CC BY 4.0): <a href="https://huggingface.co/datasets/Genrius-AI/2bit-gguf-wikitext2-eval" target="_blank" rel="noopener noreferrer">Genrius-AI/2bit-gguf-wikitext2-eval</a></li> | |
| <li>Method page: <a href="https://genrius-ai.github.io/pingala8-llm/paper.html" target="_blank" rel="noopener noreferrer">Pingala-8 paper (HTML)</a> and <a href="https://genrius-ai.github.io/pingala8-llm/" target="_blank" rel="noopener noreferrer">project site</a></li> | |
| <li>Paper DOI: <a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">10.5281/zenodo.23481757</a></li> | |
| <li>Model repos: <a href="https://huggingface.co/jkim96/Qwen3.8-27B-DASHQ-Q2-GGUF" target="_blank" rel="noopener noreferrer">jkim96/Qwen3.8-27B-DASHQ-Q2-GGUF</a>, <a href="https://huggingface.co/jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF" target="_blank" rel="noopener noreferrer">jkim96/Qwen3.5-2B-DASHQ-Q2-GGUF</a>, | |
| <a href="https://huggingface.co/unsloth/Qwen3.8-27B-GGUF" target="_blank" rel="noopener noreferrer">unsloth/Qwen3.8-27B-GGUF</a>, <a href="https://huggingface.co/unsloth/Qwen3.5-2B-GGUF" target="_blank" rel="noopener noreferrer">unsloth/Qwen3.5-2B-GGUF</a>, | |
| <a href="https://huggingface.co/Genrius-AI/Qwen3.8-27B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.8-27B-Pingala8-2bit</a>, <a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Genrius-AI/Qwen3.5-2B-Pingala8-2bit</a></li> | |
| <li>Collection: <a href="https://huggingface.co/collections/Genrius-AI/pingala-8-2-bit-llms-for-llamacpp-6adcae08d49bbdd8dd446ca9" target="_blank" rel="noopener noreferrer">Pingala-8: 2-bit LLMs for llama.cpp</a></li> | |
| </ul> | |
| <footer>Measured by Genrius AI. Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus. Paper: DOI 10.5281/zenodo.23481757.</footer> | |
| </main></body></html> | |