Card: full 9-tier ladder at MTP sizes, RAM+VRAM run guidance, proofread
Browse files
README.md
CHANGED
|
@@ -31,7 +31,7 @@ tags:
|
|
| 31 |
|
| 32 |
# Straight to the point
|
| 33 |
|
| 34 |
-
*Tiel* is the fast coder of the arsenal. At 4-bit quantization and
|
| 35 |
the right GPU) of Opus 4.6 medium, while holding the best multi-turn conversation of any local model we
|
| 36 |
have measured. It is also cheerfully bad at trivia.
|
| 37 |
|
|
@@ -79,15 +79,28 @@ the dense 27B that solves 15 of the same 25 — one behind stock Qwen3.8-27B, at
|
|
| 79 |
|
| 80 |
## Run it
|
| 81 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
| file | size | fits | notes |
|
| 83 |
|---|--:|:--|---|
|
| 84 |
-
| `Tiel-Coder-35B-A3B-MTP-UD-
|
| 85 |
-
| `Tiel-Coder-35B-A3B-MTP-UD-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
MTP
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
than letting it ride the tier's own bit-width.
|
| 90 |
|
|
|
|
|
|
|
| 91 |
```bash
|
| 92 |
hf download peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP \
|
| 93 |
Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf mmproj-BF16.gguf --local-dir Tiel-MTP
|
|
@@ -106,7 +119,7 @@ less space.
|
|
| 106 |
Sampling: `temperature 1.0`, `top_p 0.95`, `top_k 20`. For agentic coding we ran `temperature 0.6`.
|
| 107 |
|
| 108 |
**It can see.** Vision works exactly as on the base repo — `mmproj-BF16.gguf`, Ornith's own projector
|
| 109 |
-
passed through unmodified, shared by
|
| 110 |
|
| 111 |
## The multi-token-prediction head
|
| 112 |
|
|
|
|
| 31 |
|
| 32 |
# Straight to the point
|
| 33 |
|
| 34 |
+
*Tiel* is the fast coder of the arsenal. At 4-bit quantization and 23 GB it fixes real codebase issues at the rate (and speed, with
|
| 35 |
the right GPU) of Opus 4.6 medium, while holding the best multi-turn conversation of any local model we
|
| 36 |
have measured. It is also cheerfully bad at trivia.
|
| 37 |
|
|
|
|
| 79 |
|
| 80 |
## Run it
|
| 81 |
|
| 82 |
+
The MoE architecture makes *Tiel* fast, even on smaller GPUs with partial GPU offloading, and makes the context KV small in RAM (<5 GB RAM for 262k context at 16-bit KV precision) compared to 27B dense. We do not recommend going below UD-Q4 simply to fit the whole model in GPU VRAM: when you can fit the model and context KV across your RAM+VRAM, pick a Q4 quant or larger that you can run with a sizable context (131k-262k) in at least q8_0 KV, for agentic coding.
|
| 83 |
+
|
| 84 |
+
The "fits" column below is therefore about your **combined available RAM+VRAM**, after OS and other processes take their share.
|
| 85 |
+
|
| 86 |
| file | size | fits | notes |
|
| 87 |
|---|--:|:--|---|
|
| 88 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q2_K_XL.gguf` | 13.2 GB | 16 GB | **THE LAST RESORT**; 2-bit gives up real ability, and struggles with agentic coding. Use anything larger, wherever it fits |
|
| 89 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-IQ3_XXS.gguf` | 14.1 GB | 16 GB | **the 16 GB pick** — significantly better than Q2_K_XL for under a gigabyte more |
|
| 90 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q3_K_XL.gguf` | 17.7 GB | 24 GB | 3-bit with plenty of context room; prefer IQ4_XS below unless you need the extra ~1 GB |
|
| 91 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-IQ4_XS.gguf` | 18.6 GB | 24 GB | 4-bit quality with the most context headroom of any 4-bit tier |
|
| 92 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q4_K_S.gguf` | 21.8 GB | 24 GB | tight 4-bit; useful when Q4_K_XL leaves too little room |
|
| 93 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf` | 23.3 GB | 24-32 GB | **start here** — the benchmarked tier plus the head; snug on 24 GB, comfortable on 32 |
|
| 94 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q5_K_XL.gguf` | 27.5 GB | 32 GB | the **32 GB** pick |
|
| 95 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q6_K_XL.gguf` | 32.7 GB | 48 GB | near-lossless; will not leave usable context on 32 GB |
|
| 96 |
+
| `Tiel-Coder-35B-A3B-MTP-UD-Q8_K_XL.gguf` | 39.4 GB | 48 GB | reference |
|
| 97 |
+
|
| 98 |
+
Each tier is its stripped counterpart in [Tiel-Coder-35B-A3B-GGUF](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF) plus **0.9 GB** of
|
| 99 |
+
MTP head — the same block at the same precision in every tier, because the recipe pins it rather
|
| 100 |
than letting it ride the tier's own bit-width.
|
| 101 |
|
| 102 |
+
To download and serve, replace the gguf names in the command below with your preferred quant:
|
| 103 |
+
|
| 104 |
```bash
|
| 105 |
hf download peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP \
|
| 106 |
Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf mmproj-BF16.gguf --local-dir Tiel-MTP
|
|
|
|
| 119 |
Sampling: `temperature 1.0`, `top_p 0.95`, `top_k 20`. For agentic coding we ran `temperature 0.6`.
|
| 120 |
|
| 121 |
**It can see.** Vision works exactly as on the base repo — `mmproj-BF16.gguf`, Ornith's own projector
|
| 122 |
+
passed through unmodified, shared by every tier.
|
| 123 |
|
| 124 |
## The multi-token-prediction head
|
| 125 |
|