peculiar-ragdoll commited on
Commit
199cff2
·
verified ·
1 Parent(s): 3afc522

Card: full 9-tier ladder at MTP sizes, RAM+VRAM run guidance, proofread

Browse files
Files changed (1) hide show
  1. README.md +20 -7
README.md CHANGED
@@ -31,7 +31,7 @@ tags:
31
 
32
  # Straight to the point
33
 
34
- *Tiel* is the fast coder of the arsenal. At 4-bit quantization and 22 GB it fixes real codebase issues at the rate (and speed, with
35
  the right GPU) of Opus 4.6 medium, while holding the best multi-turn conversation of any local model we
36
  have measured. It is also cheerfully bad at trivia.
37
 
@@ -79,15 +79,28 @@ the dense 27B that solves 15 of the same 25 — one behind stock Qwen3.8-27B, at
79
 
80
  ## Run it
81
 
 
 
 
 
82
  | file | size | fits | notes |
83
  |---|--:|:--|---|
84
- | `Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf` | 23.3 GB | 32 GB | **start here** — the benchmarked tier plus the head |
85
- | `Tiel-Coder-35B-A3B-MTP-UD-Q5_K_XL.gguf` | 27.5 GB | 32 GB | more precision headroom |
86
-
87
- Each is its stripped counterpart in [Tiel-Coder-35B-A3B-GGUF](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF) plus **0.9 GB** of
88
- MTP head — the same block at the same precision in both tiers, because the recipe pins it rather
 
 
 
 
 
 
 
89
  than letting it ride the tier's own bit-width.
90
 
 
 
91
  ```bash
92
  hf download peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP \
93
  Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf mmproj-BF16.gguf --local-dir Tiel-MTP
@@ -106,7 +119,7 @@ less space.
106
  Sampling: `temperature 1.0`, `top_p 0.95`, `top_k 20`. For agentic coding we ran `temperature 0.6`.
107
 
108
  **It can see.** Vision works exactly as on the base repo — `mmproj-BF16.gguf`, Ornith's own projector
109
- passed through unmodified, shared by both tiers.
110
 
111
  ## The multi-token-prediction head
112
 
 
31
 
32
  # Straight to the point
33
 
34
+ *Tiel* is the fast coder of the arsenal. At 4-bit quantization and 23 GB it fixes real codebase issues at the rate (and speed, with
35
  the right GPU) of Opus 4.6 medium, while holding the best multi-turn conversation of any local model we
36
  have measured. It is also cheerfully bad at trivia.
37
 
 
79
 
80
  ## Run it
81
 
82
+ The MoE architecture makes *Tiel* fast, even on smaller GPUs with partial GPU offloading, and makes the context KV small in RAM (<5 GB RAM for 262k context at 16-bit KV precision) compared to 27B dense. We do not recommend going below UD-Q4 simply to fit the whole model in GPU VRAM: when you can fit the model and context KV across your RAM+VRAM, pick a Q4 quant or larger that you can run with a sizable context (131k-262k) in at least q8_0 KV, for agentic coding.
83
+
84
+ The "fits" column below is therefore about your **combined available RAM+VRAM**, after OS and other processes take their share.
85
+
86
  | file | size | fits | notes |
87
  |---|--:|:--|---|
88
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q2_K_XL.gguf` | 13.2 GB | 16 GB | **THE LAST RESORT**; 2-bit gives up real ability, and struggles with agentic coding. Use anything larger, wherever it fits |
89
+ | `Tiel-Coder-35B-A3B-MTP-UD-IQ3_XXS.gguf` | 14.1 GB | 16 GB | **the 16 GB pick** — significantly better than Q2_K_XL for under a gigabyte more |
90
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q3_K_XL.gguf` | 17.7 GB | 24 GB | 3-bit with plenty of context room; prefer IQ4_XS below unless you need the extra ~1 GB |
91
+ | `Tiel-Coder-35B-A3B-MTP-UD-IQ4_XS.gguf` | 18.6 GB | 24 GB | 4-bit quality with the most context headroom of any 4-bit tier |
92
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q4_K_S.gguf` | 21.8 GB | 24 GB | tight 4-bit; useful when Q4_K_XL leaves too little room |
93
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf` | 23.3 GB | 24-32 GB | **start here** — the benchmarked tier plus the head; snug on 24 GB, comfortable on 32 |
94
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q5_K_XL.gguf` | 27.5 GB | 32 GB | the **32 GB** pick |
95
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q6_K_XL.gguf` | 32.7 GB | 48 GB | near-lossless; will not leave usable context on 32 GB |
96
+ | `Tiel-Coder-35B-A3B-MTP-UD-Q8_K_XL.gguf` | 39.4 GB | 48 GB | reference |
97
+
98
+ Each tier is its stripped counterpart in [Tiel-Coder-35B-A3B-GGUF](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF) plus **0.9 GB** of
99
+ MTP head — the same block at the same precision in every tier, because the recipe pins it rather
100
  than letting it ride the tier's own bit-width.
101
 
102
+ To download and serve, replace the gguf names in the command below with your preferred quant:
103
+
104
  ```bash
105
  hf download peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP \
106
  Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf mmproj-BF16.gguf --local-dir Tiel-MTP
 
119
  Sampling: `temperature 1.0`, `top_p 0.95`, `top_k 20`. For agentic coding we ran `temperature 0.6`.
120
 
121
  **It can see.** Vision works exactly as on the base repo — `mmproj-BF16.gguf`, Ornith's own projector
122
+ passed through unmodified, shared by every tier.
123
 
124
  ## The multi-token-prediction head
125