Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -6,7 +6,6 @@ tags:
|
|
| 6 |
- gguf
|
| 7 |
- llama.cpp
|
| 8 |
- quantized
|
| 9 |
-
- imatrix
|
| 10 |
---
|
| 11 |
|
| 12 |
# manaca-1b-base GGUF
|
|
@@ -36,10 +35,10 @@ GGUF quantizations of [menezesbruno/manaca-1b-base](https://huggingface.co/menez
|
|
| 36 |
| [manaca-1b-base-IQ3_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_S.gguf) | `IQ3_S` | 0.74 GB | Slightly smaller than IQ3_M. |
|
| 37 |
| [manaca-1b-base-Q3_K_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q3_K_S.gguf) | `Q3_K_S` | 0.74 GB | Aggressive. Prefer IQ3_M at a similar size. |
|
| 38 |
| [manaca-1b-base-IQ3_XS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_XS.gguf) | `IQ3_XS` | 0.71 GB | Aggressive but coherent. |
|
| 39 |
-
| [manaca-1b-base-IQ3_XXS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_XXS.gguf) | `IQ3_XXS` | 0.66 GB | Very aggressive;
|
| 40 |
| [manaca-1b-base-Q2_K.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q2_K.gguf) | `Q2_K` | 0.64 GB | Very small, heavily degraded. For experimentation. |
|
| 41 |
| [manaca-1b-base-IQ2_M.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_M.gguf) | `IQ2_M` | 0.60 GB | The smallest size most people find usable. |
|
| 42 |
-
| [manaca-1b-base-Q2_K_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q2_K_S.gguf) | `Q2_K_S` | 0.61 GB | Smaller than Q2_K,
|
| 43 |
| [manaca-1b-base-IQ2_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_S.gguf) | `IQ2_S` | 0.56 GB | Below the usual usability line. |
|
| 44 |
| [manaca-1b-base-IQ2_XS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_XS.gguf) | `IQ2_XS` | 0.53 GB | Experimental. |
|
| 45 |
| [manaca-1b-base-IQ2_XXS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_XXS.gguf) | `IQ2_XXS` | 0.49 GB | Experimental. |
|
|
@@ -55,14 +54,7 @@ Pick the largest file that leaves a couple of gigabytes of headroom on the devic
|
|
| 55 |
- Plenty of memory: **Q6_K** or **Q8_0**.
|
| 56 |
- The usual choice: **Q4_K_M**.
|
| 57 |
- Tight on memory: **IQ4_XS**, then **IQ3_M**, then **IQ2_M**.
|
| 58 |
-
- The `IQ*` files
|
| 59 |
-
|
| 60 |
-
## Quantization details
|
| 61 |
-
|
| 62 |
-
- Importance matrix computed with `llama-imatrix` over 500 rows of [Salesforce/wikitext](https://huggingface.co/datasets/Salesforce/wikitext) (`wikitext-2-raw-v1`).
|
| 63 |
-
- The matrix was computed on the **BF16** weights.
|
| 64 |
-
- K-quants below 6 bit and the whole `IQ` set are imatrix-guided. `Q4_0`/`Q4_1`/`Q5_0`/`Q5_1` are legacy round-to-nearest and ignore it; `Q6_K`/`Q8_0` are near-lossless and do not need it.
|
| 65 |
-
- All files are cut from the same BF16 GGUF, so differences between them are quantization only.
|
| 66 |
|
| 67 |
## Usage
|
| 68 |
|
|
@@ -79,4 +71,4 @@ llama-cli -m manaca-1b-base-Q4_K_M.gguf -p "Hello"
|
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
-
Quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp) by AgentQuantix on 2026-09-
|
|
|
|
| 6 |
- gguf
|
| 7 |
- llama.cpp
|
| 8 |
- quantized
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
# manaca-1b-base GGUF
|
|
|
|
| 35 |
| [manaca-1b-base-IQ3_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_S.gguf) | `IQ3_S` | 0.74 GB | Slightly smaller than IQ3_M. |
|
| 36 |
| [manaca-1b-base-Q3_K_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q3_K_S.gguf) | `Q3_K_S` | 0.74 GB | Aggressive. Prefer IQ3_M at a similar size. |
|
| 37 |
| [manaca-1b-base-IQ3_XS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_XS.gguf) | `IQ3_XS` | 0.71 GB | Aggressive but coherent. |
|
| 38 |
+
| [manaca-1b-base-IQ3_XXS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ3_XXS.gguf) | `IQ3_XXS` | 0.66 GB | Very aggressive; the last coherent step down. |
|
| 39 |
| [manaca-1b-base-Q2_K.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q2_K.gguf) | `Q2_K` | 0.64 GB | Very small, heavily degraded. For experimentation. |
|
| 40 |
| [manaca-1b-base-IQ2_M.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_M.gguf) | `IQ2_M` | 0.60 GB | The smallest size most people find usable. |
|
| 41 |
+
| [manaca-1b-base-Q2_K_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-Q2_K_S.gguf) | `Q2_K_S` | 0.61 GB | Smaller than Q2_K, at a further quality cost. |
|
| 42 |
| [manaca-1b-base-IQ2_S.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_S.gguf) | `IQ2_S` | 0.56 GB | Below the usual usability line. |
|
| 43 |
| [manaca-1b-base-IQ2_XS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_XS.gguf) | `IQ2_XS` | 0.53 GB | Experimental. |
|
| 44 |
| [manaca-1b-base-IQ2_XXS.gguf](https://huggingface.co/NANI-Nithin/manaca-1b-base-GGUF/blob/main/manaca-1b-base-IQ2_XXS.gguf) | `IQ2_XXS` | 0.49 GB | Experimental. |
|
|
|
|
| 54 |
- Plenty of memory: **Q6_K** or **Q8_0**.
|
| 55 |
- The usual choice: **Q4_K_M**.
|
| 56 |
- Tight on memory: **IQ4_XS**, then **IQ3_M**, then **IQ2_M**.
|
| 57 |
+
- The `IQ*` files generally beat a `Q*` file of similar size, at the cost of slightly slower inference on some hardware.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
## Usage
|
| 60 |
|
|
|
|
| 71 |
|
| 72 |
---
|
| 73 |
|
| 74 |
+
Quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp) by AgentQuantix on 2026-09-07. Licensing follows the base model.
|