Add Apache-2.0 metadata and safety notice
Browse files
README.md
CHANGED
|
@@ -1,11 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# Huihui Qwen3.5-9B INT4 (RHT + Group32 RTN)
|
| 2 |
|
| 3 |
Experimental uzu/lalamo checkpoint. **RTN only**; YAQA/QAD not applied yet.
|
| 4 |
|
| 5 |
## Base model
|
| 6 |
-
- `huihui-ai/Huihui-Qwen3.5-9B-Claude-4.6-Opus-abliterated`
|
|
|
|
| 7 |
- untied embedding/lm_head, 32 layers, hidden 4096
|
| 8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
## Quantization
|
| 10 |
- Linear: RHT (Hadamard block 32, input_output) + asymmetric INT4 group 32, layout output_input
|
| 11 |
- Input embedding: INT4 (output-only RHT)
|
|
@@ -19,7 +37,7 @@ Experimental uzu/lalamo checkpoint. **RTN only**; YAQA/QAD not applied yet.
|
|
| 19 |
|
| 20 |
## uzu usage
|
| 21 |
```bash
|
| 22 |
-
hf download
|
| 23 |
uzu cli --model huihui-9b-int4-rht-rtn -m "Say hi in one word."
|
| 24 |
```
|
| 25 |
|
|
@@ -31,7 +49,7 @@ uzu cli --model huihui-9b-int4-rht-rtn -m "Say hi in one word."
|
|
| 31 |
- Experimental. Not instruction-tuned after quantization.
|
| 32 |
- No YAQA/QAD yet.
|
| 33 |
- JA output has factual errors from the base model, not necessarily quantization.
|
| 34 |
-
- License
|
| 35 |
|
| 36 |
## Provenance
|
| 37 |
See `provenance.json`.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: huihui-ai/Huihui-Qwen3.5-9B-Claude-4.6-Opus-abliterated
|
| 4 |
+
tags:
|
| 5 |
+
- quantization
|
| 6 |
+
- int4
|
| 7 |
+
- rht
|
| 8 |
+
- uzu
|
| 9 |
+
library_name: uzu
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
# Huihui Qwen3.5-9B INT4 (RHT + Group32 RTN)
|
| 13 |
|
| 14 |
Experimental uzu/lalamo checkpoint. **RTN only**; YAQA/QAD not applied yet.
|
| 15 |
|
| 16 |
## Base model
|
| 17 |
+
- `huihui-ai/Huihui-Qwen3.5-9B-Claude-4.6-Opus-abliterated` (Apache-2.0)
|
| 18 |
+
- upstream `Qwen/Qwen3.5-9B` (Apache-2.0)
|
| 19 |
- untied embedding/lm_head, 32 layers, hidden 4096
|
| 20 |
|
| 21 |
+
## Safety notice
|
| 22 |
+
The base model is "abliterated": its refusal behaviour has been removed. This
|
| 23 |
+
checkpoint inherits that behaviour and can produce unsafe content without
|
| 24 |
+
warning. Do not deploy it as a general-purpose assistant. Downstream use is the
|
| 25 |
+
user's responsibility and must comply with the upstream licenses.
|
| 26 |
+
|
| 27 |
## Quantization
|
| 28 |
- Linear: RHT (Hadamard block 32, input_output) + asymmetric INT4 group 32, layout output_input
|
| 29 |
- Input embedding: INT4 (output-only RHT)
|
|
|
|
| 37 |
|
| 38 |
## uzu usage
|
| 39 |
```bash
|
| 40 |
+
hf download yamahei21/huihui-qwen3.5-9b-int4-rht-rtn --local-dir huihui-9b-int4-rht-rtn
|
| 41 |
uzu cli --model huihui-9b-int4-rht-rtn -m "Say hi in one word."
|
| 42 |
```
|
| 43 |
|
|
|
|
| 49 |
- Experimental. Not instruction-tuned after quantization.
|
| 50 |
- No YAQA/QAD yet.
|
| 51 |
- JA output has factual errors from the base model, not necessarily quantization.
|
| 52 |
+
- License: Apache-2.0, inherited from the base model. See `LICENSE`.
|
| 53 |
|
| 54 |
## Provenance
|
| 55 |
See `provenance.json`.
|