yamahei21/huihui-qwen3.5-9b-int4-rht-rtn

Huihui Qwen3.5-9B INT4 (RHT + Group32 RTN)

Experimental uzu/lalamo checkpoint. RTN only; YAQA/QAD not applied yet.

Base model

• huihui-ai/Huihui-Qwen3.5-9B-Claude-4.6-Opus-abliterated (Apache-2.0)
• upstream Qwen/Qwen3.5-9B (Apache-2.0)
• untied embedding/lm_head, 32 layers, hidden 4096

Safety notice

The base model is "abliterated": its refusal behaviour has been removed. This checkpoint inherits that behaviour and can produce unsafe content without warning. Do not deploy it as a general-purpose assistant. Downstream use is the user's responsibility and must comply with the upstream licenses.

Quantization

• Linear: RHT (Hadamard block 32, input_output) + asymmetric INT4 group 32, layout output_input
• Input embedding: INT4 (output-only RHT)
• lm_head/readout: INT4 (input-only RHT)
• Norm: BF16/F32 as stored
• lalamo commit: e401e17761f2fcd337b0cf20218d67d656e6ecba
• uzu commit: fa0aa23d

Size

• model.safetensors: ~5.19 GB

uzu usage

hf download yamahei21/huihui-qwen3.5-9b-int4-rht-rtn --local-dir huihui-9b-int4-rht-rtn
uzu cli --model huihui-9b-int4-rht-rtn -m "Say hi in one word."

Measured on M3 16GB

• EN: TTFT 0.26s / prefill 68.8 t/s / gen 6.28 t/s / 4.95GB
• JA: TTFT 0.27s / prefill 63.0 t/s / gen 11.47 t/s / 4.96GB

Status / limitations

• Experimental. Not instruction-tuned after quantization.
• No YAQA/QAD yet.
• JA output has factual errors from the base model, not necessarily quantization.
• License: Apache-2.0, inherited from the base model. See LICENSE.

Provenance

See provenance.json.