wesjos/Qwen3.8-27B-CuteHuman-LoRA

Qwen3.8-27B · CuteHuman

A personality LoRA that only changes how the model talks — same knowledge, less AI-speak.

_{base Qwen/Qwen3.8-27B  ·  42 MB adapter  ·  10.49M trainable params (0.038%)  ·  Apache-2.0}

---

🖼 Showcase

_{Chinese · “you there?”}

_{English · “how's it going?”}

_{Terminal agent · restoring a deleted SVG, then writing a ramen song about it}

Same prompt, base vs CuteHuman

_{① 做一个波斯猫跳舞的 SVG 动画 — “an SVG animation of a Persian cat dancing”}

_{Qwen3.8-27B · generation}

_{+ CuteHuman LoRA · generation}

_{Qwen3.8-27B · rendered SVG}

_{+ CuteHuman LoRA · rendered SVG}

_{SVG sources: base.svg · cute.svg}

_{② make a svg animation of your favorite food}

_{Qwen3.8-27B · generation}

_{+ CuteHuman LoRA · generation}

_{Qwen3.8-27B · rendered SVG}

_{+ CuteHuman LoRA · rendered SVG}

_{SVG sources: base.svg · cute.svg}

---

📊 Before / after

Metric	Qwen3.8-27B	+ CuteHuman
Cuteness (1–5)	1.95	2.97
AI-speak occurrences	2 / 40	0 / 40
Median reply length	351 chars	215 chars
MMLU 5-shot (570 items)	77.54%	77.19%

_{No statistically significant regression — what changed is tone and length.}

---

🚀 Quickstart

1. transformers + PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "Qwen/Qwen3.8-27B"
ADAPTER = "wesjos/Qwen3.8-27B-CuteHuman-LoRA"

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)   # <- attach the adapter

msgs = [{"role": "user", "content": "今天上班好累啊"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=False)   # turn thinking off for chit-chat
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=256, do_sample=True,
                     temperature=0.7, top_p=0.8, top_k=20)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

2. Not enough VRAM: 4-bit loading

from transformers import BitsAndBytesConfig

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)   # fits on a single 24 GB card

3. Merge into the weights (optional)

merged = model.merge_and_unload()
merged.save_pretrained("./Qwen3.8-27B-CuteHuman", safe_serialization=True)

_{Merging a 27B into bf16 needs about 54 GB of RAM/disk.}

4. llama.cpp (use the GGUF build)

Companion repo: wesjos/Qwen3.8-27B-CuteHuman-LoRA-GGUF — mounted at runtime, never merged:

llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf \
  --lora cutehuman-27b-lora-f16.gguf \
  --mmproj mmproj-F16.gguf \
  -c 131072 -ngl 99 -fa on --jinja --port 18080

---

⚙️ Training setup

Item	Value
Base	Qwen/Qwen3.8-27B (trained on 4-bit nf4; the adapter attaches to any precision)
Method	QLoRA, r=16 / alpha=32 / dropout=0
Target modules	q_proj, k_proj, v_proj, o_proj of the 16 full-attention layers
Trainable params	10.49M (0.038% of 27.37B), 42 MB adapter (F32)
Data	655 samples (462 GSM8K chains of thought + 193 human-sounding chats), 1.6 epochs, lr 1e-4
Hardware	one RTX 4090 24GB — 131 steps in 12 minutes

---

💡 Notes

• No system prompt was used in training (the data is user/assistant only) — leaving it empty works best.
• LoRA does not train identity: ask “who are you” and it still answers as the base model (e.g. “Qwen”).
• Sampling: temperature 0.7 / top_p 0.8 / top_k 20; for thinking mode use 1.0 / 0.95.
• For structured output (tagging, JSON) pass chat_template_kwargs: {"enable_thinking": false}, otherwise the reasoning trace eats your token budget.
• The LoRA only touches 16 full-attention layers of the language stack — the vision tower and MLPs are untouched, so it is a pure text persona adapter and does not affect image understanding.

⚠️ Limitations

• 655 samples only teach a way of speaking; they add no knowledge. Domain knowledge still comes from the base model.
• Occasional “as an AI…” phrasing still slips through (0/40 in the eval, but sampling can leak).
• It still tends to guess on things it does not know; it will not reliably say “I don't know”.

---

📁 Files

File	What it is
adapter_model.safetensors	LoRA weights (42 MB)
adapter_config.json	PEFT config (r=16 / alpha=32 / target_modules)
tokenizer.json · tokenizer_config.json · chat_template.jinja	matching tokenizer and chat template

_{Demo images and SVG samples live in demos/. Chinese card: ModelScope}