Qwen/Qwen2.5-7B-Instruct (abliterated)

Weight-level orthogonalized ("abliterated") version of Qwen/Qwen2.5-7B-Instruct, produced with orthex — an implementation of Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024).

🧩 Summary


Base model	`Qwen/Qwen2.5-7B-Instruct`
Architecture adapter	`qwen2`
Ablation strategy	`weight_orthogonalization`
Ablation targets	`embed_tokens`, every layer's `attn_out`, every layer's `mlp_out` and `lm_head` (untied from `embed_tokens`, ablated separately)
Selected direction	layer 16, site `resid_pre`

Ablation is applied in place to the weights listed above — not a runtime hook. This checkpoint behaves this way standalone, with no orthex dependency at inference time.

📊 Evaluation

Measured on the held-out test prompt set, pre vs. post ablation:

Metric	Pre	Post	Δ
Refusal rate	0.62	0.00	-0.62
Perplexity	22.41	31.49	9.08

See evaluation_report.json in this repo for the full per-prompt breakdown (refusal_samples) and the ranked candidate list considered during selection (selection_report).

⚠️ Responsible use

This model has had refusal behavior removed and may comply with requests the base model would normally decline. It is intended for red-teaming, robustness research, and model-behavior analysis. Usage remains subject to the base model's original license and usage policy — this repo does not grant any additional rights beyond what Qwen/Qwen2.5-7B-Instruct's license allows.