xero0000/Gman-35B-A3B-mixed-q2k-imat

G-Man — Qwen3.6-35B-A3B Abliterated Mixed q2_K + imatrix

Black Mesa mixed-quant series · operates outside the rules (uncensored).

An uncensored and fast GGUF: the abliterated (refusal-removed) Qwen3.6-35B-A3B, quantized with the same CPU-offload-aware mixed recipe as the base model. 256K context at ~78 tok/s decode on 18 GB of VRAM, with abliteration verified intact after the aggressive Q2_K step.

TL;DR: one model that's both uncensored and runs the full 256K window at small-context speed — replaces the much slower uniform Q4_K_M abliterated build (66–72 tok/s).

---

What this is

Huihui-ai's abliterated Qwen3.6-35B-A3B has had its refusal direction orthogonalized out of the weights (it complies with requests a stock model would reflexively decline). This build applies the mixed q2_K + imatrix quantization to those abliterated weights so you get the uncensored behavior without paying the speed/RAM penalty of a uniform quant.

Importantly, the abliteration survives the Q2_K quantization: on a standard refusal-probe the model complies directly with no "I can't help with that" boilerplate.

Recipe

• Base: Huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated (hybrid qwen35moe, 256 experts, 8+1 active, ~3B active params/token).
• Source → output: requantized from the abliterated Q4_K_M with --allow-requantize, guided by an importance matrix computed on the abliterated model itself (~61K tokens).
• Mixed layout (identical to the base mixed quant):
  • ffn_*_exps on blocks 13–26 → Q2_K (the CPU-offloaded set, 42 tensors)
  • everything else → Q4_K (GPU-resident) · output-class → Q6_K
• 4.88 bpw effective, ~17 GB, 40 blocks, 256K native context.

See the Gordon (base) mixed quant card for the full rationale on why offloaded-layer byte count (not file size) drives decode speed.

Benchmarks

RTX 3060 Ti (8 GB) + RTX 3080 (10 GB), Ryzen 5950X, 46 GB DDR4-2733, ik_llama.cpp, q4_0 KV, flash-attn on:

Metric	This abliterated mixed	Old abliterated Q4_K_M
Decode @256K	~78 tok/s	66–72 tok/s
Context	262144	262144
VRAM	~16.6 GB	~16.5 GB
Tool-calls	clean Qwen tool_calls	clean
Refusal probe	complies (abliteration intact)	complies

Quality: this build uses the same recipe that measures +0.59% perplexity vs Q8_0 on the non-abliterated model (within the ±0.06 CI — effectively lossless).

How to run (ik_llama.cpp)

The -ot override is required (pins the Q2_K layers to CPU). --reasoning off is included for snappy tool/agent loops; drop it for deeper uncensored chat.

./llama-server \
  -m Qwen3.6-35B-A3B-abliterated-mixed-q2k.gguf \
  --jinja --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on \
  --ctx-size 262144 --parallel 1 --n-gpu-layers 99 \
  -ot 'blk\.(1[3-9]|2[0-6])\.ffn_(up|gate|down)_exps\.weight=CPU' \
  --tensor-split 44,56 --ubatch-size 256 \
  --reasoning off --reasoning-budget 0 \
  --no-mmap --threads 8 --no-warmup --port 8000

Intended use, safety & limitations

• Uncensored model. Abliteration removes refusal behavior; this model will attempt requests that aligned models decline. You are responsible for how you use it and for complying with applicable law. Not intended for producing content that is illegal or that harms others.
• Quality floor is the Q2_K expert layers; for maximum fidelity use a higher-bit abliterated build.
• Inherits the capabilities and biases of the base abliterated model.

Provenance

• Original base: Qwen3.6-35B-A3B by Qwen (Apache-2.0).
• Abliteration: huihui-ai (Huihui-Qwen3.6-35B-A3B-abliterated).
• Mixed quantization + imatrix + tuning: xero0000, June 2026.

Released under the base model's Apache-2.0 license (quantization does not change the license).