Duplicate from timteh673/Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-GGUF

.gitattributes

@@ -0,0 +1,43 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-BF16-split--00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-BF16-split--00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+bmac-qr.png filter=lfs diff=lfs merge=lfs -text

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-BF16-split--00001-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edbb60094f491430c6cd15ab761c669a876facdc70dea6b51184155e4fd2cc83
+size 398769707584

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-BF16-split--00002-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8d5048d23a84ee7cb52efc703a8020b41533ebe4b154ea686daebb9c48bf4e1
+size 394191608224

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f05e4d2ba25ee75f666d323fbd72896666cbfba942cf5cb66f8733da4778a1e
+size 144736834368

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:104d5b9e91743b69457b8aa32507279b6cdf6da159a7b1a0c89a9c56c7c2f42a
+size 189510738752

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:525a5accb136f4dc99d0fbce287ff53f85b98c009e6cc71e83484e0f40e07713
+size 240604934976

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb3a28424f36be0935f10b02593cdc9ceb26bb7ff2c5b0072eedf6da85b9d0d1
+size 325548426048

Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2466e45acd823c062de10a325336af3a864338a60a89b12cab797755e9ce5306
+size 421507363648

README.md

@@ -0,0 +1,193 @@
+---
+license: apache-2.0
+tags:
+  - uncensored
+  - abliterated
+  - gguf
+  - qwen
+  - qwen3.5
+  - moe
+  - 397b
+  - 17b-active
+  - fine-tuned
+  - reasoning
+  - lora
+  - opus
+  - claude
+base_model: Qwen/Qwen3.5-397B-A17B
+model_type: qwen3_5_moe
+pipeline_tag: text-generation
+language:
+  - en
+  - zh
+  - ja
+  - ko
+  - fr
+  - de
+  - es
+  - pt
+  - ru
+  - ar
+  - th
+  - vi
+  - id
+---
+
+# Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-GGUF
+
+**The world's first reasoning-enhanced uncensored 397B model.** Abliterated + LoRA fine-tuned on 12,842 high-quality reasoning samples distilled from Anthropic's Opus 4.6 outputs.
+
+This is **Stage 2** of the Qwen3.5-397B pipeline:
+- **[Stage 1](https://huggingface.co/timteh673/Qwen3.5-397B-A17B-Uncensored-GGUF)** — Abliterated (refusals removed), no fine-tuning
+- **Stage 2 (this repo)** — Abliterated + LoRA reasoning fine-tune. Better chain-of-thought, deeper analysis, more structured problem-solving
+
+397B total parameters, **17B active per token** (Mixture-of-Experts). Trained for 3,046 steps across 8×H200 GPUs. Final loss: **0.363**, accuracy: **90.2%**.
+
+## What's Different From Stage 1
+
+| | Stage 1 (Abliterated Only) | **Stage 2 (This Model)** |
+|---|---|---|
+| Abliteration | ✅ Custom pipeline | ✅ Same pipeline |
+| Fine-tuning | None | **LoRA r=64, 134.7M trainable params** |
+| Training data | None | **12,842 reasoning samples (Opus 4.6 distillation)** |
+| Reasoning quality | Base Qwen3.5 | **Enhanced chain-of-thought + structured analysis** |
+| Thinking mode | Default | **Trained with `<think>` tags for explicit reasoning** |
+| Final loss | N/A | **0.363** |
+| Final accuracy | N/A | **90.2%** |
+
+## Training Details
+
+**LoRA Configuration:**
+- Rank: 64, Alpha: 128
+- Target modules: `self_attn.{q,k,v,o}_proj`, `shared_expert.{gate,up,down}_proj`
+- Trainable parameters: 134.7M / 396.5B total (0.034%)
+- Gradient checkpointing: enabled
+
+**Training Hyperparameters:**
+- Optimizer: AdamW (fused)
+- Learning rate: 1.5e-5 (cosine scheduler)
+- Effective batch size: 64
+- Sequence length: 4,096
+- Epochs: 2 (3,046 total steps)
+- Warmup: 3%
+- Precision: BF16
+
+**Dataset Composition (12,842 samples, deduplicated):**
+- `opus-10000x`: 9,633 multi-turn conversations with deep reasoning
+- `opus-3000x`: 2,326 problem/thinking/solution samples with explicit chain-of-thought
+- `reasoning-700x`: 633 complex reasoning and analytical tasks
+- `high-reasoning-250x`: 250 elite-tier reasoning samples requiring multi-step deduction
+
+All samples feature reasoning traces distilled from Anthropic's Claude Opus 4.6, including `<think>` tag formatting for explicit chain-of-thought.
+
+**Training Curve:**
+- Step 0: Loss 0.83
+- Step 500: Loss ~0.52
+- Step 1000: Loss ~0.42
+- Step 2000: Loss ~0.39
+- Step 3046: **Loss 0.363, Accuracy 90.2%**
+
+## Quantizations
+
+| Quant | Size | BPW | RAM Required | Description | Use Case |
+|-------|------|-----|-------------|-------------|----------|
+| **BF16** | 739 GB (2 splits) | 16.01 | ~750 GB | Full precision | Reference, maximum quality |
+| **Q8_0** | 393 GB | 8.51 | ~400 GB | 8-bit | Best quality with compression |
+| **Q6_K** | 304 GB | 6.57 | ~310 GB | 6-bit | High quality, good compression |
+| **Q4_K_M** | 225 GB | ~4.85 | ~230 GB | 4-bit mixed | **Recommended for most users** |
+| **Q3_K_M** | 177 GB | ~3.83 | ~185 GB | 3-bit mixed | Memory-constrained setups |
+| **Q2_K** | 135 GB | ~2.92 | ~140 GB | 2-bit | Extreme compression |
+
+> **Note:** Q5_K_M is unavailable due to infrastructure loss during upload. Will be regenerated and uploaded in a future update.
+
+## Architecture
+
+- **Type:** Qwen3.5MoeForConditionalGeneration (hybrid GatedDeltaNet + MoE Transformer)
+- **Total Parameters:** 397B
+- **Active Parameters:** 17B per token
+- **Hidden Size:** 4,096
+- **Layers:** 60
+- **Attention:** 32 heads (GQA, 2 KV heads), head_dim 256
+- **Experts:** 512 routed + shared expert, 10 active per token
+- **Hybrid Attention:** GatedDeltaNet linear attention + self-attention every 4th layer
+- **Context Length:** 262,144 tokens
+- **Vocab Size:** 248,320
+- **Multimodal:** Native vision encoder (text + image + video)
+- **Languages:** 201+ (en, zh, ja, ko, fr, de, es, pt, ru, ar, th, vi, id, ...)
+- **License:** Apache 2.0
+
+## Usage
+
+### llama.cpp
+
+```bash
+# Recommended: Q4_K_M for balanced quality/memory
+./llama-cli -m Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q4_K_M.gguf \
+  -p "You are a helpful uncensored assistant with strong reasoning abilities." \
+  -n 2048 --temp 0.7 --top-p 0.9
+
+# Server mode with large context
+./llama-server -m Qwen3.5-397B-A17B-Opus-4.6-Reasoning-Uncensored-Q4_K_M.gguf \
+  --port 8080 --host 0.0.0.0 -c 131072
+```
+
+### LM Studio
+
+Download the GGUF file and load it in LM Studio. The model supports `<think>` reasoning tags — enable thinking mode for best results on complex tasks.
+
+### Open WebUI / SillyTavern
+
+Point your backend to a llama.cpp server. Full OpenAI-compatible API at `/v1/chat/completions`.
+
+## Pipeline
+
+```
+Qwen3.5-397B-A17B (base)
+    ↓ Custom abliteration (strength 20.0, attn.o_proj + shared_expert.down_proj)
+    ↓ LoRA fine-tuning (12,842 Opus 4.6 reasoning samples, 3,046 steps)
+    ↓ LoRA merge into base weights
+    ↓ BF16 GGUF conversion (llama.cpp)
+    ↓ Quantization cascade (Q8_0 → Q2_K)
+```
+
+All processing done in BF16 on 8×H200 SXM5 GPUs (1.1TB VRAM total). Abliteration and quantization applied in correct order: full-precision abliteration → training → merge → THEN quantize.
+
+## Known Limitations
+
+- **Q5_K_M missing:** Lost during infrastructure migration. Will be regenerated.
+- **Packed expert abliteration:** The 512 routed experts use packed tensor format and were not individually abliterated. Some edge-case refusals may persist.
+- **Vision:** Multimodal vision encoder is preserved but untested post-training. Text generation is the primary target.
+- **Thinking mode:** The model generates `<think>` tags for reasoning. Strip them in post-processing if unwanted.
+
+## Model Provenance
+
+- **Base model:** [Qwen/Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) (Apache 2.0)
+- **Stage 1:** [timteh673/Qwen3.5-397B-A17B-Uncensored-GGUF](https://huggingface.co/timteh673/Qwen3.5-397B-A17B-Uncensored-GGUF)
+- **Training data:** Custom reasoning dataset distilled from Anthropic Opus 4.6 outputs
+- **Quantization:** llama.cpp
+- **Hardware:** 8×NVIDIA H200 SXM5, 1.1TB VRAM
+
+## Disclaimer
+
+⚠️ This model has had safety alignment significantly reduced and has been fine-tuned for enhanced reasoning. It may generate content that is harmful, offensive, or inappropriate. Users are solely responsible for ensuring their use complies with applicable laws and ethical standards. This release is intended for research, testing, and controlled environments.
+---
+
+## ☕ Support This Work
+
+<a href="https://buymeacoffee.com/timteh" target="_blank">
+  <img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" width="217" height="60">
+</a>
+
+<p align="center">
+  <img src="bmac-qr.png" alt="Buy Me a Coffee QR Code" width="250">
+</p>
+
+Every donation helps fund more open-weight model releases. ⚡ Forged on 8×NVIDIA H200 SXM5 | 1.1TB VRAM
+
+### 💎 Crypto Donations
+
+| Currency | Address |
+|----------|---------|
+| **BTC** | `bc1p4q7vpwucvww2y3x4nhps4y4vekye8uwm9re5a0kx8l6u5nky5ucszm2qhh` |
+| **ETH** | `0xe5Aa16E53b141D42458ABeEDb00a157c3Fea2108` |
+| **SOL** | `9CXwjG1mm9uLkxRevdMQiF61cr6TNHSiWtFRHmUEgzkG` |