Qwen3.6-27B-INT8 — PTQ Quantized (W8A16)

Overview

Post-Training Quantized (PTQ) version of Qwen/Qwen3.6-27B.

• No calibration dataset
• One-shot quantization
• Uses llmcompressor

---

Quantization

• Scheme: W8A16
• Weights: INT8
• Activations: FP16/BF16
• Targets: Linear layers only

Ignored:

• lm_head
• visual
• linear_attn
• mtp

KLD Results:

• Mean KLD: 0.009441

Usage

vllm serve TheHouseOfTheDude/Qwen3.6-27B-INT8 \
  --quantization compressed-tensors

---

Notes

• Requires vLLM
• Not for vanilla Transformers