Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+license: apache-2.0
+base_model:
+- Qwen/Qwen3.8-27B
+tags:
+- autoround
+- llm-compressor
+---
+
+<div align="center">
+  <img 
+    src="https://cdn-uploads.huggingface.co/production/uploads/64b93e6bd6c468ac7536607e/mj6xac74jHGLqymiovObc.png" 
+    alt="The Kaitchup -- AI on a Budget" 
+    style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
+  />
+  <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;">
+    <a href="https://kaitchup.substack.com/subscribe"><strong>Subscribe and Support</strong></a>
+  </div>
+</div>
+
+
+
+This is [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) quantized with [AutoRound](https://github.com/intel/auto-round). Model packed by [llm-compressor](https://github.com/vllm-project/llm-compressor). 
+
+Quantization ("compact" settings):
+* INT data types
+* ~2.8 bits for linear layers
+* 6-bit LM head
+* BF16 MTP layers
+* 8-bit activations
+
+The model is compatible with vLLM (tested: v0.26+). Made and Tested with an RTX Pro 6000 (Google Colab's G4). 
+Currently under evaluation: [Subscribe](https://kaitchup.substack.com/subscribe) to be notified when I publish the results.
+
+- **Made by:** [The Kaitchup](https://kaitchup.substack.com/)
+
+
+## Instructions 
+
+
+```
+vllm serve [this_model_id] \
+  --tensor-parallel-size 1 \
+  --enable-auto-tool-choice \
+  --tool-call-parser qwen3_coder \
+  --reasoning-parser qwen3 
+```
+
+Add --speculative-config '{"method":"mtp","num_speculative_tokens":X}' for MTP, where good values for X are between 2 and 6 (depending on your tasks and concurrency).
+
+
+
+
+
+
+
+