Qwopus3.6-27B-v2-oQ4-mtp

This repository contains an oMLX/oQ 4-bit quantized MLX version of Jackrong/Qwopus3.6-27B-v2.

It is intended for Apple Silicon local inference testing, with MTP weights preserved for faster generation where supported.

Model lineage

• Original model: Jackrong/Qwopus3.6-27B-v2
• Quantized model: AbarthJoe/Qwopus3.6-27B-v2-oQ4-mtp
• Quantization format: MLX / oMLX oQ
• Relationship: Quantized derivative of the original model

Quantization details

• Quantization tool: oMLX / oQ
• Quantization level: oQ4
• Preserve MTP weights: Yes
• Non-quant weight dtype: bfloat16 / default
• Output format: MLX
• Target platform: Apple Silicon

Expected use case

This version is intended as the speed-focused build in this quantized model family.

Compared with higher-bit versions, oQ4 should provide:

• Lower storage size
• Lower memory usage
• Faster local inference
• Potentially lower output quality than oQ6 / oQ8

Please validate the model quality for your own use case before relying on it.

Benchmark

Tested on Apple Silicon.

Model	Context	Token generation
oQ4 + MTP	1k	~21.9 tok/s
oQ4 + MTP	4k	~20.8 tok/s

Benchmark results may vary depending on hardware, software version, prompt type, context length, and runtime settings.

Related models

Other oMLX/oQ quantized versions are available in this collection:

Qwopus oMLX oQ Quantized Models for Apple Silicon

Credits

Original model by Jackrong.

This quantized MLX/oQ version was created and uploaded by AbarthJoe.

License

The original model is licensed under Apache-2.0.
This quantized version follows the same Apache-2.0 license where applicable.

Disclaimer

This is a community quantized model for research, experimentation, and local inference testing.

It has not been fully safety-evaluated or benchmarked across all tasks.
Please test carefully before using it for production, sensitive, or high-stakes use cases.