xunlinkx/Qwopus3.8-27B-Coder-Flash-oQ5e-fp16-mtp

Qwopus3.8-27B-Coder-Flash — MLX oQ5e-fp16-mtp (Multimodal Vision + MTP)

This is an Apple MLX enhanced oQ5 (oQ5e-fp16-mtp) quantization of Jackrong/Qwopus3.8-27B-Coder-Flash, prepared by xunlinkx using the oMLX quantization pipeline.

This model combines multimodal vision capabilities, multi-token prediction (MTP) speculative decoding, and FP16 target compute specifically tuned for native execution across all Apple Silicon generations (M1, M2, M3, M4, and newer).

---

Model Details

• Base Model: Jackrong/Qwopus3.8-27B-Coder-Flash
• Architecture: Qwen3_5ForConditionalGeneration (qwen3_5 multimodal with 27-block ViT visual encoder)
• Parameters: ~27B dense foundation model
• Quantization: 5-bit enhanced mixed precision (oQ5e) with affine quantization, group size 64, and strict importance-matrix (imatrix) calibration (128 samples, 512 sequence length).
• Target Dtype (fp16): Non-quantized weights, scales, biases, norms, and embeddings are preserved in FP16 (float16), providing peak hardware execution speed on Apple Silicon M1/M2 chips (which lack native BF16 arithmetic units) while running at full throughput on M3/M4/M5 hardware.
• Vision Tower: Preserved intact in full FP16 precision (unquantized) to prevent degradation of OCR, spatial reasoning, diagram parsing, and code screenshot comprehension.
• Speculative Decoding (MTP): Preserves the official multi-token prediction draft heads (mtp.*) in FP16 precision. When enabled in supported runtimes (oMLX, llama.cpp MTP draft), decoding achieves ~1.4x–1.7x faster token throughput on coding and agentic reasoning tasks.
• License: Apache-2.0 (inherited from the upstream base model).

---

Key Configurations

Modality & Tensors

• Multimodal Vision: Complete 27-block visual transformer encoder and merger weights preserved in model.visual.*.
• Multi-Token Prediction: Native auxiliary draft heads preserved in mtp.* with mtp_num_hidden_layers: 1 in config.json.
• Layer Sensitivities: Critical attention projections (v_proj, q_proj) and MLP down-projections on sensitive layers are boosted to 6-bit/8-bit to protect reasoning depth and indentation adherence.

---

Usage

With oMLX Server

Add to your ~/.omlx/model_settings.json:

{
  "Qwopus3.8-27B-Coder-Flash-oQ5e-fp16-mtp": {
    "mtp_enabled": true,
    "mtp_num_draft_tokens": 3,
    "temperature": 0.2,
    "top_p": 0.95
  }
}

Start serving:

omlx serve Jackrong/Qwopus3.8-27B-Coder-Flash-oQ5e-fp16-mtp --port 8005

Direct Generation with MLX-LM

mlx_lm.generate \
  --model Jackrong/Qwopus3.8-27B-Coder-Flash-oQ5e-fp16-mtp \
  --prompt "<|im_start|>user\nWrite a thread-safe LRU cache in Python with O(1) ops.<|im_end|>\n<|im_start|>assistant\n" \
  --max-tokens 512 \
  --temp 0.2

---

Attribution

Base checkpoint and coder-flash fine-tuning by Jackrong. Quantization and MLX packaging by xunlinkx using the oMLX cluster pipeline.