Instructions to use Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision")
config = load_config("Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Jonandrop/Qwen3.6-27B-MTPLX-Optimized-Speed-Vision" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Jonandrop commited on Jul 18

Commit

3533a71

verified ·

1 Parent(s): 7ae8757

docs: add Performance section (M5 Pro 64GB, range/honest), vision graft zero-cost finding, M5 Max historical note

Browse files

Files changed (1) hide show

README.md +42 -1

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ This repository restores the multimodal vision tower that is typically stripped
 ## Architecture
 - **Body**: 4-bit affine, group size 64 (from [Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed](https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed))
-- **MTP sidecar (mtp/weights.safetensors)**: Prequantized MTP heads.
 - **Vision tower (vision_tower.safetensors, 333 tensors, bf16 unquantized)**: Extracted verbatim from the official checkpoint's own `model.visual.*` weights (shards 7 and 8 of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)) and renamed to the `vision_tower.*` prefix for MTPLX native VLM compatibility (~879 MB).
 ## Usage (MTPLX)
@@ -40,6 +40,47 @@ mtplx start --model <path-to-this-model-dir> --port 8092 \
 Serves an OpenAI-compatible endpoint supporting both text and image input payloads via `POST /v1/chat/completions`.
 ## Sources And Attribution
 | Component | Source | Revision / License |

 ## Architecture
 - **Body**: 4-bit affine, group size 64 (from [Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed](https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed))
+- **MTP sidecar (mtp/weights.safetensors)**: Prequantized MTP heads with a 3-bit affine group-64 draft-only LM head.
 - **Vision tower (vision_tower.safetensors, 333 tensors, bf16 unquantized)**: Extracted verbatim from the official checkpoint's own `model.visual.*` weights (shards 7 and 8 of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)) and renamed to the `vision_tower.*` prefix for MTPLX native VLM compatibility (~879 MB).
 ## Usage (MTPLX)
 Serves an OpenAI-compatible endpoint supporting both text and image input payloads via `POST /v1/chat/completions`.
+The MTPLX CLI defaults to `mtp_history_policy=committed`, which this model
+requires for healthy deep-position acceptance. The model's recommended
+draft sampler is temperature 0.7 (see `recommended_draft_sampler` in
+`mtplx_runtime.json`); pass `--draft-temperature 0.7` to match.
+## Performance (measured)
+Apple M5 Pro 64 GB, MTPLX 1.0.4, temperature 0.6, `mtp_history_policy=committed`,
+`--draft-temperature 0.7` (model recommended), thinking OFF, warm (4 warmup
+prompts discarded), 8 measured prompts from the calibration_coding suite,
+max_tokens=192, median tok/s.
+| Depth | tok/s (wall-clock e2e) | tok/s (decode-only) | speedup vs AR (e2e) | acceptance pos1/2/3 |
+|---|---|---|---|---|
+| AR (`--no-mtp`) | 13.7 | 14.5 | 1.00x | - |
+| MTP depth 1 | 19.7 | 20.9 | 1.44x | 0.923 |
+| MTP depth 2 | 19.9 | 20.8 | 1.46x | 0.950 / 0.753 |
+| MTP depth 3 | 18.1 | 19.7 | 1.32x | 0.900 / 0.759 / 0.648 |
+Two throughput definitions are reported:
+- **tok/s (wall-clock e2e)** = `generated_tokens / total_elapsed`, includes
+  prompt prefill. The real end-to-end speed and the honest denominator for
+  the speedup ratio.
+- **tok/s (decode-only)** = `generated_tokens / decode_elapsed`, excludes
+  prefill.
+Acceptance is stable across runs: greedy draft (temp 0.0) gives
+0.874/0.740/0.636, recommended draft (temp 0.7) gives 0.900/0.759/0.648,
+both within noise. Vision graft has zero acceptance cost relative to the
+text-only source (Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed measures
+identical acceptance 0.874/0.740/0.636 under the same config).
+Earlier baked `mtplx_runtime.json` figures (acceptance 1.0/0.98/0.94, ~63
+tok/s) were measured on Apple M5 Max 128 GB and are not reproducible on M5
+Pro 64 GB due to lower memory bandwidth; they are preserved in the
+`historical_m5max` field of `mtplx_runtime.json` for traceability. The
+vision tower is lazy-loaded (only when a request carries an image), so
+text-only throughput is unaffected by it. Numbers are indicative, not a
+benchmark suite.
 ## Sources And Attribution
 | Component | Source | Revision / License |