Instructions to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Youssofal commited on 3 days ago

Commit

09f71b3

verified ·

1 Parent(s): 57449c6

Quantized MTP draft head (INT8/g64 affine, all 8 head matrices)

Browse files

mtp.safetensors: 451 MB (was 849 MB). Trunk weights are
unchanged. Acceptance re-verified flat-or-better against the
previous head before publishing; mtplx_runtime.json carries the
fresh fingerprint-bound verification rows and config.json declares
the prequantized head layout (loadable by MTPLX >= 2.0.1;
this model family needs >= 2.7.0).

Files changed (3) hide show

config.json +9 -1
mtp.safetensors +2 -2
mtplx_runtime.json +168 -155

config.json CHANGED Viewed

@@ -172,5 +172,13 @@
     "temporal_patch_size": 2
   },
   "vision_end_token_id": 248054,
-  "vision_start_token_id": 248053
 }

     "temporal_patch_size": 2
   },
   "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053,
+  "mtplx_mtp_quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine",
+    "policy": "all",
+    "prequantized": true,
+    "description": "All 8 MTP draft-head matrices (fc + attention q/k/v/o + MLP gate/up/down) packed MLX INT8/g64 affine from the released sidecar; head norms keep the pack's float dtype. Verified flat-or-better acceptance vs the unquantized head before publishing."
+  }
 }

mtp.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4468f39621de68a19ffd0bcb2e2e2f352205def7436a625b3427e3752866c287
-size 849400403

 version https://git-lfs.github.com/spec/v1
+oid sha256:1e350715e0509932e337eb60c074a3e47be1a9a04ea0fcb6c29b193b67c470c5
+size 451270903

mtplx_runtime.json CHANGED Viewed

@@ -1,166 +1,179 @@
 {
- "arch_id": "qwen3-next-mtp",
- "artifact_role": "forge-local",
- "base_trunk": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
- "exactness_baseline": {},
- "forge_provenance": {
-  "depth_default_note": "Depth pin REMOVED 2026-08-14 after the gated live ABBA: D2 39.5 vs D3 40.6 tok/s matched-window (tie; D3 fewer verify rounds 697 vs 883). The earlier forge-verify \"D3 18.8 collapse\" was order/JIT-confounded (mistakes/single-forge-verify-tune-rows...). Family ceiling D3 applies.",
-  "forge_inputs": {
-   "mtp_source_path": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
-   "trunk_path": "/Users/youssof/.mtplx/models/Qwen3.8-27B-MTPLX-Optimized-Quality"
-  },
-  "forge_recipe": {
-   "body_bits": 8,
-   "body_group_size": 64,
-   "body_mode": "affine",
-   "mtp_policy": "keep_bf16"
   },
-  "forged_at": "2026-08-14T11:23:31-07:00",
-  "forged_locally": true,
   "mtp_contract": {
-   "base_hidden_variant": "post_norm",
-   "concat_order": "embedding_hidden",
-   "hidden_variant": "post_norm",
-   "mtp_position_mode": "local",
-   "mtp_quant_group_size": 64,
-   "mtp_quant_mode": "affine"
   },
-  "mtplx_version": "2.6.0",
-  "published_to_hf": null,
-  "recommended_profile_note": "Restamped sustained->turbo 2026-08-14: the sustained recommendation came from the same confounded verify rows; every gated arm served turbo, medium Flappy 59.8 tok/s blended quiet-window.",
-  "source_format": "bf16_native",
-  "source_repo": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
-  "source_sha": null
- },
- "mtp_contract": {
-  "base_hidden_variant": "post_norm",
-  "concat_order": "embedding_hidden",
-  "hidden_variant": "post_norm",
-  "mtp_position_mode": "local",
-  "mtp_quant_group_size": 64,
-  "mtp_quant_mode": "affine"
- },
- "mtp_depth_max": 3,
- "mtp_sidecar": "bf16",
- "mtplx_version": "2.6.0",
- "recommended_profile": "turbo",
- "sampler": {
-  "temperature": 0.6,
-  "top_k": 20,
-  "top_p": 0.95
- },
- "speed_evidence": {
-  "acceptance_by_depth": [
-   0.9755555555555555,
-   0.9355555555555556
-  ],
-  "acceptance_collapsed": [],
-  "depth": 2,
-  "failure_reasons": [],
-  "forge_verify_rows": [
-   {
-    "acceptance_by_position": [],
-    "depth": 0,
-    "finish_reasons": {
-     "stop": 1
-    },
-    "hit_token_budget": false,
-    "hit_token_budget_count": 0,
-    "multiplier_vs_ar": 1.0,
-    "quality_passed": true,
-    "tok_s": 14.66011370222799,
-    "verify_time_s": 137.62840137502644
-   },
-   {
-    "acceptance_by_position": [
-     0.9665271966527197
     ],
-    "depth": 1,
-    "finish_reasons": {
-     "stop": 1
-    },
-    "hit_token_budget": false,
-    "hit_token_budget_count": 0,
-    "multiplier_vs_ar": 1.8908957609763113,
-    "quality_passed": true,
-    "tok_s": 27.720746854973644,
-    "verify_time_s": 36.8759049657383
-   },
-   {
-    "acceptance_by_position": [
-     0.9755555555555555,
-     0.9355555555555556
     ],
-    "depth": 2,
-    "finish_reasons": {
-     "stop": 1
     },
-    "hit_token_budget": false,
-    "hit_token_budget_count": 0,
-    "multiplier_vs_ar": 2.3151330059181516,
-    "quality_passed": true,
-    "tok_s": 33.94011310254097,
-    "verify_time_s": 38.35901796905091
-   },
-   {
-    "acceptance_by_position": [
-     0.95,
-     0.8857142857142857,
-     0.8071428571428572
     ],
-    "depth": 3,
-    "finish_reasons": {
-     "stop": 1
-    },
-    "hit_token_budget": false,
-    "hit_token_budget_count": 0,
-    "multiplier_vs_ar": 1.2850945999364838,
-    "quality_passed": true,
-    "tok_s": 18.839632953188044,
-    "verify_time_s": 71.97223836596822
-   }
-  ],
-  "greedy_diagnostic": {
-   "tok_s": 14.66011370222799
   },
-  "quality_rejected": [],
-  "tok_s": [
-   33.94011310254097
-  ],
-  "verdict": "mtp_depth_wins"
- },
- "verified_on": {
-  "hardware": "macOS-26.3.1-arm64-arm-64bit-Mach-O",
-  "machine_arch": "arm64",
-  "macos": "26.3.1",
-  "model": "Qwen3.8-27B-MTPLX-Optimized-Quality",
-  "timestamp": "2026-08-14T11:23:31-07:00"
- },
- "mtp_depth_default": 3,
- "recommended_draft_sampler": {
-  "temperature": 1.0,
-  "top_p": 0.95,
-  "top_k": 20
- },
- "release_validation": {
-  "dropday_depth_abba": {
-   "stamped_at": "2026-08-14T21:45:00-07:00",
-   "basis": "dropday uncapped ABBA campaign (clean-receipts.jsonl, temp 1.0 xhigh/medium, die-gated, max fans verified)",
-   "decode_tps_blended_d2": [
-    27.3,
-    28.0
-   ],
-   "decode_tps_blended_d3": [
-    33.2,
-    33.1
-   ],
-   "workload": "simple flappy xhigh uncapped (33k-46k tokens, EOS stop)",
-   "verdict": "depth_3 (+19.9% blended)"
   },
-  "draft_sampler_transfer_note": {
-   "stamped_at": "2026-08-14T21:45:00-07:00",
-   "basis": "dropday uncapped ABBA campaign (clean-receipts.jsonl, temp 1.0 xhigh/medium, die-gated, max fans verified)",
-   "note": "draft 1.0 adopted by transfer: OS uncapped ABBA (+13%) + Bare scaffold strict A/B (keep_official_draft_sampler_1_0); same MTP head architecture and temp-1.0 target contract. OQ depth arms measured at draft 0.6; app-receipt validates live at ship config."
   }
- }
-}

 {
+  "arch_id": "qwen3-next-mtp",
+  "artifact_role": "forge-local",
+  "base_trunk": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
+  "exactness_baseline": {},
+  "forge_provenance": {
+    "depth_default_note": "Depth pin REMOVED 2026-08-14 after the gated live ABBA: D2 39.5 vs D3 40.6 tok/s matched-window (tie; D3 fewer verify rounds 697 vs 883). The earlier forge-verify \"D3 18.8 collapse\" was order/JIT-confounded (mistakes/single-forge-verify-tune-rows...). Family ceiling D3 applies.",
+    "forge_inputs": {
+      "mtp_source_path": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
+      "trunk_path": "/Users/youssof/.mtplx/models/Qwen3.8-27B-MTPLX-Optimized-Quality"
+    },
+    "forge_recipe": {
+      "body_bits": 8,
+      "body_group_size": 64,
+      "body_mode": "affine",
+      "mtp_policy": "keep_bf16"
+    },
+    "forged_at": "2026-08-14T11:23:31-07:00",
+    "forged_locally": true,
+    "mtp_contract": {
+      "base_hidden_variant": "post_norm",
+      "concat_order": "embedding_hidden",
+      "hidden_variant": "post_norm",
+      "mtp_position_mode": "local",
+      "mtp_quant_group_size": 64,
+      "mtp_quant_mode": "affine"
+    },
+    "mtplx_version": "2.6.0",
+    "published_to_hf": null,
+    "recommended_profile_note": "Restamped sustained->turbo 2026-08-14: the sustained recommendation came from the same confounded verify rows; every gated arm served turbo, medium Flappy 59.8 tok/s blended quiet-window.",
+    "source_format": "bf16_native",
+    "source_repo": "/Users/youssof/.mtplx/models/Qwen--Qwen3.8-27B",
+    "source_sha": null,
+    "head_quantization": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine",
+      "policy": "all",
+      "quantized_at": "2026-08-20T04:49:56Z",
+      "source_sidecar_bytes": 849400403,
+      "quantized_sidecar_bytes": 451270903,
+      "tool": "scripts/build_qwen38_q4head_sidecar.py",
+      "note": "Structural head quantization of the released sidecar; no calibration, no training. Trunk weights unchanged."
+    }
   },
   "mtp_contract": {
+    "base_hidden_variant": "post_norm",
+    "concat_order": "embedding_hidden",
+    "hidden_variant": "post_norm",
+    "mtp_position_mode": "local",
+    "mtp_quant_group_size": 64,
+    "mtp_quant_mode": "affine"
   },
+  "mtp_depth_max": 3,
+  "mtp_sidecar": "int8-g64-prequantized",
+  "mtplx_version": "2.9.0",
+  "recommended_profile": "turbo",
+  "sampler": {
+    "temperature": 0.6,
+    "top_k": 20,
+    "top_p": 0.95
+  },
+  "speed_evidence": {
+    "depth": 3,
+    "tok_s": [
+      39.192696206919734
     ],
+    "acceptance_by_depth": [
+      0.9433962264150944,
+      0.8814016172506739,
+      0.8059299191374663
     ],
+    "greedy_diagnostic": {
+      "tok_s": 13.054796369549772
     },
+    "forge_verify_rows": [
+      {
+        "acceptance_by_position": [],
+        "depth": 0,
+        "finish_reasons": {
+          "stop": 1
+        },
+        "hit_token_budget": false,
+        "hit_token_budget_count": 0,
+        "multiplier_vs_ar": 1.0,
+        "quality_passed": true,
+        "tok_s": 13.054796369549772,
+        "verify_time_s": 154.4356367919827
+      },
+      {
+        "acceptance_by_position": [
+          0.9570405727923628
+        ],
+        "depth": 1,
+        "finish_reasons": {
+          "stop": 1
+        },
+        "hit_token_budget": false,
+        "hit_token_budget_count": 0,
+        "multiplier_vs_ar": 2.0262953935160954,
+        "quality_passed": true,
+        "tok_s": 26.45287374690935,
+        "verify_time_s": 35.17482435004786
+      },
+      {
+        "acceptance_by_position": [
+          0.9776119402985075,
+          0.9129353233830846
+        ],
+        "depth": 2,
+        "finish_reasons": {
+          "stop": 1
+        },
+        "hit_token_budget": false,
+        "hit_token_budget_count": 0,
+        "multiplier_vs_ar": 2.560816141204823,
+        "quality_passed": true,
+        "tok_s": 33.430933263285176,
+        "verify_time_s": 36.15608408872504
+      },
+      {
+        "acceptance_by_position": [
+          0.9433962264150944,
+          0.8814016172506739,
+          0.8059299191374663
+        ],
+        "depth": 3,
+        "finish_reasons": {
+          "stop": 1
+        },
+        "hit_token_budget": false,
+        "hit_token_budget_count": 0,
+        "multiplier_vs_ar": 3.002168329361034,
+        "quality_passed": true,
+        "tok_s": 39.192696206919734,
+        "verify_time_s": 31.090109037584625
+      }
     ],
+    "quality_rejected": [],
+    "acceptance_collapsed": [],
+    "failure_reasons": [],
+    "verdict": "mtp_depth_wins",
+    "artifact_fingerprint": "sha256:5a7a0d400ebff91dc544f7d93454d6d48d2833b4f0f7ad832d00e1d9513fd750"
+  },
+  "verified_on": {
+    "timestamp": "2026-08-19T23:08:02-07:00",
+    "hardware": "macOS-26.3.1-arm64-arm-64bit-Mach-O",
+    "machine_arch": "arm64",
+    "macos": "26.3.1",
+    "model": "Qwen3.8-27B-MTPLX-Optimized-Quality"
   },
+  "mtp_depth_default": 3,
+  "recommended_draft_sampler": {
+    "temperature": 1.0,
+    "top_p": 0.95,
+    "top_k": 20
   },
+  "release_validation": {
+    "dropday_depth_abba": {
+      "stamped_at": "2026-08-14T21:45:00-07:00",
+      "basis": "dropday uncapped ABBA campaign (clean-receipts.jsonl, temp 1.0 xhigh/medium, die-gated, max fans verified)",
+      "decode_tps_blended_d2": [
+        27.3,
+        28.0
+      ],
+      "decode_tps_blended_d3": [
+        33.2,
+        33.1
+      ],
+      "workload": "simple flappy xhigh uncapped (33k-46k tokens, EOS stop)",
+      "verdict": "depth_3 (+19.9% blended)"
+    },
+    "draft_sampler_transfer_note": {
+      "stamped_at": "2026-08-14T21:45:00-07:00",
+      "basis": "dropday uncapped ABBA campaign (clean-receipts.jsonl, temp 1.0 xhigh/medium, die-gated, max fans verified)",
+      "note": "draft 1.0 adopted by transfer: OS uncapped ABBA (+13%) + Bare scaffold strict A/B (keep_official_draft_sampler_1_0); same MTP head architecture and temp-1.0 target contract. OQ depth arms measured at draft 0.6; app-receipt validates live at ship config."
+    }
   }
+}