PocketAiHub/Qwen3.8-27B-MLX

Qwen3.8-27B MLX

Official-source MLX releases of Qwen/Qwen3.8-27B, converted and validated by PocketAI Model Lab. This repository contains an experimental 2-bit AWQ build plus validated 4-bit, 6-bit, 8-bit, and unquantized BF16 variants derived from the same pinned official revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.

Variants

Variant	Folder	Stored size	Precision layout
MLX 2-bit AWQ (experimental)	2bit/	11,038,990,758 bytes (10.28 GiB)	498 language modules affine Q2/group 32 with AWQ; vision tower BF16
MLX 4-bit	4bit/	16,081,493,231 bytes (14.98 GiB)	498 language modules affine Q4/group 64; vision tower BF16
MLX 6-bit	6bit/	22,804,831,851 bytes (21.24 GiB)	498 language modules affine Q6/group 64; vision tower BF16
MLX 8-bit	8bit/	29,528,170,361 bytes (27.50 GiB)	498 language modules affine Q8/group 64; vision tower BF16
MLX BF16	bf16/	54,740,455,716 bytes (50.98 GiB)	Unquantized BF16 reference

Validation

The 4-bit, 6-bit, 8-bit, and BF16 variants passed the complete deterministic PocketAI feature suite:

• 12/12 quality regression cases
• 8/8 tool-calling cases
• temporal video understanding (red->blue)
• 4K-context needle retrieval (COBALT-7319)

The 4-bit variant additionally passed a 32K-context retrieval test at 32,770 actual prompt tokens.

The 2-bit build is intentionally marked experimental. It passed the text and image runtime smoke tests and exact 4K retrieval, but scored 11/12 on the compact quality gate, 3/8 on tool calling, and did not pass the synthetic temporal-video gate. Use 4-bit when instruction following, tools, or video understanding matter more than memory footprint.

4K MLX performance

These are warmed, single-run measurements on an Apple M5 Max with 128 GB unified memory using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0, and thinking disabled. The prompt contained 4,105 tokens and the exact retrieval response contained 9 generated tokens.

Variant	Prompt prefill	Generation	End-to-end	Peak MLX memory
MLX 2-bit AWQ (experimental)	541.7 tok/s	38.8 tok/s	7.82 s	16.00 GB
MLX 4-bit	733.1 tok/s	36.6 tok/s	5.86 s	21.80 GB
MLX 6-bit	584.7 tok/s	26.2 tok/s	7.38 s	29.54 GB
MLX 8-bit	600.1 tok/s	20.1 tok/s	7.30 s	37.27 GB
MLX BF16	623.8 tok/s	10.8 tok/s	7.43 s	58.29 GB

The full machine-readable evidence is in benchmarks/4k-feature-suite.json. These compact deterministic checks are regression gates, not a broad claim of quality parity across precisions.

Download and load

Install the validated runtime on an Apple Silicon Mac:

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant and load its local subfolder:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-27B-MLX"
variant = "4bit"  # "2bit", "4bit", "6bit", "8bit", or "bf16"

snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

For image input, pass an image path to mlx_vlm.generate and build the prompt with num_images=1. Video input was validated with MLX-VLM's video path and FPS arguments.

Reproducibility

• Official source: Qwen/Qwen3.8-27B
• Pinned source revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• Converter: mlx-vlm==0.6.8
• Base dtype: BF16
• Quantization: MLX affine; 2-bit uses AWQ/group 32, while 4/6/8-bit use group 64
• Every variant includes an artifact-manifest.json with exact file sizes and SHA-256 hashes.
• release-manifest.json records the release inputs and validation evidence.

License and attribution

These conversions retain the original model's Apache 2.0 license. See LICENSE and the official Qwen model card.

• PocketAI Model Lab: https://github.com/PocketAIHub/pocketai-model-lab