Qwen3.8-Flash-Next-Uncensored ยท MTPLX Optimized Speed
English summary: This is an Apple Silicon (MLX/MTPLX) quantization of
orcarouter/Qwen3.8-Flash-Next-Uncensored โ
OrcaRouter's abliterated (refusal-removed) BF16 build of Qwen's Qwen3.8-Flash-Next (125B-A6B MoE with a native MTP
speculative-decoding head). Quantization was performed locally with MTPLX Forge using the official
flash-next-optimized-speed recipe (4-bit body + 8-bit sparse-attention projections), the same recipe as
Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Speed.
The native MTP head is preserved and verified: 2.07x faster than plain decoding in Forge verification on an M5 Max 128 GB.
Requires MTPLX on Apple Silicon (96โ128 GB+ unified memory).
่ฟๆฏไปไน
่ฟๆฏ OrcaRouter ๅปๅฎกๆฅๆ้็ MTPLX ้ๅ็ใๅ ทไฝๆฅ่ฏด๏ผ
- ๆ้ๆฅๆบ๏ผorcarouter/Qwen3.8-Flash-Next-Uncensored โโ OrcaRouter ๅฏน Qwen/Qwen3.8-Flash-Next๏ผ125B ๅๆฐ / ็บฆ 6B ๆฟๆดป็ MoE๏ผๆททๅๆณจๆๅ๏ผๅ็่ง่ง๏ผ่ชๅธฆ MTP ๆๆบ่งฃ็ ๅคด๏ผๅ็ๆถ่ๅผๅปๅฎกๆฅ๏ผabliteration๏ผBF16 ็ๆฌ
- ้ๅๅถไฝ๏ผๆฌไปๅบ็้ๅๅนถ้ Orca ๅฎๆนๅๅธ๏ผ่ๆฏไฝฟ็จ MTPLX Forge๏ผv2.12.2๏ผๅจๆฌๅฐไปไธ่ฟฐ BF16
ๆ้ๆๅปบ๏ผ้็จ MTPLX ๅฎๆนๅ
็ฝฎ้
ๆน
flash-next-optimized-speedโโ ไธ Youssofal ๅฎๆนๅ ๅฎๅ จ็ธๅ็้ ๆน - ่ฟ่ก็ฏๅข๏ผMTPLX๏ผApple Silicon ไธๅฑ๏ผmacOS 14+๏ผ๏ผ ๅฉ็จๆจกๅ่ชๅธฆ็ MTP ๅคดๅ็ฒพ็กฎๆๆบ้ๆ ทๅ ้
้ๅ้ ็ฝฎ๏ผq4 + q8 ๆณจๆๅ๏ผ
| ็ปไปถ | ็ฒพๅบฆ | ่ฏดๆ |
|---|---|---|
| ๆจกๅไธปไฝ๏ผไธๅฎถใๆๅฝฑ็ญ๏ผ | 4-bit affine, group 32 | ๅจๆๆททๅ็ฒพๅบฆ |
| ็จ็ๆณจๆๅๆๅฝฑ๏ผQSA๏ผ | 8-bit, group 64 | ไฟ่ดจ้็ๅ ณ้ฎ้จไฝ |
| MTP ๆๆบ่งฃ็ ๅคด | 4-bit, group 64 | ๅ็ๅคดๅฎๆดไฟ็ |
| n-gram ๆฅ่ฏ่กจ | 4-bit, group 32๏ผ29.8 GiB๏ผ | ่ฟ่กๆถไป SSD ๆตๅผ่ฏปๅ๏ผไธๅ ๅ ๅญ |
| ่ง่งๅก | ๅฎๆดไฟ็ | 333 ไธชๅผ ้ |
| lm_head / embed_tokens | 8-bit, group 64 |
- ๆปๅคงๅฐ๏ผ115.3 GB๏ผๆๆ็ฒพๅบฆ็บฆ 5.14 bit/ๆ้
- ๅธธ้ฉปๅ ๅญ็บฆ 83 GiB๏ผๅฎๆนๅปบ่ฎฎ 128 GB ็ปไธๅ ๅญ็ Mac๏ผๆไฝ 96 GB ๆกฃไฝ่ฏท็จ Bare Speed ็ฑป้ ๆน๏ผ
- ไธไธๆ็ชๅฃ๏ผ262,144 tokens
ๅฎๆตๆฐๆฎ๏ผM5 Max 128 GB๏ผMTPLX 2.12.2๏ผๆจกๅๅ็้ๆ ท่ฎพ็ฝฎ temp 1.0 / top-p 0.95 / top-k 20๏ผ
Forge ๅฎๆน้ช่ฏ๏ผๆๅปบๆถ่ชๅจๅฎๆ๏ผ
| ๆจกๅผ | ้ๅบฆ | ๅ็ |
|---|---|---|
| ๆฎ้่ชๅๅฝ่งฃ็ ๏ผdepth 0๏ผ | 41.3 tok/s | 1.0x |
| MTP ๆๆบ่งฃ็ ๏ผdepth 3๏ผ | 85.7 tok/s | 2.07x |
้ช่ฏ็ป่ฎบ mtp_depth_wins๏ผMTP ๅฅ็บฆ็ป 32 ็ปๅ้ๅฎๆตๆ กๅ๏ผไฟ็ๅฎถๆ้ป่ฎคๅฅ็บฆใ
ๅฎ้ ไฝฟ็จๅบๆฏๅคๆต๏ผOpenAI ๅ ผๅฎน API๏ผๅ ณ้ญๆ่ๆจกๅผ๏ผ
| ๅบๆฏ | ๆฌๅ | ๅฎๆนๅ ๏ผๅฏน็ ง๏ผ | ่ฏดๆ |
|---|---|---|---|
| ็ผ็จไปปๅก๏ผPython ็ผๅญ็ฑป + ๅๅ ๆต่ฏ๏ผ | 52โ71 tok/s | 85 tok/s | ่็จฟๅฝไธญ็็บฆ 92%/86%/81%๏ผไธๅฎๆนๅ ๅ ไนไธ่ด๏ผๆ ก้ช่ทฏๅพ็ฅๆ ขไบๅฎๆน็็ซ ็ |
| ไธญๆ้ฟๆๅไฝ๏ผ2500 ๅญๆ็ซ ๏ผ | 26โ34 tok/s | 32 tok/s | ๅๆๅไฝไธๅฏ้ขๆตๆง้ซ๏ผๅฝไธญ็ๅคฉ็ถๅไฝ๏ผไธคๅ ่กจ็ฐไธ่ด๏ผ๏ผ้ไฝๆธฉๅบฆๅฏๆๆพๆ้ |
| ้ฆ token ๅปถ่ฟ | 0.22โ0.29 s | 0.22 s | ๅฐๆ็คบ่ฏไธๅบๆฌ็ฌๆถ |
ๆฌๆบๆทฑๅบฆ่ฐไผ๏ผmtplx tune --retune๏ผM5 Max 128 GB๏ผ
| ๆจกๅผ | ้ๅบฆ | ็ธๅฏน่ฃธ่งฃ็ |
|---|---|---|
| AR๏ผๆ ๆๆบ่งฃ็ ๏ผ | 42.0 tok/s | 1.00x |
| D1 | 56.0 tok/s | 1.33x |
| D2 | 40.8 tok/s | 0.97x๏ผๆฏ่ฃธ่งฃ็ ่ฟๆ ข๏ผ |
| D3 | 66.4 tok/s | 1.58x๏ผๆฌๆบๆไผ |
็ป่ฎบ๏ผ็ผ็จ / ็ปๆๅ่พๅบๅบๆฏๆ้ๆพ่๏ผ็ธๅฏน่ฃธ่งฃ็ 1.3โ2.1 ๅ๏ผ๏ผ้ซๆธฉๅๆๅไฝๅบๆฏๅปบ่ฎฎ่ฐไฝ temperature ๏ผๅฆ 0.6โ0.7๏ผไปฅๆ้ซ่็จฟๅฝไธญ็ใ
ไฝฟ็จๆนๆณ
# ๅฎ่ฃ
MTPLX๏ผ้่ฆ Apple Silicon + macOS 14+๏ผ
brew install youssofal/mtplx/mtplx
# ๆๅๅนถ่ฟ่กๆฌๆจกๅ
mtplx pull NeosQ/Qwen3.8-Flash-Next-Uncensored-MTPLX-Optimized-Speed
mtplx serve --model NeosQ/Qwen3.8-Flash-Next-Uncensored-MTPLX-Optimized-Speed
ๆๅกๅฏๅจๅๅจ http://127.0.0.1:8000 ๆไพ OpenAI / Anthropic ๅ
ผๅฎน API๏ผๅฏ็ดๆฅๆฅ Claude CodeใClineใ
OpenCode ็ญๅฎขๆท็ซฏใไนๅฏไปฅ็ดๆฅไฝฟ็จ MTPLX ๆก้ขๅบ็จ๏ผๅจๆจกๅๅ่กจไธญ้ๆฉๅณๅฏใ
โ ๏ธ ๅ ่ดฃๅฃฐๆ๏ผๆฒฟ็จ่ชไธๆธธ๏ผ
ๆฌๆจกๅ็ไธๆธธๆ้็ป่ฟไบๅปๅฎกๆฅๅค็๏ผabliteration๏ผ๏ผๅ ถๅฎๅ จๅฏน้ฝๅทฒ่ขซๅคงๅน ็งป้ค๏ผ
- ๅฎไผๅๅบๅ็ Qwen3.8-Flash-Next ๆ็ป็ๆๅฎณใไธ้ๅพทๆ้ๆณ่ฏทๆฑ๏ผๆฒกๆๅ ็ฝฎๆคๆ
- ไป ไพๅๆณ็ ็ฉถ็จ้๏ผๅฏ่งฃ้ๆงใAI ๅฎๅ จไธๆ็ญๆบๅถ็ ็ฉถใ็บข้ๆต่ฏใๅๆงๅฎ้ช๏ผ
- ไฝฟ็จ่ ๅฏนไฝฟ็จๆนๅผๅ็ๆๅ ๅฎนๆฟๆ ๅ จ้จ่ดฃไปป๏ผ่ฏทๅฟๅจๆชๆทปๅ ่ชๆๅฎๅ จๅฎกๆ ธๅฑ็ๆ ๅตไธ้จ็ฝฒ็ป็ป็ซฏ็จๆท
- ไฝฟ็จ้กป้ตๅฎ็ปงๆฟ่ชๅบๅบงๆจกๅ็ Apache 2.0 ่ฎธๅฏ่ฏๅๆๆ้็จๆณๅพๆณ่ง
่ฎธๅฏไธ่ด่ฐข
- License: Apache 2.0๏ผ็ปงๆฟ่ช Qwen/Qwen3.8-Flash-Next๏ผ
- ๅปๅฎกๆฅๆ้๏ผOrcaRouter๏ผorcarouter/Qwen3.8-Flash-Next-Uncensored๏ผ
- ่ฟ่กๆถไธ้ๅ้ ๆน๏ผMTPLX by Youssof Altoukhi๏ผgithub.com/youssofal/MTPLX๏ผApache-2.0๏ผ
- ๅบๅบงๆจกๅ๏ผQwen / Alibaba
- Downloads last month
- 1,032
4-bit