ShinMK3/Mega-Brain-Distill
Viewer • Updated • 42.5k • 664 • 2
ShinMK
ShinMK3
·
AI & ML interests
None yet
Recent Activity
reacted to ManniX-ITA's post with 🚀 6 days ago
🚀 New release: Qwen3.6-27B-A3B-Coder
A code-specialized MoE carved out of Qwen3.6-35B-A3B by pure expert pruning — no fine-tuning, no distillation. I profiled all 256 experts on balanced corpora plus targeted code benchmarks (LiveCodeBench +
MultiPL-E), built a competence map with the code classes up-weighted 1.5×, and dropped the 72 weakest experts per layer (256→184, ~35B→27B). Router, attention, norms, the MTP head and the vision tower are all
preserved; active params stay at A3B and routing is baked to top-10 (revert to top-8 anytime).
📊 Benchmarks (Q6_K, temp 0.6):
• MultiPL-E 0.840
• HumanEval 0.970
• LiveCodeBench 0.688
• GSM8K 0.970 · ARC-C 0.944 · AIME 0.733
• GPQA-Diamond 0.773 · MATH-500 0.620 · IFEval 0.730
• Average 0.808
27B footprint, A3B speed, coding that punches well above its size — and the preserved MTP head gives you speculative decoding out of the box (text + vision).
🔗 Model: https://huggingface.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder
📦 GGUF (+MTP): https://huggingface.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder-MTP-GGUF
🦙 Ollama: https://ollama.com/mannix/qwen3.6-27b-a3b-coder updated a dataset 14 days ago
ShinMK3/Mega-Brain-Distill liked a dataset 15 days ago
ShinMK3/Mega-Brain-DistillOrganizations
