| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <title>Hy-MT2-30B-A3B MLX Mixed 4-6 Quantization Diagram</title> |
| <style> |
| :root { |
| --bg: #ffffff; |
| --card: #f8f9fa; |
| --border: #dee2e6; |
| --text: #212529; |
| --muted: #6c757d; |
| --accent: #0d6efd; |
| --low: #fd7e14; |
| --high: #198754; |
| } |
| * { margin: 0; padding: 0; box-sizing: border-box; } |
| body { |
| font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif; |
| background: var(--bg); |
| color: var(--text); |
| padding: 2rem; |
| line-height: 1.6; |
| } |
| .container { max-width: 1200px; margin: 0 auto; } |
| h1 { color: var(--accent); margin-bottom: 0.5rem; font-size: 1.8rem; } |
| .subtitle { color: var(--muted); margin-bottom: 2rem; font-size: 0.95rem; } |
| .legend { |
| display: flex; |
| gap: 1.5rem; |
| margin-bottom: 2rem; |
| flex-wrap: wrap; |
| } |
| .legend-item { |
| display: flex; |
| align-items: center; |
| gap: 0.5rem; |
| font-size: 0.85rem; |
| } |
| .legend-color { |
| width: 16px; |
| height: 16px; |
| border-radius: 3px; |
| border: 1px solid rgba(0,0,0,0.1); |
| } |
| .color-low { background: var(--low); } |
| .color-high { background: var(--high); } |
| .section { |
| background: var(--card); |
| border: 1px solid var(--border); |
| border-radius: 8px; |
| padding: 1.5rem; |
| margin-bottom: 1.5rem; |
| } |
| .section h2 { |
| color: var(--accent); |
| font-size: 1.1rem; |
| margin-bottom: 1rem; |
| } |
| .grid-container { |
| display: grid; |
| grid-template-columns: repeat(12, 1fr); |
| gap: 4px; |
| margin-bottom: 1rem; |
| } |
| .layer-block { |
| aspect-ratio: 1; |
| border-radius: 4px; |
| display: flex; |
| align-items: center; |
| justify-content: center; |
| font-size: 0.65rem; |
| font-weight: 600; |
| color: #fff; |
| cursor: default; |
| transition: transform 0.15s; |
| border: 1px solid rgba(0,0,0,0.1); |
| } |
| .layer-block:hover { transform: scale(1.15); z-index: 10; } |
| .block-low { background: var(--low); } |
| .block-high { background: var(--high); } |
| .stats-grid { |
| display: grid; |
| grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); |
| gap: 1rem; |
| } |
| .stat-card { |
| background: var(--bg); |
| border: 1px solid var(--border); |
| border-radius: 6px; |
| padding: 1rem; |
| text-align: center; |
| } |
| .stat-value { |
| font-size: 1.5rem; |
| font-weight: 700; |
| color: var(--accent); |
| } |
| .stat-label { |
| font-size: 0.75rem; |
| color: var(--muted); |
| margin-top: 0.25rem; |
| } |
| .bit-allocation { |
| display: grid; |
| grid-template-columns: 1fr 1fr; |
| gap: 1rem; |
| } |
| .bit-group { |
| background: var(--bg); |
| border: 1px solid var(--border); |
| border-radius: 6px; |
| padding: 1rem; |
| } |
| .bit-group h3 { |
| font-size: 0.9rem; |
| margin-bottom: 0.5rem; |
| display: flex; |
| align-items: center; |
| gap: 0.5rem; |
| } |
| .bit-group ul { |
| list-style: none; |
| font-size: 0.8rem; |
| } |
| .bit-group li { |
| padding: 0.2rem 0; |
| color: var(--muted); |
| } |
| .bit-group li strong { color: var(--text); } |
| .footer { |
| text-align: center; |
| color: var(--muted); |
| font-size: 0.75rem; |
| margin-top: 2rem; |
| padding-top: 1rem; |
| border-top: 1px solid var(--border); |
| } |
| @media (prefers-color-scheme: dark) { |
| :root { |
| --bg: #0d1117; |
| --card: #161b22; |
| --border: #30363d; |
| --text: #c9d1d9; |
| --muted: #8b949e; |
| } |
| } |
| </style> |
| </head> |
| <body> |
| <div class="container"> |
| <h1>Hy-MT2-30B-A3B MLX Mixed 4-6 Quantization</h1> |
| <p class="subtitle">Per-layer bit allocation across 48 decoder layers — 4.830 bits per weight</p> |
|
|
| <div class="legend"> |
| <div class="legend-item"><div class="legend-color color-low"></div> 4-bit (standard precision)</div> |
| <div class="legend-item"><div class="legend-color color-high"></div> 6-bit (high precision)</div> |
| </div> |
|
|
| <div class="section"> |
| <h2>Layer Map (48 Layers × 8 Experts per MoE Block)</h2> |
| <div class="grid-container" id="layerGrid"></div> |
| <p style="font-size:0.75rem;color:var(--muted);margin-top:0.5rem;"> |
| Each block represents one decoder layer. Colors indicate quantization level applied to that layer's attention and MoE weights. |
| Layers 0–5 and 42–47 use 6-bit (first/last 1/8). Every 3rd layer in the middle (8, 11, 14, 17, 20, 23, 26, 29, 32, 35, 38, 41) also uses 6-bit. |
| </p> |
| </div> |
|
|
| <div class="section"> |
| <h2>Quantization Statistics</h2> |
| <div class="stats-grid"> |
| <div class="stat-card"> |
| <div class="stat-value">4.830</div> |
| <div class="stat-label">Bits Per Weight</div> |
| </div> |
| <div class="stat-card"> |
| <div class="stat-value">17 GB</div> |
| <div class="stat-label">Total Size</div> |
| </div> |
| <div class="stat-card"> |
| <div class="stat-value">48</div> |
| <div class="stat-label">Decoder Layers</div> |
| </div> |
| <div class="stat-card"> |
| <div class="stat-value">128</div> |
| <div class="stat-label">MoE Experts</div> |
| </div> |
| <div class="stat-card"> |
| <div class="stat-value">8</div> |
| <div class="stat-label">Experts per Token</div> |
| </div> |
| <div class="stat-card"> |
| <div class="stat-value">30.6</div> |
| <div class="stat-label">tok/s (M5 Max)</div> |
| </div> |
| </div> |
| </div> |
|
|
| <div class="section"> |
| <h2>Bit Allocation Strategy</h2> |
| <div class="bit-allocation"> |
| <div class="bit-group"> |
| <h3><span class="legend-color color-low" style="width:12px;height:12px;border-radius:2px;display:inline-block;"></span> 4-bit Layers (majority)</h3> |
| <ul> |
| <li><strong>Attention:</strong> q_proj, k_proj, o_proj</li> |
| <li><strong>MoE Experts:</strong> gate_proj, up_proj</li> |
| <li><strong>Shared MLP:</strong> gate_proj, up_proj, down_proj</li> |
| <li><strong>Router:</strong> gate</li> |
| </ul> |
| </div> |
| <div class="bit-group"> |
| <h3><span class="legend-color color-high" style="width:12px;height:12px;border-radius:2px;display:inline-block;"></span> 6-bit Layers (sensitive)</h3> |
| <ul> |
| <li><strong>Attention:</strong> v_proj, down_proj</li> |
| <li><strong>MoE Experts:</strong> down_proj</li> |
| <li><strong>Shared MLP:</strong> down_proj</li> |
| <li><strong>Output:</strong> lm_head</li> |
| <li><strong>First 1/8 layers:</strong> 0–5</li> |
| <li><strong>Last 1/8 layers:</strong> 42–47</li> |
| <li><strong>Every 3rd middle:</strong> 8,11,14,17,20,23,26,29,32,35,38,41</li> |
| </ul> |
| </div> |
| </div> |
| </div> |
|
|
| <div class="section"> |
| <h2>Architecture Overview</h2> |
| <div style="font-size:0.85rem;color:var(--muted);"> |
| <p>Hy-MT2-30B-A3B is a sparse Mixture-of-Experts (MoE) model with 128 experts and 8 active per token. The model uses a custom <code>hy_v3</code> architecture with:</p> |
| <ul style="margin:0.5rem 0; padding-left: 1.5rem;"> |
| <li>RoPE positional encoding with θ = 11,158,840</li> |
| <li>QKV normalization (qk_norm: true)</li> |
| <li>Sigmoid-based routing with expert bias</li> |
| <li>Shared expert alongside sparse MoE</li> |
| <li>262K max position embeddings</li> |
| </ul> |
| </div> |
| </div> |
|
|
| <div class="footer"> |
| Generated by Hermes Agent • Mixed 4-6 Quantization Recipe • mlx-lm 0.31.3 |
| </div> |
| </div> |
|
|
| <script> |
| const numLayers = 48; |
| const grid = document.getElementById('layerGrid'); |
| |
| function getLayerClass(idx) { |
| const eighth = numLayers / 8; |
| if (idx < eighth || idx >= numLayers - eighth) return 'block-high'; |
| if ((idx - eighth) % 3 === 2) return 'block-high'; |
| return 'block-low'; |
| } |
| |
| for (let i = 0; i < numLayers; i++) { |
| const block = document.createElement('div'); |
| block.className = `layer-block ${getLayerClass(i)}`; |
| block.textContent = i; |
| block.title = `Layer ${i}: ${getLayerClass(i) === 'block-high' ? '6-bit' : '4-bit'}`; |
| grid.appendChild(block); |
| } |
| </script> |
| </body> |
| </html> |
|
|
