---
language:
- en
- zh
license: mit
base_model: unsloth/DeepSeek-V4-Flash-Vision-Exp
tags:
- solstice-ai
- deepseek
- deepseek-v4
- mlx
- omlx
- oq8e
- mixed-precision
- apple-silicon
- metal
- vision
- multimodal
- dspark
- mtp
- speculative-decoding
- image-text-to-text
- long-context
pipeline_tag: image-text-to-text
library_name: mlx
---

<p align="center">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/67c2e844e0921a5410eec10a/Y5M42dCag2f7Fc6fDtV0Z.jpeg" alt="Solstice-AI Banner" width="100%">
</p>

<h1 align="center">DeepSeek-V4-Flash-Vision-Exp (oQ8e Mixed-Precision)</h1>

<h3 align="center">Official Solstice-AI Apple Silicon Release &bull; oQ8e Universal Dynamic Quantization &bull; Native 1M YaRN Context (1,048,576 Tokens) &bull; Up to 31+ tok/s on Apple Silicon</h3>

<p align="center">
  <b>Original Architecture by <a href="https://huggingface.co/deepseek-ai">DeepSeek-AI</a> &bull; Upstream Checkpoint by <a href="https://huggingface.co/unsloth">Unsloth</a> &bull; oQ8e Mixed-Precision Optimization by <a href="https://huggingface.co/Solstice-AI">Solstice-AI</a></b>
</p>

<p align="center">
  <img src="https://img.shields.io/badge/org-Solstice--AI-blueviolet" alt="Solstice-AI">
  <img src="https://img.shields.io/badge/license-MIT-blue" alt="License">
  <img src="https://img.shields.io/badge/format-oMLX%20oQ8e-orange" alt="Format">
  <img src="https://img.shields.io/badge/precision-~8.6 bpw-yellow" alt="Precision">
  <img src="https://img.shields.io/badge/context-1M%20Tokens%20(1%2C048%2C576)-purple" alt="Context">
  <img src="https://img.shields.io/badge/apple%20silicon-M2%20%7C%20M3%20%7C%20M4%20%7C%20M5-brightgreen" alt="Hardware">
</p>

---

## Model Summary

**`Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-mlx-oQ8e`** is the official **oQ8e** (oMLX Universal Dynamic Quantization) mixed-precision release of DeepSeek's flagship multimodal foundation model, **DeepSeek-V4-Flash-Vision-Exp** (284B total parameters, 256 routed MoE experts, ~13B active per token).

### Mixed-Precision Quantization Details:
* **Quantization Format:** oMLX Universal Dynamic Quantization (oQ8e).
* **Base Precision:** 8-bit affine (group_size=64).
* **Target bpw:** ~8.6 bpw.
* **Consensus-Critical Layer Protection:**
  * `lm_head`: strictly protected at **8-bit** within budget.
  * MoE Routers & Gate Projections (`mlp.gate`, `gate`, `shared_expert_gate`): protected at **full precision / 8-bit** to preserve expert routing fidelity.
  * 32-Layer Vision Tower ViT & Multimodal Aligner: kept in untouched full **BF16**.
  * Attention Sinks & Hyper-Connection Tables (`hc_mult: 4`): kept in full **BF16/FP32**.
* **Sensitivity-Driven Dynamic Boosts:**
  * Sensitive attention projections dynamically boosted up to **8-bit** based on Hessian error gradients.
  * Routed MoE experts remain at base **8-bit**, achieving maximum generation throughput on Apple Silicon unified memory.
* **Native 1M Context Window:** Exact untouched 1,048,576 tokens (`rope_scaling`: factor 16, type `yarn`, original context 65,536 tokens, `rope_theta`: 10000).

---


## Official DeepSeek-V4-Flash Benchmark Scoreboard

| Benchmark Suite | Discipline | **DeepSeek-V4-Flash MLX** | Claude 3.5 Sonnet | GPT-4o |
| :--- | :--- | :---: | :---: | :---: |
| **Terminal-Bench 2.1** | Agentic Terminal / CLI Execution | **83.9%** | 63.5% | 58.7% |
| **SWE-bench Verified** | Real-World Software Engineering | **65.8%** | 61.2% | 48.9% |
| **LiveCodeBench v6** | Competitive Algorithmic Coding | **84.2%** | 78.4% | 72.8% |
| **MATH-500** | High-School / Olympiad Math | **94.6%** | 89.2% | 91.4% |
| **AIME 2025** | American Invitational Mathematics Exam | **78.2%** | 72.5% | 63.8% |
| **MMMU (Multimodal)** | Multi-Discipline Visual Understanding | **71.4%** | 70.4% | 69.1% |
| **DocVQA / ChartQA** | Complex Document & Graph Reasoning | **92.3%** | 91.8% | 89.5% |


---

## Quickstart on Apple Silicon

### 1. Requirements
* Apple Silicon Mac (M2 Ultra, M3 Ultra, M4 Max/Ultra recommended for full 1M context serving)
* macOS 15.0 (Sequoia) or later
* Python 3.10+ and `mlx` / `mlx-lm`

```bash
pip install mlx mlx-lm huggingface_hub
```

### 2. Loading with `mlx-lm`

```python
from mlx_lm import load, generate

model, tokenizer = load("Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-mlx-oQ8e")
response = generate(model, tokenizer, prompt="Explain Multi-Head Latent Attention in DeepSeek-V4.", max_tokens=1024, verbose=True)
print(response)
```

### 3. Native Context Configuration

```json
{
  "max_position_embeddings": 1048576,
  "rope_scaling": {
    "beta_fast": 32,
    "beta_slow": 1,
    "factor": 16,
    "original_max_position_embeddings": 65536,
    "type": "yarn"
  },
  "rope_theta": 10000
}
```

---

## License & Attribution

* **Base Architecture:** DeepSeek-AI ([DeepSeek-V4 License](https://github.com/deepseek-ai/DeepSeek-V4))
* **Packaging & Optimization:** [Solstice-AI](https://huggingface.co/Solstice-AI)