Text Generation
Transformers
outlier_150b_rexmoe
mixture-of-experts
Mixture of Experts
ternary
quantized
qwen2.5
outlier
local-llm
on-device
edge-ai
energy-efficient
sparse
overlay
research
apple-silicon
mac
mmlu-verified
custom_code
Eval Results (legacy)
Instructions to use Outlier-Ai/Outlier-150B-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Outlier-Ai/Outlier-150B-V3.2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Outlier-Ai/Outlier-150B-V3.2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Outlier-Ai/Outlier-150B-V3.2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
- SGLang
How to use Outlier-Ai/Outlier-150B-V3.2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Outlier-Ai/Outlier-150B-V3.2 with Docker Model Runner:
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
docs(card): SEO + cross-link refresh (HF-DISCOVERABILITY-001)
Browse files
README.md
CHANGED
|
@@ -64,153 +64,28 @@ model-index:
|
|
| 64 |
value: 84.46
|
| 65 |
name: MMLU 5-shot
|
| 66 |
---
|
|
|
|
| 67 |
|
| 68 |
-
# Outlier-150B
|
| 69 |
|
| 70 |
-
|
| 71 |
-
Sparse architecture: shared full-precision FFN plus a gated ternary expert FFN per layer.
|
| 72 |
-
Built by a solo founder on a Mac Studio as part of the Outlier research line feeding
|
| 73 |
-
the [Outlier desktop app](https://outlier.host) for Apple Silicon.
|
| 74 |
|
| 75 |
-
##
|
| 76 |
|
| 77 |
-
|
| 78 |
-
- **Architecture:** Ternary MoE overlay (shared FFN + alpha times expert FFN, TQ1_0 packing)
|
| 79 |
-
- **Experts:** 88 shared x 4
|
| 80 |
-
- **Frozen base:** [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct)
|
| 81 |
-
- **MMLU 5-shot:** 84.46% plus or minus 0.29% (n=14042, lm-evaluation-harness v0.4.9.1)
|
| 82 |
-
- **License:** Apache 2.0
|
| 83 |
-
- **Intended use:** research, benchmarking, derivative fine-tunes
|
| 84 |
-
- **Production Mac deployment:** use an [MLX 4-bit shipping tier](https://huggingface.co/Outlier-Ai) instead
|
| 85 |
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
-
|
| 89 |
|
| 90 |
-
|
| 91 |
-
config. Inference loads the base model separately and applies this overlay via the
|
| 92 |
-
reference runtime.
|
| 93 |
|
| 94 |
-
|
| 95 |
|
| 96 |
-
|
| 97 |
-
- Not calibrated for production throughput; shipping-tier MLX 4-bit variants exist for that
|
| 98 |
-
- Not claiming to beat the base Qwen on raw MMLU at every scale; the product thesis is
|
| 99 |
-
MMLU per GB of RAM, not raw MMLU (see comparison below)
|
| 100 |
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
| Metric | Outlier-150B V3.2 | Qwen2.5-72B-Instruct FP16 |
|
| 104 |
-
|---|---|---|
|
| 105 |
-
| MMLU 5-shot | 84.46% | Qwen published values (see base card) |
|
| 106 |
-
|
| 107 |
-
The overlay compresses the expert path to ~1.6 bits per weight via TQ1_0 while preserving
|
| 108 |
-
the shared FFN at full precision. This trades some raw accuracy for a smaller expert memory
|
| 109 |
-
footprint when paired with int4 base quantization.
|
| 110 |
-
|
| 111 |
-
## Quickstart
|
| 112 |
-
|
| 113 |
-
```python
|
| 114 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 115 |
-
import torch
|
| 116 |
-
|
| 117 |
-
base_id = "Qwen/Qwen2.5-72B-Instruct"
|
| 118 |
-
overlay_id = "Outlier-Ai/Outlier-150B-V3.2"
|
| 119 |
-
|
| 120 |
-
tok = AutoTokenizer.from_pretrained(base_id)
|
| 121 |
-
model = AutoModelForCausalLM.from_pretrained(
|
| 122 |
-
overlay_id,
|
| 123 |
-
trust_remote_code=True,
|
| 124 |
-
torch_dtype=torch.float16,
|
| 125 |
-
)
|
| 126 |
-
|
| 127 |
-
prompt = "Explain ternary mixture-of-experts in one paragraph."
|
| 128 |
-
inputs = tok(prompt, return_tensors="pt")
|
| 129 |
-
out = model.generate(**inputs, max_new_tokens=256)
|
| 130 |
-
print(tok.decode(out[0], skip_special_tokens=True))
|
| 131 |
-
```
|
| 132 |
-
|
| 133 |
-
Apple Silicon production use: pick an MLX 4-bit repo from the
|
| 134 |
-
[Outlier-Ai organization](https://huggingface.co/Outlier-Ai). Those load via `mlx_lm.generate`.
|
| 135 |
-
|
| 136 |
-
## Benchmarks (verified)
|
| 137 |
-
|
| 138 |
-
| Benchmark | Score | n | Harness |
|
| 139 |
-
|---|---|---|---|
|
| 140 |
-
| MMLU 5-shot | 84.46% plus or minus 0.29% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 141 |
-
| HellaSwag 10-shot | 77.00% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 142 |
-
| ARC-Challenge 25-shot | 68.50% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 143 |
-
| ARC-Easy 25-shot | 90.00% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 144 |
-
| Winogrande 5-shot | 85.50% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 145 |
-
| TruthfulQA 0-shot | 69.19% | 14042 | lm-evaluation-harness v0.4.9.1 |
|
| 146 |
-
|
| 147 |
-
Evaluation artifacts retained on Outlier infrastructure; ground-truth summary at
|
| 148 |
-
[outlier.host](https://outlier.host).
|
| 149 |
-
|
| 150 |
-
## Provenance
|
| 151 |
-
|
| 152 |
-
- Harness: lm-evaluation-harness v0.4.9.1
|
| 153 |
-
- Sample size: n = 14042
|
| 154 |
-
- Config: 5-shot, bf16, batch_size=1
|
| 155 |
-
- Date produced: 2026-04-13
|
| 156 |
-
- Canonical ground truth: `OUTLIER_GROUND_TRUTH_v13.md`
|
| 157 |
-
|
| 158 |
-
## Related
|
| 159 |
-
|
| 160 |
-
- **Production Mac shipping tiers:** [Outlier-Ai organization](https://huggingface.co/Outlier-Ai)
|
| 161 |
-
- **Consumer Edition collection:** [Outlier-Ai/outlier-consumer-edition-69e2fb4a0df119ea1747275e](https://huggingface.co/collections/Outlier-Ai/outlier-consumer-edition-69e2fb4a0df119ea1747275e)
|
| 162 |
-
- **Research collection:** [Outlier-Ai/outlier-research-69e2fb3a71984614b3c7a279](https://huggingface.co/collections/Outlier-Ai/outlier-research-69e2fb3a71984614b3c7a279)
|
| 163 |
-
- **Server V3.2 collection:** [Outlier-Ai/outlier-server-v32-69e2fb4b71984614b3c7a4a3](https://huggingface.co/collections/Outlier-Ai/outlier-server-v32-69e2fb4b71984614b3c7a4a3)
|
| 164 |
-
- **Desktop app:** [https://outlier.host](https://outlier.host)
|
| 165 |
-
- **Founders lifetime ($200, 500-seat cap):** [https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW](https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW)
|
| 166 |
-
- **Discord:** [discord.gg/Hapennmdn9](https://discord.gg/Hapennmdn9)
|
| 167 |
-
|
| 168 |
-
## What is Outlier?
|
| 169 |
-
|
| 170 |
-
Outlier is a Mac-native, offline by default AI platform. One desktop app, a curated model library,
|
| 171 |
-
Bring-Your-Own-Model support, projects with codebase indexing, a 9-tool coding agent, an
|
| 172 |
-
artifacts panel, SQLite memory, and an OpenAI-compatible local API - all Apache 2.0,
|
| 173 |
-
all local, no cloud round-trip, no per-token billing.
|
| 174 |
-
|
| 175 |
-
- **Desktop app:** [https://outlier.host](https://outlier.host)
|
| 176 |
-
- **Founders lifetime (500-seat cap, $200 one-time):** [support Outlier](https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW)
|
| 177 |
-
- **Discord:** [discord.gg/Hapennmdn9](https://discord.gg/Hapennmdn9)
|
| 178 |
-
- **Org:** [huggingface.co/Outlier-Ai](https://huggingface.co/Outlier-Ai)
|
| 179 |
-
|
| 180 |
-
Built solo in Grand Rapids, Michigan. 19 days to 35+ HF repos and 8,305 downloads, under
|
| 181 |
-
$1,200 in compute spend. Mission: a local AI model that codes as well as the cloud flagships,
|
| 182 |
-
funded by users who want it - not investors who need exits.
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
## Known limits
|
| 186 |
-
|
| 187 |
-
- Overlay checkpoint; the frozen base above is required.
|
| 188 |
-
- Shared FFN runs at full precision on load - plan RAM accordingly if you compose this
|
| 189 |
-
without int4 quantization on the base.
|
| 190 |
-
- Raw MMLU at 84.46% is compared above against the same-scale Qwen FP16 release;
|
| 191 |
-
the product thesis is MMLU per GB of RAM, not raw MMLU.
|
| 192 |
-
- English-tuned. Multilingual behavior inherits the base model directly.
|
| 193 |
-
|
| 194 |
-
## Attribution
|
| 195 |
-
|
| 196 |
-
Base weights by the **Qwen team at Alibaba**, released under Apache 2.0. Outlier contributes
|
| 197 |
-
the ternary MoE overlay, alpha-fix scalars, and training pipeline. All capability credit for
|
| 198 |
-
the base is upstream. Qwen team: [qwenlm.github.io](https://qwenlm.github.io/).
|
| 199 |
-
|
| 200 |
-
## Patents and citation
|
| 201 |
-
|
| 202 |
-
Architecture, training pipeline, and inference engine covered by US provisional patents
|
| 203 |
-
64/026,886, 64/030,368, and 64/034,028 (Kerr & Company LLC, 2026).
|
| 204 |
-
|
| 205 |
-
```bibtex
|
| 206 |
-
@misc{kerr2026outlier150bv3.2,
|
| 207 |
-
title={ Outlier-150B V3.2: Ternary Mixture-of-Experts Overlay for Qwen2.5 },
|
| 208 |
-
author={ Kerr, Matthew },
|
| 209 |
-
year={ 2026 },
|
| 210 |
-
howpublished={ \url{ https://huggingface.co/Outlier-Ai/Outlier-150B-V3.2 } },
|
| 211 |
-
}
|
| 212 |
-
```
|
| 213 |
-
|
| 214 |
-
## Contact
|
| 215 |
-
|
| 216 |
-
Matt Kerr - [outlier.host](https://outlier.host) - [@outlier_ai](https://twitter.com/outlier_ai)
|
|
|
|
| 64 |
value: 84.46
|
| 65 |
name: MMLU 5-shot
|
| 66 |
---
|
| 67 |
+
> **Superseded.** This repo is a research artifact from an earlier Outlier lineage and is no longer the recommended download. The current shipping tier is at **[outlier.host](https://outlier.host/?utm_source=hf&utm_medium=modelcard&utm_campaign=outlier_150b_v3_2)**.
|
| 68 |
|
| 69 |
+
# Outlier-150B-V3.2
|
| 70 |
|
| 71 |
+
This repo predates the v1.8 Outlier lineup. It is preserved here for reproducibility and historical reference, not as a production recommendation.
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
+
## What replaced it
|
| 74 |
|
| 75 |
+
Current shipping tiers (see Outlier app v1.8+):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
+
- [Outlier Nano 4B — current entry tier](https://huggingface.co/Outlier-Ai/Outlier-Nano-4B-MLX-4bit)
|
| 78 |
+
- [Outlier Core 27B — current default tier](https://huggingface.co/Outlier-Ai/Outlier-Core-27B-MLX-4bit)
|
| 79 |
+
- [Outlier Vision 35B-A3B — current multimodal tier](https://huggingface.co/Outlier-Ai/Outlier-Vision-35B-A3B-MLX-4bit)
|
| 80 |
+
- [DeepSeek-R1-Distill-Qwen-7B — popular reasoning model](https://huggingface.co/Outlier-Ai/DeepSeek-R1-Distill-Qwen-7B-MLX-4bit)
|
| 81 |
+
- [Qwen3-Coder-30B-A3B — popular coding model](https://huggingface.co/Outlier-Ai/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit)
|
| 82 |
|
| 83 |
+
For the latest verified benchmarks and downloads, visit **[outlier.host](https://outlier.host/?utm_source=hf&utm_medium=modelcard&utm_campaign=outlier_150b_v3_2)**.
|
| 84 |
|
| 85 |
+
## Original notes
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
This was a research / preview artifact. It may contain experimental adapters, overlays, or quantization variants that did not graduate into the shipping product. Treat any technical claims in earlier revisions of this card as provisional.
|
| 88 |
|
| 89 |
+
## License
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
+
See YAML frontmatter above. Original license terms preserved.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|