ur-dad-matt commited on
Commit
8d655c1
·
verified ·
1 Parent(s): 29a82bd

docs(card): SEO + cross-link refresh (HF-DISCOVERABILITY-001)

Browse files
Files changed (1) hide show
  1. README.md +15 -140
README.md CHANGED
@@ -64,153 +64,28 @@ model-index:
64
  value: 84.46
65
  name: MMLU 5-shot
66
  ---
 
67
 
68
- # Outlier-150B V3.2
69
 
70
- Ternary Mixture-of-Experts overlay for [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct).
71
- Sparse architecture: shared full-precision FFN plus a gated ternary expert FFN per layer.
72
- Built by a solo founder on a Mac Studio as part of the Outlier research line feeding
73
- the [Outlier desktop app](https://outlier.host) for Apple Silicon.
74
 
75
- ## Quick facts
76
 
77
- - **Scale:** 150B total / 70B active
78
- - **Architecture:** Ternary MoE overlay (shared FFN + alpha times expert FFN, TQ1_0 packing)
79
- - **Experts:** 88 shared x 4
80
- - **Frozen base:** [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct)
81
- - **MMLU 5-shot:** 84.46% plus or minus 0.29% (n=14042, lm-evaluation-harness v0.4.9.1)
82
- - **License:** Apache 2.0
83
- - **Intended use:** research, benchmarking, derivative fine-tunes
84
- - **Production Mac deployment:** use an [MLX 4-bit shipping tier](https://huggingface.co/Outlier-Ai) instead
85
 
86
- Re-measured on lm-eval 0.4.9.1 (prior 83.16% was harness-version artifact).
 
 
 
 
87
 
88
- ## What this is
89
 
90
- An overlay checkpoint. This repo contains expert delta weights, router, modeling file, and
91
- config. Inference loads the base model separately and applies this overlay via the
92
- reference runtime.
93
 
94
- ## What this is NOT
95
 
96
- - Not a standalone checkpoint; the frozen base above is required
97
- - Not calibrated for production throughput; shipping-tier MLX 4-bit variants exist for that
98
- - Not claiming to beat the base Qwen on raw MMLU at every scale; the product thesis is
99
- MMLU per GB of RAM, not raw MMLU (see comparison below)
100
 
101
- ## Honest comparison vs base Qwen
102
-
103
- | Metric | Outlier-150B V3.2 | Qwen2.5-72B-Instruct FP16 |
104
- |---|---|---|
105
- | MMLU 5-shot | 84.46% | Qwen published values (see base card) |
106
-
107
- The overlay compresses the expert path to ~1.6 bits per weight via TQ1_0 while preserving
108
- the shared FFN at full precision. This trades some raw accuracy for a smaller expert memory
109
- footprint when paired with int4 base quantization.
110
-
111
- ## Quickstart
112
-
113
- ```python
114
- from transformers import AutoModelForCausalLM, AutoTokenizer
115
- import torch
116
-
117
- base_id = "Qwen/Qwen2.5-72B-Instruct"
118
- overlay_id = "Outlier-Ai/Outlier-150B-V3.2"
119
-
120
- tok = AutoTokenizer.from_pretrained(base_id)
121
- model = AutoModelForCausalLM.from_pretrained(
122
- overlay_id,
123
- trust_remote_code=True,
124
- torch_dtype=torch.float16,
125
- )
126
-
127
- prompt = "Explain ternary mixture-of-experts in one paragraph."
128
- inputs = tok(prompt, return_tensors="pt")
129
- out = model.generate(**inputs, max_new_tokens=256)
130
- print(tok.decode(out[0], skip_special_tokens=True))
131
- ```
132
-
133
- Apple Silicon production use: pick an MLX 4-bit repo from the
134
- [Outlier-Ai organization](https://huggingface.co/Outlier-Ai). Those load via `mlx_lm.generate`.
135
-
136
- ## Benchmarks (verified)
137
-
138
- | Benchmark | Score | n | Harness |
139
- |---|---|---|---|
140
- | MMLU 5-shot | 84.46% plus or minus 0.29% | 14042 | lm-evaluation-harness v0.4.9.1 |
141
- | HellaSwag 10-shot | 77.00% | 14042 | lm-evaluation-harness v0.4.9.1 |
142
- | ARC-Challenge 25-shot | 68.50% | 14042 | lm-evaluation-harness v0.4.9.1 |
143
- | ARC-Easy 25-shot | 90.00% | 14042 | lm-evaluation-harness v0.4.9.1 |
144
- | Winogrande 5-shot | 85.50% | 14042 | lm-evaluation-harness v0.4.9.1 |
145
- | TruthfulQA 0-shot | 69.19% | 14042 | lm-evaluation-harness v0.4.9.1 |
146
-
147
- Evaluation artifacts retained on Outlier infrastructure; ground-truth summary at
148
- [outlier.host](https://outlier.host).
149
-
150
- ## Provenance
151
-
152
- - Harness: lm-evaluation-harness v0.4.9.1
153
- - Sample size: n = 14042
154
- - Config: 5-shot, bf16, batch_size=1
155
- - Date produced: 2026-04-13
156
- - Canonical ground truth: `OUTLIER_GROUND_TRUTH_v13.md`
157
-
158
- ## Related
159
-
160
- - **Production Mac shipping tiers:** [Outlier-Ai organization](https://huggingface.co/Outlier-Ai)
161
- - **Consumer Edition collection:** [Outlier-Ai/outlier-consumer-edition-69e2fb4a0df119ea1747275e](https://huggingface.co/collections/Outlier-Ai/outlier-consumer-edition-69e2fb4a0df119ea1747275e)
162
- - **Research collection:** [Outlier-Ai/outlier-research-69e2fb3a71984614b3c7a279](https://huggingface.co/collections/Outlier-Ai/outlier-research-69e2fb3a71984614b3c7a279)
163
- - **Server V3.2 collection:** [Outlier-Ai/outlier-server-v32-69e2fb4b71984614b3c7a4a3](https://huggingface.co/collections/Outlier-Ai/outlier-server-v32-69e2fb4b71984614b3c7a4a3)
164
- - **Desktop app:** [https://outlier.host](https://outlier.host)
165
- - **Founders lifetime ($200, 500-seat cap):** [https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW](https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW)
166
- - **Discord:** [discord.gg/Hapennmdn9](https://discord.gg/Hapennmdn9)
167
-
168
- ## What is Outlier?
169
-
170
- Outlier is a Mac-native, offline by default AI platform. One desktop app, a curated model library,
171
- Bring-Your-Own-Model support, projects with codebase indexing, a 9-tool coding agent, an
172
- artifacts panel, SQLite memory, and an OpenAI-compatible local API - all Apache 2.0,
173
- all local, no cloud round-trip, no per-token billing.
174
-
175
- - **Desktop app:** [https://outlier.host](https://outlier.host)
176
- - **Founders lifetime (500-seat cap, $200 one-time):** [support Outlier](https://buy.polar.sh/polar_cl_mJfYZsEpEMDcYrgxzvTdnahSeSQNq1UYLqV0l08CUhW)
177
- - **Discord:** [discord.gg/Hapennmdn9](https://discord.gg/Hapennmdn9)
178
- - **Org:** [huggingface.co/Outlier-Ai](https://huggingface.co/Outlier-Ai)
179
-
180
- Built solo in Grand Rapids, Michigan. 19 days to 35+ HF repos and 8,305 downloads, under
181
- $1,200 in compute spend. Mission: a local AI model that codes as well as the cloud flagships,
182
- funded by users who want it - not investors who need exits.
183
-
184
-
185
- ## Known limits
186
-
187
- - Overlay checkpoint; the frozen base above is required.
188
- - Shared FFN runs at full precision on load - plan RAM accordingly if you compose this
189
- without int4 quantization on the base.
190
- - Raw MMLU at 84.46% is compared above against the same-scale Qwen FP16 release;
191
- the product thesis is MMLU per GB of RAM, not raw MMLU.
192
- - English-tuned. Multilingual behavior inherits the base model directly.
193
-
194
- ## Attribution
195
-
196
- Base weights by the **Qwen team at Alibaba**, released under Apache 2.0. Outlier contributes
197
- the ternary MoE overlay, alpha-fix scalars, and training pipeline. All capability credit for
198
- the base is upstream. Qwen team: [qwenlm.github.io](https://qwenlm.github.io/).
199
-
200
- ## Patents and citation
201
-
202
- Architecture, training pipeline, and inference engine covered by US provisional patents
203
- 64/026,886, 64/030,368, and 64/034,028 (Kerr & Company LLC, 2026).
204
-
205
- ```bibtex
206
- @misc{kerr2026outlier150bv3.2,
207
- title={ Outlier-150B V3.2: Ternary Mixture-of-Experts Overlay for Qwen2.5 },
208
- author={ Kerr, Matthew },
209
- year={ 2026 },
210
- howpublished={ \url{ https://huggingface.co/Outlier-Ai/Outlier-150B-V3.2 } },
211
- }
212
- ```
213
-
214
- ## Contact
215
-
216
- Matt Kerr - [outlier.host](https://outlier.host) - [@outlier_ai](https://twitter.com/outlier_ai)
 
64
  value: 84.46
65
  name: MMLU 5-shot
66
  ---
67
+ > **Superseded.** This repo is a research artifact from an earlier Outlier lineage and is no longer the recommended download. The current shipping tier is at **[outlier.host](https://outlier.host/?utm_source=hf&utm_medium=modelcard&utm_campaign=outlier_150b_v3_2)**.
68
 
69
+ # Outlier-150B-V3.2
70
 
71
+ This repo predates the v1.8 Outlier lineup. It is preserved here for reproducibility and historical reference, not as a production recommendation.
 
 
 
72
 
73
+ ## What replaced it
74
 
75
+ Current shipping tiers (see Outlier app v1.8+):
 
 
 
 
 
 
 
76
 
77
+ - [Outlier Nano 4B current entry tier](https://huggingface.co/Outlier-Ai/Outlier-Nano-4B-MLX-4bit)
78
+ - [Outlier Core 27B — current default tier](https://huggingface.co/Outlier-Ai/Outlier-Core-27B-MLX-4bit)
79
+ - [Outlier Vision 35B-A3B — current multimodal tier](https://huggingface.co/Outlier-Ai/Outlier-Vision-35B-A3B-MLX-4bit)
80
+ - [DeepSeek-R1-Distill-Qwen-7B — popular reasoning model](https://huggingface.co/Outlier-Ai/DeepSeek-R1-Distill-Qwen-7B-MLX-4bit)
81
+ - [Qwen3-Coder-30B-A3B — popular coding model](https://huggingface.co/Outlier-Ai/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit)
82
 
83
+ For the latest verified benchmarks and downloads, visit **[outlier.host](https://outlier.host/?utm_source=hf&utm_medium=modelcard&utm_campaign=outlier_150b_v3_2)**.
84
 
85
+ ## Original notes
 
 
86
 
87
+ This was a research / preview artifact. It may contain experimental adapters, overlays, or quantization variants that did not graduate into the shipping product. Treat any technical claims in earlier revisions of this card as provisional.
88
 
89
+ ## License
 
 
 
90
 
91
+ See YAML frontmatter above. Original license terms preserved.