ur-dad-matt commited on
Commit
317db24
·
verified ·
1 Parent(s): ff8d9e7

hf: rewrite org card for v1.8 product story (Day 35)

Browse files
Files changed (1) hide show
  1. README.md +60 -208
README.md CHANGED
@@ -1,228 +1,80 @@
1
  ---
2
- language:
3
- - en
4
- library_name: transformers
5
  license: apache-2.0
6
- pipeline_tag: text-generation
7
  tags:
 
 
 
 
 
 
 
 
8
  - mixture-of-experts
9
  - moe
10
  - ternary
11
  - quantization
12
- - apple-silicon
13
- - mlx
14
- - local-inference
15
- - offline
16
- - mac
17
- base_model:
18
- - Qwen/Qwen2.5-7B-Instruct
19
- - Qwen/Qwen2.5-14B-Instruct
20
- - Qwen/Qwen2.5-32B-Instruct
21
- - Qwen/Qwen2.5-72B-Instruct
22
- - Qwen/Qwen3-1.7B
23
- - Qwen/Qwen3-Coder-30B-A3B-Instruct
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  ---
25
-
26
  # Outlier
27
 
28
- **Local AI for Mac, plus a ternary Mixture-of-Experts research track.**
29
-
30
- Outlier is two things in one org: a Mac-native desktop app that runs the best open-weights models offline, and a research effort training our own ternary MoE language models as overlay deltas on a frozen Qwen2.5 base. Both are Apache 2.0. Both ship here.
31
-
32
- - **App:** one 8.8 MB DMG, five curated shipping tiers, no tokens, no cloud, no account. Free forever for v1.
33
- - **Research:** four MoE scales (10B / 40B / 70B / 150B) built as {-1, 0, +1} overlays on a frozen Qwen base, plus the **alpha-fix** recovery primitive — 280 per-expert scalar gates in a 15 KB overlay that recovered +1.61pp MMLU on 70B where a 68M-parameter LoRA regressed.
34
-
35
- Built solo in 19 days on a Mac Studio M1 Ultra plus spot B200 GPUs. Total compute spend under $1,200. Three U.S. provisional patents filed.
36
-
37
- **Website:** [outlier.host](https://outlier.host) · **Engine:** [github.com/Outlier-host/outlier](https://github.com/Outlier-host/outlier) · **Contact:** [matt@outlier.host](mailto:matt@outlier.host)
38
-
39
- ---
40
-
41
- ## What you can run today
42
-
43
- The desktop app ships five curated tiers. Every tier is a Mac-optimized build of an open-weights base model, bundled in the one 8.8 MB installer.
44
-
45
- | Tier | Base | Quant | RAM | Speed (M1 Ultra) | Use case |
46
- |---|---|---|---|---|---|
47
- | **Nano** | Qwen3 1.7B | MLX 4-bit | < 2 GB | bench pending | Fast drafts, low-battery |
48
- | **Lite** | Qwen 2.5 7B | MLX 4-bit AWQ | 4.47 GB | **71.30 tok/s** | Daily driver, chat, writing |
49
- | **Compact** | Qwen 2.5 14B | MLX 4-bit AWQ | 8.24 GB | **37.26 tok/s** | Reasoning, deeper context |
50
- | **Max** | Qwen 2.5 32B | GGUF Q4 | ~18 GB | bench pending | Long-form, complex tasks |
51
- | **Code** | Qwen3-Coder-30B-A3B | MLX 4-bit | ~16 GB | **55 tok/s** | Agentic coding, repo-scale |
52
-
53
- **Lite / Compact speeds:** `[VERIFIED]` — Mac Studio M1 Ultra 64 GB, mlx_lm, 5-prompt steady-state, 3-prompt warmup, temp 0.7, April 17, 2026. Source: `bench_7b.json` / `bench_14b.json`.
54
-
55
- **Code tier (Outlier-Coder 30B):** Our Mac-optimized build of [Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct) by the Alibaba Qwen team (Apache 2.0). We quantize to MLX 4-bit (16 GB on disk vs. 61 GB FP16), tune sampling defaults for Apple Silicon (top_p=0.8, top_k=20, rep_penalty=1.05), and bundle it into the installer. The base model is the Qwen team's work; credit there.
56
-
57
- **Download the app:** [outlier.host](https://outlier.host) · Apple Silicon · macOS 13+ · 8 GB RAM minimum.
58
-
59
- ---
60
-
61
- ## Quickstart — MLX (Mac)
62
-
63
- ```python
64
- # pip install mlx-lm
65
- from mlx_lm import load, generate
66
-
67
- model, tokenizer = load("Outlier-Ai/Outlier-Lite-7B-MLX-4bit")
68
- prompt = "Explain mixture of experts in one paragraph."
69
- response = generate(model, tokenizer, prompt=prompt, max_tokens=256)
70
- print(response)
71
- ```
72
-
73
- ## Quickstart — transformers (GPU / CUDA)
74
-
75
- ```python
76
- # pip install transformers accelerate
77
- from transformers import AutoModelForCausalLM, AutoTokenizer
78
-
79
- model_name = "Outlier-Ai/Outlier-70B-V3.3" # research-track MoE overlay
80
- tokenizer = AutoTokenizer.from_pretrained(model_name)
81
- model = AutoModelForCausalLM.from_pretrained(
82
- model_name, torch_dtype="auto", device_map="auto", trust_remote_code=True,
83
- )
84
-
85
- messages = [{"role": "user", "content": "Write a quicksort in Rust."}]
86
- inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
87
- outputs = model.generate(inputs, max_new_tokens=512)
88
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
89
- ```
90
-
91
- ---
92
-
93
- ## Research track — ternary MoE overlays
94
-
95
- Our own MoE family is trained as overlay deltas on a frozen Qwen2.5 base. Routed experts are stored in {-1, 0, +1} ternary at ~1.6 bits per weight. A frozen full-precision Qwen base acts as the shared expert. Top-2 routing per MoE layer. This means our repos are **not standalone checkpoints** — they attach to an unmodified base model at load time.
96
 
97
- ### MMLU (primary)
98
 
99
- Every number has provenance. Sample size n = 14,042 (full MMLU). Harness: [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness). 5-shot, bfloat16.
100
 
101
- | Model | MMLU | Stderr | Harness | Status |
102
- |---|---|---|---|---|
103
- | Outlier-150B V3.2 | **84.46%** | 0.29% | v0.4.9.1 | `[VERIFIED]` Day 13 |
104
- | Outlier-70B V3.3 (alpha-fixed) | **83.10%** | 0.30% | v0.4.9.1 | `[VERIFIED]` Day 13 |
105
- | Outlier-40B V3.3 | **77.80%** | 0.33% | v0.4.11 | `[VERIFIED]` Day 12 |
106
- | Outlier-10B V3.3 | **70.87%** | ≈0.38%* | v0.4.9.1 | `[VERIFIED]` Day 13 |
 
 
 
107
 
108
- *10B stderr calculated from binomial formula √(p(1-p)/n); matches pattern of reported values at other scales.
109
 
110
- **Harness caveat:** v0.4.9.1 → v0.4.11 produced a 1.30pp delta on the same 150B weights. We've locked v0.4.9.1 as our reference harness and document both numbers in our ground-truth file so reviewers can reproduce either.
111
 
112
- ### Secondary benchmarks (V3.3, verified)
113
 
114
- | Model | HellaSwag | ARC-C | ARC-E | Winogrande | TruthfulQA |
115
- |---|---|---|---|---|---|
116
- | 150B | 77.00% | 68.50% | 90.00% | 85.50% | 69.19% |
117
- | 70B | 85.95% | 73.46% | 91.62% | 81.29% | 67.12% |
118
- | 40B | 84.64% | 73.12% | 91.29% | 80.98% | 67.49% |
119
- | 10B | 78.30% | 62.88% | 85.98% | 73.80% | 62.11% |
120
 
121
- All `[VERIFIED]` at n = 14,042, v0.4.9.1.
122
-
123
- ### MMLU vs. base Qwen (radical honesty)
124
-
125
- | Scale | Outlier V3.3 | Base Qwen FP16 | Delta |
126
- |---|---|---|---|
127
- | 10B | 70.87% | Qwen 7B: 74.2% | **−3.33pp** |
128
- | 40B | 77.80% | Qwen 14B: 79.7% | **−1.88pp** |
129
- | 70B | 83.10% | Qwen 32B: 83.3% | **−0.20pp (tied)** |
130
- | 150B | 84.46% | Qwen 72B: 86.1% | **−1.64pp** |
131
-
132
- Our MoE overlays trail base Qwen by 0.2–3.3pp on raw MMLU. That's the honest number. The defensible story is **MMLU per GB of RAM** at the slot our 70B occupies (≈20 GB / 83% MMLU), not raw MMLU.
133
-
134
- For additional reference: Llama 3.1 70B full-sample MMLU is around 83.1%. Outlier-70B V3.3 alpha-fixed lands in that neighborhood, on a model family trained solo for under $1,200 of total compute.
135
-
136
- ### Model naming — V3.3 convention
137
-
138
- The old `10B / 40B / 70B / 150B` labels counted routed-expert parameters and understated real model sizes. V3.3 moves to the industry-standard `TotalB-AyyB` convention (DeepSeek, Mixtral, Llama 4):
139
-
140
- | Old name | V3.3 name | Total params | Active params |
141
- |---|---|---|---|
142
- | Outlier-10B | Outlier-13B-A7B | 13B | 7B |
143
- | Outlier-40B | Outlier-30B-A14B | 30B | 14B |
144
- | Outlier-70B | Outlier-68B-A32B | 68B | 32B |
145
- | Outlier-150B | Outlier-150B-A70B | 150B | 70B |
146
-
147
- V3.2 repos remain available as `[SUPERSEDED]`, pointing to V3.3.
148
-
149
- ---
150
-
151
- ## Engine
152
-
153
- - **Open-source at [github.com/Outlier-host/outlier](https://github.com/Outlier-host/outlier)** (Apache 2.0). Ternary MoE loader, three-tier paged cache, MPS + CPU backends, lm-eval compatible, alpha-overlay loader for post-training recovery.
154
- - **GPU-resident expert dequantization.** A patched modeling file materializes ternary experts to bf16 at load time — ~56× speedup over the original CPU→GPU path on a single B200.
155
- - **Alpha-fix technique.** 280 per-expert scalar gates trained in 18 minutes on one B200 recovered +1.61pp MMLU on 70B (81.49% → 83.10%). Overlay file is 15 KB — roughly 250,000× fewer trainable parameters than the LoRA approach it outperformed.
156
- - **Desktop app (v1.3.0).** Tauri + FastAPI + mlx_lm. 8.8 MB DMG, Apple Silicon, macOS 13+. SHA-256: `1837df5739eda279a564a2ef8fc33a366d9e018900e98f391bfbbf6b9408448b`. Ad-hoc signed (Apple Dev ID pending).
157
-
158
- ---
159
-
160
- ## What we're not claiming
161
-
162
- - We do not match Kimi K2.5, GLM-5, Claude Opus 4.6, Gemini 3 Pro, or GPT-5 on pure MMLU.
163
- - We are not the first ternary MoE. Microsoft + Apple's MoTE (arXiv:2506.14435, June 2025) published a shared-FP + ternary-expert architecture for vision-language models. Our contribution is the text-LLM variant, the overlay-on-frozen-base deployment artifact, and the alpha-fix recovery primitive.
164
- - We are not shipping models trained on trillions of tokens. Our distillation pipeline uses DeepSeek V3 as teacher and touches a fraction of the tokens a Llama-class pretrain does. The comparison we care about is **quality per dollar of training**, not parameter count or token count.
165
- - **Production-ready local inference lives in the shipping tiers (curated Qwen), not yet in the MoE research repos.** Our 70B V3.3 runs on a Mac Studio via the engine, but the app ships Qwen tiers for now. MoE graduates to the app when a scale validates against a real Mac RAM tier.
166
-
167
- ---
168
-
169
- ## Status — Day 19 (April 19, 2026)
170
-
171
- - **v1.3.0 desktop app:** built and shipping (ad-hoc signed, Apple Dev ID in queue)
172
- - **V3.3 MoE models:** four scales verified, uploaded
173
- - **Shipping tiers:** Nano / Lite / Compact / Max / Code — AWQ builds live
174
- - **Outlier-Coder 30B:** Mac build of Qwen3-Coder-30B-A3B shipped in app
175
- - **Website:** [outlier.host](https://outlier.host) live
176
- - **Downloads:** ~5,881 across 15 repos (CLAIM — live re-verify pending)
177
- - **Patents:** 3 U.S. provisional filed (61 claims), non-provisional deadline April 3–9, 2027
178
-
179
- ---
180
-
181
- ## Links
182
-
183
- - **Website:** [outlier.host](https://outlier.host)
184
- - **Engine:** [github.com/Outlier-host/outlier](https://github.com/Outlier-host/outlier)
185
- - **Paper:** `outlier_ternary_moe_2026.pdf` v6 (landing with the launch)
186
- - **Responsible use reports:** [abuse@outlier.host](mailto:abuse@outlier.host)
187
- - **Contact:** [matt@outlier.host](mailto:matt@outlier.host)
188
- - **Built by:** [Matt Kerr](https://linkedin.com/in/mattkerr09) · Kerr & Company LLC · Grand Rapids, MI
189
-
190
- ---
191
-
192
- ## License & attribution
193
-
194
- - **Engine & app:** Apache 2.0
195
- - **MoE overlays:** Apache 2.0, overlay on frozen Qwen2.5 bases by the Alibaba Qwen team (Apache 2.0)
196
- - **Outlier-Coder 30B:** Mac-optimized build of Qwen3-Coder-30B-A3B-Instruct by the Alibaba Qwen team, licensed Apache 2.0. Base weights are the Qwen team's work; modifications are ours.
197
- - **Shipping tiers (Nano / Lite / Compact / Max):** Mac-optimized builds of Qwen 2.5 / Qwen 3 bases, all Apache 2.0.
198
-
199
- **Patents** — defensive, not enforced against open-source implementations:
200
- - #64/026,886 (April 3, 2026) — Ternary-Quantized MoE Language Model System
201
- - #64/030,368 (April 6, 2026) — Systems and Methods for Ternary MoE Inference, Training, and Modular Deployment
202
- - #64/034,028 (April 9, 2026) — Zero-Delta Expert Initialization, Residual Error Correction, and Adaptive Inference
203
-
204
- ## Citation
205
-
206
- ```bibtex
207
- @misc{kerr2026outlier,
208
- title = {Outlier: Ternary Mixture-of-Experts Language Models on Consumer Hardware},
209
- author = {Kerr, Matt},
210
- year = {2026},
211
- howpublished = {\url{https://outlier.host}},
212
- note = {Outlier-Ai org, Hugging Face}
213
- }
214
- ```
215
-
216
- ---
217
 
218
- ## Changelog
219
 
220
- - **April 19, 2026 (Day 19):** Platform-first thesis locked. Org card rewritten to reflect dual-track (Platform + MoE research). Five shipping tiers canonical: Nano / Lite / Compact / Max / Code. Outlier-Coder 30B (Mac build of Qwen3-Coder-30B-A3B) added as new Code-tier flagship. Changelog catches up through Day 19.
221
- - **April 18, 2026 (Day 18):** v1.3.0 desktop DMG built (8.8 MB, Tauri, ad-hoc signed). Free-forever-v1 pricing decision made — subscription deferred until 1,000+ free downloads + 30-day retention data. Strategic pivot from "best model company" to "Apple of local AI" platform.
222
- - **April 17, 2026 (Day 17):** Mac AWQ shipping tiers verified. Lite (7B) hits 71.30 tok/s / 4.47 GB peak RSS; Compact (14B) hits 37.26 tok/s / 8.24 GB. Canonical Mac numbers. Exp 5 alpha validation loader fixed.
223
- - **April 16, 2026 (Day 16):** 10B V3.3 secondary benchmarks verified. Real backend wired end-to-end through web UI. Website deployed at outlier.host.
224
- - **April 13, 2026 (Day 13):** Five cluster wins. 70B V3.3 alpha-fixed at 83.10% MMLU (+1.61pp). 150B V3.2 re-measured at 84.46% MMLU on v0.4.9.1 (supersedes the earlier 83.16% value). YaRN 4x validated for 128K context on 70B / 150B. V4 HESTIA+LoRA killed after −1.17pp / −1.34pp regressions; fully recovered via 280-scalar alpha-fix on a 15 KB overlay.
225
- - **April 11, 2026 (Day 11):** Removed an unverified four-row MMLU table that relied on a decommissioned cluster's unsaved source files. Forensic cleanup led directly to the Day 9 provenance rules that now govern every number on this card.
226
- - **April 9, 2026 (Day 9):** Third provisional patent filed (#64/034,028). Provenance discipline rules 66–78 added to project canon after trust-laundering incident.
227
- - **April 6, 2026 (Day 6):** Second provisional patent filed (#64/030,368).
228
- - **April 3, 2026 (Day 3):** First provisional patent filed (#64/026,886). Outlier project formally begins.
 
1
  ---
 
 
 
2
  license: apache-2.0
 
3
  tags:
4
+ - mlx
5
+ - apple-silicon
6
+ - mac
7
+ - macos
8
+ - local-llm
9
+ - on-device
10
+ - offline
11
+ - private
12
  - mixture-of-experts
13
  - moe
14
  - ternary
15
  - quantization
16
+ - 4-bit
17
+ - quantized
18
+ - text-generation
19
+ - multimodal
20
+ - vision
21
+ - image-to-text
22
+ - code-generation
23
+ - reasoning
24
+ - thinking
25
+ - function-calling
26
+ - tool-use
27
+ - m1
28
+ - m2
29
+ - m3
30
+ - m4
31
+ - metal
32
+ - mlx-lm
33
+ - llama-cpp
34
+ - ollama
35
+ - outlier
36
+ - outlier-app
37
+ - llm
38
+ - open-source
39
+ - apache-2-0
40
+ - macbook-pro
41
+ - macbook-air
42
+ - mac-studio
43
+ - mac-mini
44
+ - apple-intelligence
45
+ - chatgpt-alternative
46
+ - claude-alternative
47
+ - private-ai
48
+ - secure-ai
49
  ---
 
50
  # Outlier
51
 
52
+ Local AI on your Mac. No API token caps from Outlier. Free Nano + Lite, $20/mo Pro adds the full stack, $200 once for lifetime Founders.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
54
+ [outlier.host](https://outlier.host?utm_source=hf&utm_medium=org_card&utm_campaign=org_card_day35) · [Download Mac app](https://github.com/Outlier-host/outlier-app-releases/releases/latest/download/Outlier.dmg)
55
 
56
+ ## Shipping tiers (v1.8)
57
 
58
+ | Tier | Plan | Disk | Min RAM |
59
+ |------|------|------|---------|
60
+ | Outlier Nano (Qwen3.5-4B, MLX 4-bit) | Free | 2.37 GB | 6 GB |
61
+ | Outlier Lite (Qwen3.5-9B, MLX 4-bit) | Free | 5.04 GB | 12 GB |
62
+ | Outlier Quick (Gemma-4-26B-A4B) | Pro | 15.61 GB | 16 GB |
63
+ | Outlier Core (Qwen3.6-27B) | Pro | 15.13 GB | 24 GB |
64
+ | Outlier Code (Core weights, code config) | Pro | 15.13 GB | 24 GB |
65
+ | Outlier Plus (Qwen3.5-397B-A17B, paged) | Pro | 209 GB | 32 GB (64 GB recommended) |
66
+ | Outlier Vision (Qwen3.6-35B-A3B) | Pro | 19.0 GB | 24 GB |
67
 
68
+ Verified benchmarks live on outlier.host (with source files, exact commands, sample size, and standard error per measurement).
69
 
70
+ ## Patents
71
 
72
+ Three U.S. provisional applications filed April 2026, covering ternary Mixture-of-Experts inference, paged expert routing on consumer hardware, and zero-delta initialization with residual error correction. Total filing cost $195. 12-month conversion clock per application.
73
 
74
+ ## Research artifacts
 
 
 
 
 
75
 
76
+ The V3.2 family (10B / 40B / 70B / 150B) on this org are research artifacts from the training pipeline that shipped in v1.8. They are not the shipping product — the shipping product is the Mac app at outlier.host. The earlier Outlier-10B, Outlier-10B-V2, and Outlier-40B repos are preserved for continuity and download-history value.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
 
78
+ ## Built by
79
 
80
+ Matt Kerr · Kerr & Company LLC · Grand Rapids, MI · matt@outlier.host