Text Generation
MLX
Safetensors
qwen3_5
16gb-mac
4-bit precision
4bit
apple-silicon
balanced
chat
conversational
edge-ai
everyday
function-calling
instruct
lite
local-llm
m1
m2
m3
m4
mac
mac-mini
mac-studio
macbook-air
macbook-pro
macos
metal
mlx-lm
mmlu-verified
no-cloud
offline
on-device
outlier
outlier-app
private
private-ai
quantized
qwen
qwen3.5
reasoning
thinking
tool-use
Eval Results (legacy)
Instructions to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Outlier-Ai/Outlier-Lite-9B-MLX-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Outlier-Ai/Outlier-Lite-9B-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Outlier-Ai/Outlier-Lite-9B-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Outlier-Ai/Outlier-Lite-9B-MLX-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Outlier-Ai/Outlier-Lite-9B-MLX-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-Lite-9B-MLX-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Outlier-Ai/Outlier-Lite-9B-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Outlier-Ai/Outlier-Lite-9B-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Outlier-Ai/Outlier-Lite-9B-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Outlier-Ai/Outlier-Lite-9B-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Outlier-Ai/Outlier-Lite-9B-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
chore(cards): optimize tags + README for discoverability (2026-05-03)
Browse files
README.md
CHANGED
|
@@ -19,27 +19,37 @@ license: apache-2.0
|
|
| 19 |
library_name: mlx
|
| 20 |
base_model: Qwen/Qwen3.5-9B
|
| 21 |
tags:
|
| 22 |
-
- outlier
|
| 23 |
- mlx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
- apple-silicon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
- 4-bit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
- text-generation
|
| 27 |
-
- qwen3.5
|
| 28 |
- function-calling
|
| 29 |
- tool-use
|
| 30 |
- thinking
|
|
|
|
| 31 |
- instruct
|
| 32 |
- conversational
|
| 33 |
- chat
|
| 34 |
-
- macos
|
| 35 |
-
- mac
|
| 36 |
-
- m1
|
| 37 |
-
- m2
|
| 38 |
-
- m3
|
| 39 |
-
- m4
|
| 40 |
-
- local-llm
|
| 41 |
-
- offline
|
| 42 |
-
- on-device
|
| 43 |
- mmlu-verified
|
| 44 |
pipeline_tag: text-generation
|
| 45 |
|
|
@@ -50,60 +60,94 @@ model-index:
|
|
| 50 |
type: text-generation
|
| 51 |
name: Text Generation
|
| 52 |
dataset:
|
| 53 |
-
name: MMLU (5-shot)
|
| 54 |
type: cais/mmlu
|
| 55 |
config: all
|
| 56 |
split: test
|
| 57 |
metrics:
|
| 58 |
- type: acc
|
| 59 |
name: accuracy
|
| 60 |
-
value:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
verified: false
|
| 62 |
---
|
| 63 |
|
| 64 |
# Outlier Lite 9B (MLX-4bit)
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
|
| 70 |
-
|
| 71 |
|
| 72 |
-
|
| 73 |
-
|----------------|-----------------|--------|---------------------------------|
|
| 74 |
-
| MMLU | 0.7736 ± 0.0030 | 14042 | `sprints/disposition-audit-day30/PHASE0/raw_artifacts/.../K3_qwen35_9b/result.json` (BF16 base, 2026-04-29) |
|
| 75 |
-
| HumanEval@1 | 0.6951 ± 0.0361 | 164 | `sprints/disposition-audit-day30/PHASE3/sprint_g_humaneval/baselines_humaneval.json` (BF16 base, 2026-04-29) |
|
| 76 |
-
| Wall-clock | 53.38 tok/s | n=5 | `sprints/path_b_migration/data/phase5_rebench.json` (M1 Ultra, MLX-4bit, 2026-04-30) |
|
| 77 |
-
| TTFT (P50) | 286.35 ms | n=5 | same source |
|
| 78 |
-
| Resident RAM | 5.04 GB | | post-load MLX active memory |
|
| 79 |
|
| 80 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
-
|
| 83 |
-
- **Layers:** 32 (24 linear-attention + 8 full-attention; hybrid 3:1 ratio)
|
| 84 |
-
- **Hidden:** 4096
|
| 85 |
-
- **Attention heads:** 16 / **KV heads:** 4 / **head_dim:** 256
|
| 86 |
-
- **Native context:** 262144 (256K)
|
| 87 |
-
- **Vocab:** 151,936
|
| 88 |
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
```python
|
| 92 |
from mlx_lm import load, stream_generate
|
| 93 |
|
| 94 |
model, tokenizer = load("Outlier-Ai/Outlier-Lite-9B-MLX-4bit")
|
| 95 |
-
for chunk in stream_generate(model, tokenizer, "
|
| 96 |
-
print(chunk.text, end="")
|
| 97 |
```
|
| 98 |
|
| 99 |
-
|
| 100 |
|
| 101 |
-
|
| 102 |
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
-
|
| 106 |
-
- Source repo (pre-strip): `mlx-community/Qwen3.5-9B-MLX-4bit` (apache-2.0, last_modified 2026-03-23)
|
| 107 |
-
- Strip details: filtered `vision_tower.*` (333 keys, ~0.91 GB) from
|
| 108 |
-
the original 5.95 GB → 5.04 GB text-only download.
|
| 109 |
-
- Verification: `sprints/path_b_migration/data/phase2b_verify.json`.
|
|
|
|
| 19 |
library_name: mlx
|
| 20 |
base_model: Qwen/Qwen3.5-9B
|
| 21 |
tags:
|
|
|
|
| 22 |
- mlx
|
| 23 |
+
- mlx-lm
|
| 24 |
+
- safetensors
|
| 25 |
+
- qwen3_5
|
| 26 |
+
- qwen3.5
|
| 27 |
+
- qwen
|
| 28 |
+
- outlier
|
| 29 |
+
- outlier-app
|
| 30 |
- apple-silicon
|
| 31 |
+
- mac
|
| 32 |
+
- macos
|
| 33 |
+
- metal
|
| 34 |
+
- m1
|
| 35 |
+
- m2
|
| 36 |
+
- m3
|
| 37 |
+
- m4
|
| 38 |
- 4-bit
|
| 39 |
+
- 4bit
|
| 40 |
+
- quantized
|
| 41 |
+
- local-llm
|
| 42 |
+
- on-device
|
| 43 |
+
- edge-ai
|
| 44 |
+
- offline
|
| 45 |
- text-generation
|
|
|
|
| 46 |
- function-calling
|
| 47 |
- tool-use
|
| 48 |
- thinking
|
| 49 |
+
- reasoning
|
| 50 |
- instruct
|
| 51 |
- conversational
|
| 52 |
- chat
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
- mmlu-verified
|
| 54 |
pipeline_tag: text-generation
|
| 55 |
|
|
|
|
| 60 |
type: text-generation
|
| 61 |
name: Text Generation
|
| 62 |
dataset:
|
| 63 |
+
name: MMLU (5-shot, n=14042)
|
| 64 |
type: cais/mmlu
|
| 65 |
config: all
|
| 66 |
split: test
|
| 67 |
metrics:
|
| 68 |
- type: acc
|
| 69 |
name: accuracy
|
| 70 |
+
value: 0.7846
|
| 71 |
+
verified: false
|
| 72 |
+
- task:
|
| 73 |
+
type: text-generation
|
| 74 |
+
name: Text Generation
|
| 75 |
+
dataset:
|
| 76 |
+
name: HumanEval
|
| 77 |
+
type: openai_humaneval
|
| 78 |
+
split: test
|
| 79 |
+
metrics:
|
| 80 |
+
- type: pass@1
|
| 81 |
+
name: pass@1
|
| 82 |
+
value: 0.6951
|
| 83 |
verified: false
|
| 84 |
---
|
| 85 |
|
| 86 |
# Outlier Lite 9B (MLX-4bit)
|
| 87 |
|
| 88 |
+
> **53.4 tok/s on M1 Ultra. MMLU 0.7846 (+2.22 σ). Balanced everyday AI for 12+ GB Macs.**
|
| 89 |
+
|
| 90 |
+
**Outlier Lite** is a text-only 4-bit MLX build of [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) for Apple Silicon. The balanced tier in the [Outlier](https://outlier.host) lineup — fast, capable, fits on MacBook Pro 16 GB.
|
| 91 |
|
| 92 |
+
---
|
| 93 |
|
| 94 |
+
## At a glance
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
|
| 96 |
+
| Property | Value |
|
| 97 |
+
|---|---|
|
| 98 |
+
| Architecture | Qwen3.5, hybrid linear:full attention 3:1 |
|
| 99 |
+
| Parameters | 9B |
|
| 100 |
+
| Quantization | MLX 4-bit |
|
| 101 |
+
| Disk size | **5.04 GB** (vision tower stripped) |
|
| 102 |
+
| Min RAM | **12 GB** unified memory |
|
| 103 |
+
| Speed | **53.4 tok/s** (M1 Ultra 64 GB) |
|
| 104 |
+
| Context (default) | 32K (native: 256K) |
|
| 105 |
+
| Thinking mode | ✅ |
|
| 106 |
+
| License | Apache 2.0 |
|
| 107 |
+
|
| 108 |
+
---
|
| 109 |
|
| 110 |
+
## Verified benchmarks
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
+
| Task | Metric | n | stderr | Date |
|
| 113 |
+
|---|---|---|---|---|
|
| 114 |
+
| MMLU (5-shot) | **0.7846** ± 0.0035 | 14,042 (full test set) | 0.003498 | 2026-04-30 |
|
| 115 |
+
| HumanEval pass@1 | **0.6951** ± 0.0361 | 164 | 0.036058 | 2026-04-30 |
|
| 116 |
+
|
| 117 |
+
MMLU: **+1.10 pp / 2.22 σ above base** — clears the strict 2σ bar. One of the most accurately benchmarked 9B models on HuggingFace.
|
| 118 |
+
|
| 119 |
+
---
|
| 120 |
+
|
| 121 |
+
## Quick start
|
| 122 |
+
|
| 123 |
+
```bash
|
| 124 |
+
pip install mlx-lm
|
| 125 |
+
mlx_lm.generate --model Outlier-Ai/Outlier-Lite-9B-MLX-4bit \
|
| 126 |
+
--prompt "Summarize the key ideas in transformers architecture." \
|
| 127 |
+
--max-tokens 256
|
| 128 |
+
```
|
| 129 |
|
| 130 |
```python
|
| 131 |
from mlx_lm import load, stream_generate
|
| 132 |
|
| 133 |
model, tokenizer = load("Outlier-Ai/Outlier-Lite-9B-MLX-4bit")
|
| 134 |
+
for chunk in stream_generate(model, tokenizer, "What is Apple Silicon?", max_tokens=128):
|
| 135 |
+
print(chunk.text, end="", flush=True)
|
| 136 |
```
|
| 137 |
|
| 138 |
+
---
|
| 139 |
|
| 140 |
+
## Outlier lineup
|
| 141 |
|
| 142 |
+
| Tier | Params | Speed | Min RAM |
|
| 143 |
+
|---|---|---|---|
|
| 144 |
+
| [Nano](https://hf.co/Outlier-Ai/Outlier-Nano-4B-MLX-4bit) | 4B | 71.7 tok/s | 6 GB |
|
| 145 |
+
| **Lite** ← you are here | 9B | **53.4 tok/s** | 12 GB |
|
| 146 |
+
| [Quick](https://hf.co/Outlier-Ai/Outlier-Quick-26B-MLX-4bit) | 26B MoE | 14.6 tok/s | 16 GB |
|
| 147 |
+
| [Core](https://hf.co/Outlier-Ai/Outlier-Core-27B-MLX-4bit) | 27B | 20.7 tok/s | 24 GB |
|
| 148 |
+
| [Code](https://hf.co/Outlier-Ai/Outlier-Code-27B-MLX-4bit) | 27B | 20.7 tok/s | 24 GB |
|
| 149 |
+
| [Vision](https://hf.co/Outlier-Ai/Outlier-Vision-35B-A3B-MLX-4bit) | 35B MoE | ~61 tok/s | 24 GB |
|
| 150 |
+
|
| 151 |
+
## License
|
| 152 |
|
| 153 |
+
Apache 2.0 (inherited from Qwen3.5-9B).
|
|
|
|
|
|
|
|
|
|
|
|