Text Generation
Transformers
outlier_150b_rexmoe
mixture-of-experts
Mixture of Experts
ternary
quantized
qwen2.5
outlier
local-llm
on-device
edge-ai
energy-efficient
sparse
overlay
research
apple-silicon
mac
mmlu-verified
custom_code
Eval Results (legacy)
Instructions to use Outlier-Ai/Outlier-150B-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Outlier-Ai/Outlier-150B-V3.2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Outlier-Ai/Outlier-150B-V3.2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Outlier-Ai/Outlier-150B-V3.2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
- SGLang
How to use Outlier-Ai/Outlier-150B-V3.2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Outlier-Ai/Outlier-150B-V3.2 with Docker Model Runner:
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
Upload checkpoints/alpha.json with huggingface_hub
Browse files- checkpoints/alpha.json +90 -0
checkpoints/alpha.json
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"group_01_expert_00": 0.013282203115522861,
|
| 3 |
+
"group_01_expert_01": 0.007036757655441761,
|
| 4 |
+
"group_01_expert_02": 0.022870026528835297,
|
| 5 |
+
"group_01_expert_03": 0.013156086206436157,
|
| 6 |
+
"group_01_expert_04": 0.015288042835891247,
|
| 7 |
+
"group_01_expert_05": 0.005860608536750078,
|
| 8 |
+
"group_01_expert_06": 0.02469489350914955,
|
| 9 |
+
"group_01_expert_07": 0.016237638890743256,
|
| 10 |
+
"group_02_expert_00": 0.018590012565255165,
|
| 11 |
+
"group_02_expert_01": 0.005804466549307108,
|
| 12 |
+
"group_02_expert_02": 0.02403106354176998,
|
| 13 |
+
"group_02_expert_03": 0.007332759443670511,
|
| 14 |
+
"group_02_expert_04": 0.0130878034979105,
|
| 15 |
+
"group_02_expert_05": 0.015886439010500908,
|
| 16 |
+
"group_02_expert_06": 0.0069729238748550415,
|
| 17 |
+
"group_02_expert_07": 0.019574716687202454,
|
| 18 |
+
"group_03_expert_00": 0.010419095866382122,
|
| 19 |
+
"group_03_expert_01": 0.005606003105640411,
|
| 20 |
+
"group_03_expert_02": 0.008018190972507,
|
| 21 |
+
"group_03_expert_03": 0.003162325592711568,
|
| 22 |
+
"group_03_expert_04": 0.006889252923429012,
|
| 23 |
+
"group_03_expert_05": 0.010474657639861107,
|
| 24 |
+
"group_03_expert_06": 0.0032552790362387896,
|
| 25 |
+
"group_03_expert_07": 0.006326879840344191,
|
| 26 |
+
"group_04_expert_00": 0.0012291143648326397,
|
| 27 |
+
"group_04_expert_01": 0.00388744892552495,
|
| 28 |
+
"group_04_expert_02": 0.004821837414056063,
|
| 29 |
+
"group_04_expert_03": 0.003917478024959564,
|
| 30 |
+
"group_04_expert_04": 0.014478147961199284,
|
| 31 |
+
"group_04_expert_05": 0.005330056417733431,
|
| 32 |
+
"group_04_expert_06": 0.006041828542947769,
|
| 33 |
+
"group_04_expert_07": 0.005014166701585054,
|
| 34 |
+
"group_05_expert_00": 0.006031031254678965,
|
| 35 |
+
"group_05_expert_01": 0.0034807720221579075,
|
| 36 |
+
"group_05_expert_02": 0.003914565779268742,
|
| 37 |
+
"group_05_expert_03": 0.006052051670849323,
|
| 38 |
+
"group_05_expert_04": 0.004927619826048613,
|
| 39 |
+
"group_05_expert_05": 0.0028023945633322,
|
| 40 |
+
"group_05_expert_06": 0.002118324162438512,
|
| 41 |
+
"group_05_expert_07": 0.007412611041218042,
|
| 42 |
+
"group_06_expert_00": 0.003685481147840619,
|
| 43 |
+
"group_06_expert_01": 0.007255200762301683,
|
| 44 |
+
"group_06_expert_02": 0.0038398539181798697,
|
| 45 |
+
"group_06_expert_03": 0.004417807329446077,
|
| 46 |
+
"group_06_expert_04": 0.0025816322304308414,
|
| 47 |
+
"group_06_expert_05": 0.0033090233337134123,
|
| 48 |
+
"group_06_expert_06": 0.005978161934763193,
|
| 49 |
+
"group_06_expert_07": 0.006351307965815067,
|
| 50 |
+
"group_07_expert_00": 0.002889202209189534,
|
| 51 |
+
"group_07_expert_01": 0.0024275293108075857,
|
| 52 |
+
"group_07_expert_02": 0.0031850712839514017,
|
| 53 |
+
"group_07_expert_03": 0.0038473173044621944,
|
| 54 |
+
"group_07_expert_04": 0.004140378441661596,
|
| 55 |
+
"group_07_expert_05": 0.004112926311790943,
|
| 56 |
+
"group_07_expert_06": 0.005547539331018925,
|
| 57 |
+
"group_07_expert_07": 0.003908976446837187,
|
| 58 |
+
"group_08_expert_00": 0.002477911999449134,
|
| 59 |
+
"group_08_expert_01": 0.004909783136099577,
|
| 60 |
+
"group_08_expert_02": 0.002660754369571805,
|
| 61 |
+
"group_08_expert_03": 0.003642808413133025,
|
| 62 |
+
"group_08_expert_04": 0.0037152792792767286,
|
| 63 |
+
"group_08_expert_05": 0.006980478297919035,
|
| 64 |
+
"group_08_expert_06": 0.002834885846823454,
|
| 65 |
+
"group_08_expert_07": 0.005522340070456266,
|
| 66 |
+
"group_09_expert_00": 0.004305507522076368,
|
| 67 |
+
"group_09_expert_01": 0.005085402633994818,
|
| 68 |
+
"group_09_expert_02": 0.0030334556940943003,
|
| 69 |
+
"group_09_expert_03": 0.0053315781988203526,
|
| 70 |
+
"group_09_expert_04": 0.004353753291070461,
|
| 71 |
+
"group_09_expert_05": 0.006517808418720961,
|
| 72 |
+
"group_09_expert_06": 0.005120622925460339,
|
| 73 |
+
"group_09_expert_07": 0.007203881163150072,
|
| 74 |
+
"group_10_expert_00": 0.0071843089535832405,
|
| 75 |
+
"group_10_expert_01": 0.006992374546825886,
|
| 76 |
+
"group_10_expert_02": 0.0066714719869196415,
|
| 77 |
+
"group_10_expert_03": 0.0071832784451544285,
|
| 78 |
+
"group_10_expert_04": 0.010271300561726093,
|
| 79 |
+
"group_10_expert_05": 0.005207856651395559,
|
| 80 |
+
"group_10_expert_06": 0.005580703262239695,
|
| 81 |
+
"group_10_expert_07": 0.007606270257383585,
|
| 82 |
+
"group_11_expert_00": 0.003223187057301402,
|
| 83 |
+
"group_11_expert_01": 0.005411272402852774,
|
| 84 |
+
"group_11_expert_02": 0.002807617885991931,
|
| 85 |
+
"group_11_expert_03": 0.004755697213113308,
|
| 86 |
+
"group_11_expert_04": 0.008131218142807484,
|
| 87 |
+
"group_11_expert_05": 0.0035522400867193937,
|
| 88 |
+
"group_11_expert_06": 0.005941495299339294,
|
| 89 |
+
"group_11_expert_07": 0.005118831992149353
|
| 90 |
+
}
|