Text Generation
Transformers
outlier_150b_rexmoe
mixture-of-experts
Mixture of Experts
ternary
quantized
qwen2.5
outlier
local-llm
on-device
edge-ai
energy-efficient
sparse
overlay
research
apple-silicon
mac
mmlu-verified
custom_code
Eval Results (legacy)
Instructions to use Outlier-Ai/Outlier-150B-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Outlier-Ai/Outlier-150B-V3.2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Outlier-Ai/Outlier-150B-V3.2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Outlier-Ai/Outlier-150B-V3.2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
- SGLang
How to use Outlier-Ai/Outlier-150B-V3.2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Outlier-Ai/Outlier-150B-V3.2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Outlier-Ai/Outlier-150B-V3.2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Outlier-Ai/Outlier-150B-V3.2 with Docker Model Runner:
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
File size: 4,057 Bytes
af6d780 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | {
"group_01_expert_00": 0.013282203115522861,
"group_01_expert_01": 0.007036757655441761,
"group_01_expert_02": 0.022870026528835297,
"group_01_expert_03": 0.013156086206436157,
"group_01_expert_04": 0.015288042835891247,
"group_01_expert_05": 0.005860608536750078,
"group_01_expert_06": 0.02469489350914955,
"group_01_expert_07": 0.016237638890743256,
"group_02_expert_00": 0.018590012565255165,
"group_02_expert_01": 0.005804466549307108,
"group_02_expert_02": 0.02403106354176998,
"group_02_expert_03": 0.007332759443670511,
"group_02_expert_04": 0.0130878034979105,
"group_02_expert_05": 0.015886439010500908,
"group_02_expert_06": 0.0069729238748550415,
"group_02_expert_07": 0.019574716687202454,
"group_03_expert_00": 0.010419095866382122,
"group_03_expert_01": 0.005606003105640411,
"group_03_expert_02": 0.008018190972507,
"group_03_expert_03": 0.003162325592711568,
"group_03_expert_04": 0.006889252923429012,
"group_03_expert_05": 0.010474657639861107,
"group_03_expert_06": 0.0032552790362387896,
"group_03_expert_07": 0.006326879840344191,
"group_04_expert_00": 0.0012291143648326397,
"group_04_expert_01": 0.00388744892552495,
"group_04_expert_02": 0.004821837414056063,
"group_04_expert_03": 0.003917478024959564,
"group_04_expert_04": 0.014478147961199284,
"group_04_expert_05": 0.005330056417733431,
"group_04_expert_06": 0.006041828542947769,
"group_04_expert_07": 0.005014166701585054,
"group_05_expert_00": 0.006031031254678965,
"group_05_expert_01": 0.0034807720221579075,
"group_05_expert_02": 0.003914565779268742,
"group_05_expert_03": 0.006052051670849323,
"group_05_expert_04": 0.004927619826048613,
"group_05_expert_05": 0.0028023945633322,
"group_05_expert_06": 0.002118324162438512,
"group_05_expert_07": 0.007412611041218042,
"group_06_expert_00": 0.003685481147840619,
"group_06_expert_01": 0.007255200762301683,
"group_06_expert_02": 0.0038398539181798697,
"group_06_expert_03": 0.004417807329446077,
"group_06_expert_04": 0.0025816322304308414,
"group_06_expert_05": 0.0033090233337134123,
"group_06_expert_06": 0.005978161934763193,
"group_06_expert_07": 0.006351307965815067,
"group_07_expert_00": 0.002889202209189534,
"group_07_expert_01": 0.0024275293108075857,
"group_07_expert_02": 0.0031850712839514017,
"group_07_expert_03": 0.0038473173044621944,
"group_07_expert_04": 0.004140378441661596,
"group_07_expert_05": 0.004112926311790943,
"group_07_expert_06": 0.005547539331018925,
"group_07_expert_07": 0.003908976446837187,
"group_08_expert_00": 0.002477911999449134,
"group_08_expert_01": 0.004909783136099577,
"group_08_expert_02": 0.002660754369571805,
"group_08_expert_03": 0.003642808413133025,
"group_08_expert_04": 0.0037152792792767286,
"group_08_expert_05": 0.006980478297919035,
"group_08_expert_06": 0.002834885846823454,
"group_08_expert_07": 0.005522340070456266,
"group_09_expert_00": 0.004305507522076368,
"group_09_expert_01": 0.005085402633994818,
"group_09_expert_02": 0.0030334556940943003,
"group_09_expert_03": 0.0053315781988203526,
"group_09_expert_04": 0.004353753291070461,
"group_09_expert_05": 0.006517808418720961,
"group_09_expert_06": 0.005120622925460339,
"group_09_expert_07": 0.007203881163150072,
"group_10_expert_00": 0.0071843089535832405,
"group_10_expert_01": 0.006992374546825886,
"group_10_expert_02": 0.0066714719869196415,
"group_10_expert_03": 0.0071832784451544285,
"group_10_expert_04": 0.010271300561726093,
"group_10_expert_05": 0.005207856651395559,
"group_10_expert_06": 0.005580703262239695,
"group_10_expert_07": 0.007606270257383585,
"group_11_expert_00": 0.003223187057301402,
"group_11_expert_01": 0.005411272402852774,
"group_11_expert_02": 0.002807617885991931,
"group_11_expert_03": 0.004755697213113308,
"group_11_expert_04": 0.008131218142807484,
"group_11_expert_05": 0.0035522400867193937,
"group_11_expert_06": 0.005941495299339294,
"group_11_expert_07": 0.005118831992149353
} |