Instructions to use Outlier-Ai/Outlier-150B-V3.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Outlier-Ai/Outlier-150B-V3.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Outlier-Ai/Outlier-150B-V3.2", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Outlier-Ai/Outlier-150B-V3.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Outlier-Ai/Outlier-150B-V3.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Outlier-Ai/Outlier-150B-V3.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2

SGLang

How to use Outlier-Ai/Outlier-150B-V3.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Outlier-Ai/Outlier-150B-V3.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Outlier-Ai/Outlier-150B-V3.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Outlier-Ai/Outlier-150B-V3.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Outlier-Ai/Outlier-150B-V3.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Outlier-Ai/Outlier-150B-V3.2 with Docker Model Runner:
```
docker model run hf.co/Outlier-Ai/Outlier-150B-V3.2
```

Outlier-150B-V3.2 / alpha.json

ur-dad-matt

Upload alpha.json with huggingface_hub

af6d780 verified 4 months ago

Raw

History Blame Contribute Delete

4.06 kB

	{
	"group_01_expert_00": 0.013282203115522861,
	"group_01_expert_01": 0.007036757655441761,
	"group_01_expert_02": 0.022870026528835297,
	"group_01_expert_03": 0.013156086206436157,
	"group_01_expert_04": 0.015288042835891247,
	"group_01_expert_05": 0.005860608536750078,
	"group_01_expert_06": 0.02469489350914955,
	"group_01_expert_07": 0.016237638890743256,
	"group_02_expert_00": 0.018590012565255165,
	"group_02_expert_01": 0.005804466549307108,
	"group_02_expert_02": 0.02403106354176998,
	"group_02_expert_03": 0.007332759443670511,
	"group_02_expert_04": 0.0130878034979105,
	"group_02_expert_05": 0.015886439010500908,
	"group_02_expert_06": 0.0069729238748550415,
	"group_02_expert_07": 0.019574716687202454,
	"group_03_expert_00": 0.010419095866382122,
	"group_03_expert_01": 0.005606003105640411,
	"group_03_expert_02": 0.008018190972507,
	"group_03_expert_03": 0.003162325592711568,
	"group_03_expert_04": 0.006889252923429012,
	"group_03_expert_05": 0.010474657639861107,
	"group_03_expert_06": 0.0032552790362387896,
	"group_03_expert_07": 0.006326879840344191,
	"group_04_expert_00": 0.0012291143648326397,
	"group_04_expert_01": 0.00388744892552495,
	"group_04_expert_02": 0.004821837414056063,
	"group_04_expert_03": 0.003917478024959564,
	"group_04_expert_04": 0.014478147961199284,
	"group_04_expert_05": 0.005330056417733431,
	"group_04_expert_06": 0.006041828542947769,
	"group_04_expert_07": 0.005014166701585054,
	"group_05_expert_00": 0.006031031254678965,
	"group_05_expert_01": 0.0034807720221579075,
	"group_05_expert_02": 0.003914565779268742,
	"group_05_expert_03": 0.006052051670849323,
	"group_05_expert_04": 0.004927619826048613,
	"group_05_expert_05": 0.0028023945633322,
	"group_05_expert_06": 0.002118324162438512,
	"group_05_expert_07": 0.007412611041218042,
	"group_06_expert_00": 0.003685481147840619,
	"group_06_expert_01": 0.007255200762301683,
	"group_06_expert_02": 0.0038398539181798697,
	"group_06_expert_03": 0.004417807329446077,
	"group_06_expert_04": 0.0025816322304308414,
	"group_06_expert_05": 0.0033090233337134123,
	"group_06_expert_06": 0.005978161934763193,
	"group_06_expert_07": 0.006351307965815067,
	"group_07_expert_00": 0.002889202209189534,
	"group_07_expert_01": 0.0024275293108075857,
	"group_07_expert_02": 0.0031850712839514017,
	"group_07_expert_03": 0.0038473173044621944,
	"group_07_expert_04": 0.004140378441661596,
	"group_07_expert_05": 0.004112926311790943,
	"group_07_expert_06": 0.005547539331018925,
	"group_07_expert_07": 0.003908976446837187,
	"group_08_expert_00": 0.002477911999449134,
	"group_08_expert_01": 0.004909783136099577,
	"group_08_expert_02": 0.002660754369571805,
	"group_08_expert_03": 0.003642808413133025,
	"group_08_expert_04": 0.0037152792792767286,
	"group_08_expert_05": 0.006980478297919035,
	"group_08_expert_06": 0.002834885846823454,
	"group_08_expert_07": 0.005522340070456266,
	"group_09_expert_00": 0.004305507522076368,
	"group_09_expert_01": 0.005085402633994818,
	"group_09_expert_02": 0.0030334556940943003,
	"group_09_expert_03": 0.0053315781988203526,
	"group_09_expert_04": 0.004353753291070461,
	"group_09_expert_05": 0.006517808418720961,
	"group_09_expert_06": 0.005120622925460339,
	"group_09_expert_07": 0.007203881163150072,
	"group_10_expert_00": 0.0071843089535832405,
	"group_10_expert_01": 0.006992374546825886,
	"group_10_expert_02": 0.0066714719869196415,
	"group_10_expert_03": 0.0071832784451544285,
	"group_10_expert_04": 0.010271300561726093,
	"group_10_expert_05": 0.005207856651395559,
	"group_10_expert_06": 0.005580703262239695,
	"group_10_expert_07": 0.007606270257383585,
	"group_11_expert_00": 0.003223187057301402,
	"group_11_expert_01": 0.005411272402852774,
	"group_11_expert_02": 0.002807617885991931,
	"group_11_expert_03": 0.004755697213113308,
	"group_11_expert_04": 0.008131218142807484,
	"group_11_expert_05": 0.0035522400867193937,
	"group_11_expert_06": 0.005941495299339294,
	"group_11_expert_07": 0.005118831992149353
	}