Instructions to use IFM/K2-Horizon-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IFM/K2-Horizon-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IFM/K2-Horizon-7B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IFM/K2-Horizon-7B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IFM/K2-Horizon-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IFM/K2-Horizon-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IFM/K2-Horizon-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/IFM/K2-Horizon-7B

SGLang

How to use IFM/K2-Horizon-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IFM/K2-Horizon-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IFM/K2-Horizon-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IFM/K2-Horizon-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IFM/K2-Horizon-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use IFM/K2-Horizon-7B with Docker Model Runner:
```
docker model run hf.co/IFM/K2-Horizon-7B
```

K2-Horizon-7B / README.md

Commit History

Document tool_call_format API options (#2)

14985b2

hunterhector

hanseungwook commited on about 12 hours ago

Update README.md

1414192
verified

chen11003 commited on 5 days ago

Update README.md

2c9659a
verified

chen11003 commited on 5 days ago

Update README.md

0d7718a
verified

ztwang commited on 5 days ago

Update README.md

8be407b
verified

ztwang commited on 5 days ago

Update README.md

aea085b
verified

ztwang commited on 5 days ago

Update README.md

00737dc
verified

chen11003 commited on 5 days ago

Update README.md

d51dceb
verified

chen11003 commited on 5 days ago

Update README.md

ab75f0a
verified

chen11003 commited on 5 days ago

Update README.md

e2a157c
verified

chen11003 commited on 5 days ago

Update README.md

28b2295
verified

aaryamonvikram commited on 5 days ago

docs: update SGLang quickstart (#1)

cf70472

akhauriyash commited on 5 days ago

Initialize private staging repo

3a1d865
verified

zhou94539 commited on 6 days ago

Commit History

Document tool_call_format API options (#2) 14985b2

Update README.md 1414192 verified

Update README.md 2c9659a verified

Update README.md 0d7718a verified

Update README.md 8be407b verified

Update README.md aea085b verified

Update README.md 00737dc verified

Update README.md d51dceb verified

Update README.md ab75f0a verified

Update README.md e2a157c verified

Update README.md 28b2295 verified

docs: update SGLang quickstart (#1) cf70472

Initialize private staging repo 3a1d865 verified

Document tool_call_format API options (#2)

14985b2

Update README.md

1414192
verified

Update README.md

2c9659a
verified

Update README.md

0d7718a
verified

Update README.md

8be407b
verified

Update README.md

aea085b
verified

Update README.md

00737dc
verified

Update README.md

d51dceb
verified

Update README.md

ab75f0a
verified

Update README.md

e2a157c
verified

Update README.md

28b2295
verified

docs: update SGLang quickstart (#1)

cf70472

Initialize private staging repo

3a1d865
verified