Instructions to use MANSTAGE/analysis-llm-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use MANSTAGE/analysis-llm-v2 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use MANSTAGE/analysis-llm-v2 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MANSTAGE/analysis-llm-v2
# Run inference directly in the terminal:
llama cli -hf MANSTAGE/analysis-llm-v2

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MANSTAGE/analysis-llm-v2
# Run inference directly in the terminal:
llama cli -hf MANSTAGE/analysis-llm-v2

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf MANSTAGE/analysis-llm-v2
# Run inference directly in the terminal:
./llama-cli -hf MANSTAGE/analysis-llm-v2

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf MANSTAGE/analysis-llm-v2
# Run inference directly in the terminal:
./build/bin/llama-cli -hf MANSTAGE/analysis-llm-v2

Use Docker

docker model run hf.co/MANSTAGE/analysis-llm-v2

LM Studio
Jan

vLLM

How to use MANSTAGE/analysis-llm-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MANSTAGE/analysis-llm-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MANSTAGE/analysis-llm-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MANSTAGE/analysis-llm-v2

Ollama
How to use MANSTAGE/analysis-llm-v2 with Ollama:
```
ollama run hf.co/MANSTAGE/analysis-llm-v2
```
Unsloth Desktop
Docker Model Runner
How to use MANSTAGE/analysis-llm-v2 with Docker Model Runner:
```
docker model run hf.co/MANSTAGE/analysis-llm-v2
```

Lemonade

How to use MANSTAGE/analysis-llm-v2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull MANSTAGE/analysis-llm-v2

Run and chat with the model

lemonade run user.analysis-llm-v2-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

analysis-llm-v2 / chat_template.jinja

MANSTAGE

Upload merged model files

9d6b97e verified 12 months ago

Raw

History Blame Contribute Delete

2.25 kB

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜><think>\n'}}{% endif %}