Russian speaking 7B models
Collection
There is some my 7B models good speak and understand Russian language. Approved by some data-set my own tests. Will be link to github repo soon...🪬 • 7 items • Updated • 3
GGUF
English
Russian
Thai
Merge
mergekit
lazymergekit
AlekseiPravdin/KukulStanta-InfinityRP-7B-slerp
AlekseiPravdin/NSK-128k-7B-slerp
Q2_K
Q3_K_L
Q3_K_M
Q3_K_S
Q4_0
Q4_1
Q4_K_S
Q4_k_m
Q5_0
Q5_1
Q6_K
Q5_K_S
Q5_k_m
Q8_0
128k
conversational
How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:# Run inference directly in the terminal:
llama cli -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:# Run inference directly in the terminal:
./llama-cli -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:# Run inference directly in the terminal:
./build/bin/llama-cli -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:Use Docker
docker model run hf.co/AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:Quick Links
KSI-RP-NSK-128k-7B-GGUF ⭐️⭐️⭐️⭐️
KSI-RP-NSK-128k-7B is a merge of the following models using mergekit:
🧩 Configuration
slices:
- sources:
- model: AlekseiPravdin/KukulStanta-InfinityRP-7B-slerp
layer_range: [0, 32]
- model: AlekseiPravdin/NSK-128k-7B-slerp
layer_range: [0, 32]
merge_method: slerp
base_model: AlekseiPravdin/NSK-128k-7B-slerp
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16
Eval embedding benchmark (with 70 specific quesions):
- Downloads last month
- 200
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf:# Run inference directly in the terminal: llama cli -hf AlekseiPravdin/KSI-RP-NSK-128k-7B-gguf: