How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Timoche/gaby-translator-models:Q4_0# Run inference directly in the terminal:
llama cli -hf Timoche/gaby-translator-models:Q4_0Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Timoche/gaby-translator-models:Q4_0# Run inference directly in the terminal:
./build/bin/llama cli -hf Timoche/gaby-translator-models:Q4_0Use Docker
docker model run hf.co/Timoche/gaby-translator-models:Q4_0Quick Links
EuroLLM-1.7B-Instruct, Q4_0 GGUF
Q4_0 quantization of utter-project/EuroLLM-1.7B-Instruct (revision a25c7fa65fc2a644e6270b8940dbe295b51da681), used for on-device French <-> English translation in the Gaby Translator app.
- Converted to f16 with
convert_hf_to_gguf.py, then quantized withllama-quantize(llama.cpp tagb11347, commit5fc4f3c8c7103ffd0b7ff5ee4855bcc78a3ed5cd). No imatrix. eurollm-1.7b-instruct-q4_0.gguf: 1002984736 bytes, SHA-2566a07547f180dc16e14df1605ae3ba92c555665bd7d7f30ebcf267ff3146f073f.- Chat template: ChatML, end of turn
<|im_end|>.
Licensed under Apache-2.0, as the original model. All credit for the model goes to the EuroLLM authors.
- Downloads last month
- 7
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for Timoche/gaby-translator-models
Base model
utter-project/EuroLLM-1.7B Finetuned
utter-project/EuroLLM-1.7B-Instruct
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf Timoche/gaby-translator-models:Q4_0# Run inference directly in the terminal: llama cli -hf Timoche/gaby-translator-models:Q4_0