Instructions to use tensorblock/pythia-31m-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tensorblock/pythia-31m-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/pythia-31m-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/pythia-31m-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/pythia-31m-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/pythia-31m-GGUF:Q2_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf tensorblock/pythia-31m-GGUF:Q2_K # Run inference directly in the terminal: ./llama-cli -hf tensorblock/pythia-31m-GGUF:Q2_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf tensorblock/pythia-31m-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf tensorblock/pythia-31m-GGUF:Q2_K
Use Docker
docker model run hf.co/tensorblock/pythia-31m-GGUF:Q2_K
- LM Studio
- Jan
- Ollama
How to use tensorblock/pythia-31m-GGUF with Ollama:
ollama run hf.co/tensorblock/pythia-31m-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use tensorblock/pythia-31m-GGUF with Docker Model Runner:
docker model run hf.co/tensorblock/pythia-31m-GGUF:Q2_K
- Lemonade
How to use tensorblock/pythia-31m-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tensorblock/pythia-31m-GGUF:Q2_K
Run and chat with the model
lemonade run user.pythia-31m-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
Remove .gguf files (keep Q2_K.gguf)
Browse files- pythia-31m-Q3_K_L.gguf +0 -3
- pythia-31m-Q3_K_M.gguf +0 -3
- pythia-31m-Q3_K_S.gguf +0 -3
- pythia-31m-Q4_0.gguf +0 -3
- pythia-31m-Q4_K_M.gguf +0 -3
- pythia-31m-Q4_K_S.gguf +0 -3
- pythia-31m-Q5_0.gguf +0 -3
- pythia-31m-Q5_K_M.gguf +0 -3
- pythia-31m-Q5_K_S.gguf +0 -3
- pythia-31m-Q6_K.gguf +0 -3
- pythia-31m-Q8_0.gguf +0 -3
pythia-31m-Q3_K_L.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:1a7deba6bce795be6d4cd943bc9aeac158e1648caa3b60d24a0774556c6f89db
|
| 3 |
-
size 20643328
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q3_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:ba09f44daf1af762e04187fcd3cb6d41ebb62b959730ddfb36540a3e037efc75
|
| 3 |
-
size 20397568
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q3_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:d47da7f2e37a7fff3c433db65c0acb9c8e514759e8195b7397daa9d5c7997ede
|
| 3 |
-
size 19979776
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q4_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:0d78d1d1f5e395e74036362f51116d6e5a056bd143da02efa9aa468e514fd683
|
| 3 |
-
size 22316800
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q4_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7726402feb64fb2cb972c32b4f067cf07565c7c9f0b65efb8ce61d23d94e4d09
|
| 3 |
-
size 22599424
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q4_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:3bf741367f54dbabd4106d05381834338bfc9e8a735047538fef4ff2d9822f00
|
| 3 |
-
size 22316800
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q5_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:19a9ad43adc2787a4f71ea2ecbb6aaf35c6f7ef19bb18bb5ee92b4320e7d71de
|
| 3 |
-
size 24516352
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q5_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:288cb6b49d4fa474e3cb5c9affe9170d702e2f365840928e0491ed9cf883917f
|
| 3 |
-
size 24742656
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q5_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:5a7f700ce5850b7ce94dab4592fb49dfb2cb404f37a215947c49735231f207f2
|
| 3 |
-
size 24516352
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q6_K.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:b8986482aa9355765258e60b9cac905ffa1eecc140ea293071113fa765368510
|
| 3 |
-
size 26853376
|
|
|
|
|
|
|
|
|
|
|
|
pythia-31m-Q8_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:0d098e169d706f0d32a281bed98dce7a1de53ac59c826f7d538cbc289c7f51f4
|
| 3 |
-
size 34233856
|
|
|
|
|
|
|
|
|
|
|
|