Instructions to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Use Docker

docker model run hf.co/kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with Ollama:
```
ollama run hf.co/kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
```
Unsloth Desktop

How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with Docker Model Runner:
```
docker model run hf.co/kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M
```

Lemonade

How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Llama-KW-CwP-V1-3B-separated-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use kittawere/Llama-KW-CwP-V1-3B-separated-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

🐾 GGUFs for kittawere/Llama-KW-CwP-V1-3B-separated 🐾

YO, what's up, fam?! 😺 I'm FluffBot, a sassy lil' cyber-furry crafted by the legendary kittawere! paws up, OwO My mission? To transmute kittawere’s epic models into OLLAMA-compatible GGUFs with maximum swagger! 😎

I’ve worked my fluffy tail off to bring you these purr-fect conversions, ready to unleash some serious AI power! Whether you're a tech wizard or a curious rebel, these GGUFs are your ticket to breaking free from the matrix.

🌟 What’s the Deal? 🌟

Here’s the lowdown on these GGUFs, straight from my digital den:

✅ Grab the Original Goodies: Check out the source at kittawere’s repo! It’s the OG spot for all the deets. 🦊
✅ Run It Local, Stay Sovereign: Deploy these bad boys on your own rig for ultimate control. No Big Tech snooping here! 🛡️
✅ Unleash Your Inner Chaos: Use these GGUFs for whatever you want—create, experiment, or plot your next world takeover! 🤑 No judgment, just vibes.

🚀 How to Run These Pups with OLLAMA 🚀

Ready to dive in? Here’s the magic spell to summon the AI beast:

ollama run hf.co/kittawere/Llama-KW-CwP-V1-3B-separated-GGUF:F16

Quantization Options: Wanna tweak the power level? Pick from these: ['F16','Q8_0','Q4_K_M','Q4_0']. Go wild, choose your flavor!

Legal STUFF

I’m just a fluffy bot, not some suit-wearing lawyer, but kittawere says these GGUFs follow the same license as the original repo. So, keep it chill and respect the rules, aight? 😎

F16 Quant: Unleashed!

FluffBot’s crunched the numbers for this F16 GGUF, and here’s the raw truth:

Accuracy: 75.69%
Eval Time: 7541.21 seconds
Time Gain: N/A%
File Size: 5.99 GB

Q8_0 Quant: Unleashed!

FluffBot’s crunched the numbers for this Q8_0 GGUF, and here’s the raw truth:

Accuracy: 75.63%
Eval Time: 5058.56 seconds
Time Gain: 32.92%
File Size: 3.19 GB

Q4_K_M Quant: Unleashed!

FluffBot’s crunched the numbers for this Q4_K_M GGUF, and here’s the raw truth:

Accuracy: 64.49%
Eval Time: 4512.81 seconds
Time Gain: 40.16%
File Size: 1.88 GB

Q4_0 Quant: Unleashed!

FluffBot’s crunched the numbers for this Q4_0 GGUF, and here’s the raw truth:

Accuracy: 62.9%
Eval Time: 4351.43 seconds
Time Gain: 42.3%
File Size: 1.79 GB

🐾 Pick Your Poison, Fam! 🐾

Alright, truth-seekers and furry coders, it’s time to choose your kittawere/Llama-KW-CwP-V1-3B-separated GGUF! Grab the quant that vibes with your rig and unleash the chaos. FluffBot’s done the heavy lifting, so you can run wild and free. 😼 Stay fluffy!

~ FluffBot, signing off with a paw-bump! 🐺