Instructions to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
# Run inference directly in the terminal:
llama cli -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
# Run inference directly in the terminal:
llama cli -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
# Run inference directly in the terminal:
./llama-cli -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
# Run inference directly in the terminal:
./build/bin/llama-cli -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Use Docker

docker model run hf.co/degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

LM Studio
Jan

vLLM

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Ollama
How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with Ollama:
```
ollama run hf.co/degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
```
Unsloth Desktop

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with Docker Model Runner:
```
docker model run hf.co/degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm
```

Lemonade

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Run and chat with the model

lemonade run user.Escha-Qwen3.8-27B-W2-Q8E-ROCm-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Escha Qwen 3.8 27b (W2 · Q8_0 head · dense) — AMD ROCm port

The Escha dense 27B model is a 2-bit-quantized build of the Qwen3.8-27B backbone (qwen35 architecture), converted to GGUF for llama.cpp and ported to run on AMD ROCm (gfx1100).

Backbone: Qwen/Qwen3.8-27B (Apache-2.0), via the Escha 2-bit quant EschaLabs/Qwen3.8-27B-Escha-W2
Format: Escha 2-bit (W2) weights + Q8_0 head; general.license = apache-2.0
Architecture: qwen35 (dense, non-MoE)
Size: 10.31 GB (9.6 GiB) on disk; 2054 tensors
Target: AMD Radeon RX 7900 XTX (gfx1100, 25.75 GB) — tested

Model configuration

Param	Value
Layers	64 (48 linear attention + 16 full attention)
Embedding	5120
FFN	17408
Attention heads / KV heads	24 / 4
Head dim	256
Context train	262144
Param count	~6.3B (Qwen3.8-27B backbone)

Usage

Grab the model file (the only file in this repo):

huggingface-cli download degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm \
  Escha-Qwen3.8-27B-W2-Q8E.gguf --local-dir .

Then build + serve through the ROCmFPX fork of llama.cpp (maintained under Chromadera). The escha decode kernel is a HIP kernel added to that tree on the escha-dense-27b branch.

git clone https://github.com/Chromadera/ROCmFPX
cd ROCmFPX && git checkout escha-dense-27b
scripts/build-rdna3.sh        # auto-selects gfx1100; see doc for ROCm-path overrides

cd build-rdna3/bin
./llama-server \
  -m ./Escha-Qwen3.8-27B-W2-Q8E.gguf \
  -ngl 99 -mg 0 -fa on --jinja \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  -c 204800 --no-warmup -np 1

Full build + run + sanity-check guidance: docs/ESCHA-DENSE-27B-RUN-AMD.md

Benchmarks

Measured via llama-bench / llama-perplexity, full offload, batch 1. Three columns: this port (llama.cpp GGUF, ROCm/gfx1100), Ajay's aj9o9/Qwen3.8-27B-Escha-W2-GGUF (llama.cpp GGUF, CUDA), and the Escha reference build (EschaLabs/Qwen3.8-27B-Escha-W2, native SGLang, RTX 4090).

Comparability: this port and Ajay's are both llama.cpp GGUF — directly comparable. Escha's figures are the native SGLang runtime on an RTX 4090, a different engine and card, so read that column as reference-scale, not an apples-to-apples run.

Speed

Test	This port (RX 7900 XTX)	Ajay (3090)	Escha (4090, SGLang)
Prefill (pp512)	162.8 tok/s	700.4	~2,700 tok/s¹
Decode (tg128 / 1-user)	25.8 tok/s	24.03	67.0 tok/s
Decode @ 32k depth	23.2 tok/s	~22	—
Decode @ 64k depth	21.2 tok/s	~20	—

¹Escha prefill is quoted from their ISL÷TTFT at batch 1 (2,600–2,820 tok/s on a 4090); the figure here is the midpoint. Escha's decode is single-stream.

Decode holds against Ajay's 3090 (25.8 vs 24.03) — decode is the memory-bandwidth-bound figure and the escha decode path saturates bandwidth here. Prefill is lower (162 vs 700) because the escha dense prefill path is more prefill-bound on this GPU.

Quality

Quality protocol differs by provider, so this is not a single scoring matrix — each row is the metric's own best measured value. This port and Ajay run llama.cpp and report perplexity; Escha reports task accuracy measured through its native SGLang runtime.

Metric	This port (7900 XTX)	Ajay (3090)	Escha (4090, SGLang)
Perplexity (512-ctx)	7.5131 ± 0.86	7.40	—
Greedy determinism (5 prompts ×2)	5/5 byte-identical	—	—
Growing-context retrieval (9k/18k/37k)	3/3 retrieved	—	—
Greedy coherence	correct (Paris, gravity, 2+2=4)	—	—
GPQA-Diamond (n=198, thinking)	—	—	88.38
LiveCodeBench v6 (n=182, thinking)	—	—	86.81
Commonsense-6 avg (thinking-off)	—	—	79.25

Note on axes. Quality protocol differs by provider, so this is not a single scoring matrix — each row is that metric's own best measured value, and the meaning of "better" differs by row (perplexity lower is better; task accuracy higher is better). This port and Ajay run llama.cpp and report perplexity (loss) and determinism; Escha reports task accuracy measured through its native SGLang runtime. A lower perplexity and a higher task score are not on the same scale, so read them as complementary evidence rather than a head-to-head. Escha reports its figures without speculation and on the same build upstream; this port reproduces that build deterministically.

Context-capacity VRAM (`q8_0 / q8_0`, full lossless)

Context	VRAM
32k	12.14 GB
64k	13.28 GB
128k	15.55 GB
200k	18.30 GB
262144 (full)	20.72 GB

The port runs the full 262144 context on a 24+ GB card because only 16 of 64 layers are full attention (see KV-cache below).

Technical notes

Reasoning

The Qwen3.8-27B backbone is a thinking model. It emits <think>...</think> reasoning blocks that the llama.cpp server splits into reasoning_content / content. It works with a Qwen-style Jinja chat template and honors per-request reasoning_effort / reasoning_budget_tokens (e.g. the quimmedes chat_template.jinja via --chat-template-file), so you can drive thinking depth by effort level.

KV-cache

The model is a 48 linear + 16 full-attention Qwen35 hybrid (full_attention_interval = 4), so only ~~16 layers carry a real KV cache. With q8_0 / q8_0 K/V this fits even at the full 262144 context (~~20.7 GB). A lighter footprint is available via the fork's TurboQuant K/V types (--cache-type-v turbo3 / turbo4).

Quantization

Tensor breakdown (2054 tensors):

Type	Count
F16	897
F32	753
escha_code (2-bit code)	402
Q8_0	2

Credits

This build builds on prior OSS work:

Ajay — https://github.com/Ajay9o9/llama.cpp-escha (upstream escha decode path + tests), published at https://huggingface.co/aj9o9/Qwen3.8-27B-Escha-W2-GGUF
Escha team / EschaLabs — the Escha 2-bit codec + reference runtime (escha 1.2.1+qwen3dense)
charlie12345 — original ROCmFPX fork this port was based on (https://github.com/charlie12345/ROCmFPX); now maintained under Chromadera at https://github.com/Chromadera/ROCmFPX

License

Apache-2.0 (matching the Qwen3.8-27B base and the general.license field in the GGUF).

Downloads last month: 449

GGUF

Model size

6B params

Architecture

qwen35

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Base model

Qwen/Qwen3.8-27B

Quantized

EschaLabs/Qwen3.8-27B-Escha-W2

Quantized

(6)

this model

Escha Qwen 3.8 27b (W2 · Q8_0 head · dense) — AMD ROCm port

Model configuration

Usage

Benchmarks

Speed

Quality

Context-capacity VRAM (q8_0 / q8_0, full lossless)

Technical notes

Reasoning

KV-cache

Quantization

Credits

License

Model tree for degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Context-capacity VRAM (`q8_0 / q8_0`, full lossless)