Instructions to use argyelan/Qwen3.8-Flash-Next-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use argyelan/Qwen3.8-Flash-Next-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf argyelan/Qwen3.8-Flash-Next-GGUF
# Run inference directly in the terminal:
llama cli -hf argyelan/Qwen3.8-Flash-Next-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf argyelan/Qwen3.8-Flash-Next-GGUF
# Run inference directly in the terminal:
llama cli -hf argyelan/Qwen3.8-Flash-Next-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf argyelan/Qwen3.8-Flash-Next-GGUF
# Run inference directly in the terminal:
./llama-cli -hf argyelan/Qwen3.8-Flash-Next-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf argyelan/Qwen3.8-Flash-Next-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf argyelan/Qwen3.8-Flash-Next-GGUF

Use Docker

docker model run hf.co/argyelan/Qwen3.8-Flash-Next-GGUF

LM Studio
Jan
Ollama
How to use argyelan/Qwen3.8-Flash-Next-GGUF with Ollama:
```
ollama run hf.co/argyelan/Qwen3.8-Flash-Next-GGUF
```
Unsloth Desktop

How to use argyelan/Qwen3.8-Flash-Next-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf argyelan/Qwen3.8-Flash-Next-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "argyelan/Qwen3.8-Flash-Next-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use argyelan/Qwen3.8-Flash-Next-GGUF with Docker Model Runner:
```
docker model run hf.co/argyelan/Qwen3.8-Flash-Next-GGUF
```

Lemonade

How to use argyelan/Qwen3.8-Flash-Next-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull argyelan/Qwen3.8-Flash-Next-GGUF

Run and chat with the model

lemonade run user.Qwen3.8-Flash-Next-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use argyelan/Qwen3.8-Flash-Next-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf argyelan/Qwen3.8-Flash-Next-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default argyelan/Qwen3.8-Flash-Next-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use argyelan/Qwen3.8-Flash-Next-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf argyelan/Qwen3.8-Flash-Next-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "argyelan/Qwen3.8-Flash-Next-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-Flash-Next — GGUF (day-0 conversion)

GGUF conversion of Qwen/Qwen3.8-Flash-Next-FP8 (125B MoE + 51B n-gram embeddings, 6B active, Qwen4 architecture preview) for llama.cpp.

Converted on day 0 on a DGX Spark, before quants were available anywhere — by now several groups ship Flash-Next GGUFs (unsloth and others), so pick what fits your box. This one is built for a single 128 GB Spark-class machine, and the FP8 converter fixes it needed are documented below.

Files

File	Size	What
Qwen3.8-Flash-Next-IQ4XS-NGQ5-*.gguf (3 shards)	98 GB	IQ4_XS weights + 51B n-gram table at Q5_0 (custom streaming requant). Fits and runs on a single DGX Spark / GX10 — 34 tok/s measured. Point llama.cpp at the first shard.
(Q8_0 master, 176 GB — upload follows after multi-box testing)

About the n-gram table: llama-quantize cannot process the single 51B-parameter tensor (dequantizing it needs ~204 GB RAM, and its 160-wide rows make k-quants impossible). This build uses a custom chunked streaming requantizer to bring it to Q5_0 — tool + full walkthrough: marknx/flash-next-gguf-tools. The table is lookup-only, so it tolerates this well (all gates below passed).

Requirements

llama.cpp with Qwen4-Exp support: PR #27742 (not yet in a release)
Full recipe, converter fixes and the streaming requantizer: marknx/flash-next-gguf-tools
Tight on memory? The tools repo also has a two-box recipe (DGX Spark + RTX 5090 over 10 GbE RPC) that frees ~25 GB on the main box — that's the setup the eval numbers below were measured on.

How this was made

python convert_hf_to_gguf.py Qwen3.8-Flash-Next-FP8 --outtype q8_0 --fp8-as-q8
llama-quantize --allow-requantize --tensor-type "per_layer_token_embd=q8_0" q8.gguf iq4xs.gguf iq4_xs

Honest caveats

Requantized from the official FP8 checkpoint (not BF16), without imatrix. Good enough to run day 0; expect refined quants (imatrix, tuned mixes) from the usual sources soon.
Gates run before upload: loads clean on a DGX Spark, coherent German/English output, 47*83 correct, JSON-schema output valid, needle retrieval clean. ~34 tok/s single stream, prefill ~550 tok/s.
Full evals on this exact quant: HumanEval 97.0 % (159/164) · GSM8K 96.0 % (192/200, no empty answers). Run day 0 via llama.cpp server, greedy-ish defaults.
New architecture: expect llama.cpp PR updates; rebuild when #27742 moves.

Measured and documented at argyelan.ai. Numbers, warts and all.

Downloads last month: 519

GGUF

Model size

177B params

Architecture

qwen4exp

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for argyelan/Qwen3.8-Flash-Next-GGUF

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

(144)

this model