kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-GGUF

🔧 Runtime: build the ROCmFPX fork below

Stock llama.cpp will not load this file. You need both the muse-glimmer architecture and the ROCmFP4 tensor types in one tree. Upstream charlie12345/ROCmFPX has the ROCmFP4 types but not muse-glimmer. Our fork has both:

kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

git clone https://github.com/kingjones30/ROCmFPX.git
cd ROCmFPX
cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server llama-quantize -j$(nproc)

Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a muse-glimmer ROCmFP4 GGUF from this family and generates coherent text.

Muse-Glimmer-30B — ROCmFPX 8-bit for AMD Strix Halo (gfx1151)

✅ the complete muse-glimmer port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo

muse-glimmer is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in patches/muse-glimmer-complete.patch (20 files, 81,968 bytes, git apply --check clean).

An 8-bit ROCmFPX quantisation of Muse-Glimmer-30B for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, quantised from the BF16 GGUF — a lossless source, not a requantisation of a smaller file. ROCmFPX is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp.

Metric	Result
Quantization	Q8_0_ROCMFPX (ggml ftype 111)
Model size	26.85 GiB (28,826,594,688 bytes)
Effective BPW	8.28 (measured, not advertised)
Source	BF16 GGUF, 55,725,514,112 bytes
Tested hardware	AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified
GPU	Radeon 8060S, gfx1151
Decode, no draft head	7.48 tok/s (median of 3: 7.54, 7.48, 7.35)
Decode with DFlash, prose	11.31 tok/s
Correctness	✅ 3/3 — 391 / Tokyo / 366
Vision (256×256 placement)	✅ pass, requires -fa off
Chat parser	✅ clean content — no control-token leak
Speculative head	DFlash, not MTP

Why this build?

• Quantised from BF16, source revision verified by fetch (a4e59da52a7bc87ae7251dd5545c0dd437c44b68) rather than assumed
• Output head and token embeddings pinned to q8_0 explicitly, not left to inherit the block type
• The full architecture port is in the repo — you do not have to reconstruct it
• Vision works through the BF16 projector built by stage 2 of that patch
• Sizes are reported from stat, and checked against the --dry-run projection: the ~12.5 MiB delta is GGUF header, which is the signature of a complete file

Which file should I use?

If you want speed, take a 4-bit build — on this model the 4-bit files are roughly 3.5× the code-transform throughput of the 8-bits and less than half the size. The 8-bits are here for bit-count, not for speed.

Build	ftype	Size	BPW	prose	code-transform	accept len (code)
ROCmFP4-FAST	103	13.80 GiB	4.25	15.07	39.35	7.12
ROCmFP4-STRIX	105	14.17 GiB	4.39	14.96	37.55	6.80
Q8_0_ROCMFPX	111	26.85 GiB	8.28	11.31	—	2.65
Q8_0_ROCMFPX_AGENT	115	27.23 GiB	8.39	11.27	—	2.51

⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves 2.5 → 7.1 across the same binary and the same weights. Quote a range for this model, not a point.

muse-glimmer-30B-Q8_0_ROCMFPX.gguf and muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf are within noise of each other (7.48 vs the other build's figure on the same harness). The AGENT routing lifts draft acceptance on MTP models; this model uses DFlash, so there is nothing for it to win here. Choose on size.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-GGUF --local-dir glimmer

The DFlash drafter is not duplicated here — pull it from the 4-bit repo, which carries every drafter and projector variant:

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "dflash-ROCmFP4-STRIX.gguf" --local-dir glimmer

⚠️ hf download silently ignores --include when given more than one pattern — one call per file.

llama-server \
  -m glimmer/muse-glimmer-30B-Q8_0_ROCMFPX.gguf \
  --mmproj glimmer/mmproj-muse-glimmer-30B-BF16.gguf \
  --spec-type draft-dflash --model-draft glimmer/dflash-ROCmFP4-STRIX.gguf \
  --spec-draft-ngl 99 \
  -ngl 999 -c 4096 -fa off -fit off --jinja \
  --host 127.0.0.1 --port 8080

Three flags that matter more than which file you pick

Flag	Why
--model-draft	Serve it with the DFlash head. Without one, the 8-bit build drops 11.62 → 7.65 tok/s (−34%). This single flag outweighs the quantisation choice.
-fa off	Required for the vision path on gfx1151. Text-only serving can use -fa on.
-fit off	llama.cpp's autofit reads MemAvailable on integrated GPUs, which is at its lowest right after a model unload — leaving it on can silently shrink context or push tensors to CPU.

⚠️ This head is DFlash, not MTP. Read mean accepted length, and do not pass MTP flags to it.

Vision

Verified on spatial placement rather than plausible-sounding output: a solid-colour image is scored on whether the model names the colour that is actually there.

Input	Result
256×256 solid red	✅ red

Serve with -fa off and the BF16 projector from this repo (mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors).

⚠️ Minimum useful image size is 28×28 px. The preprocessor snaps to patch 14 × merge 2, so anything smaller collapses to a single merge token, carries no spatial signal, and the model reports the dominant colour of the padded canvas. Feed 256×256 or larger. llama-mtmd-cli behaves identically — this is the preprocessing geometry, not the projector.

Files

File	Size	Role
muse-glimmer-30B-Q8_0_ROCMFPX.gguf	26.85 GiB	model — this repo
mmproj-muse-glimmer-30B-BF16.gguf	3.58 GiB	vision projector — use this one
patches/muse-glimmer-complete.patch	80 KiB	20-file architecture port
dflash-ROCmFP4-STRIX.gguf	1.39 GiB	DFlash drafter — lives in the 4-bit repo, not here

🩹 The muse-glimmer architecture port — complete, three stages

patches/muse-glimmer-complete.patch — 81,968 bytes, 20 files, git apply --check clean against charlie12345/ROCmFPX.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6

muse-glimmer is not an upstream architecture. Running it end-to-end takes three independent pieces of work, and all three are in this patch.

Stage 1 — text graph and conversion

File	Role
src/llama-arch.{h,cpp}	LLM_ARCH_MUSE_GLIMMER + tensor-name table
src/llama-model.cpp	model factory case
src/models/muse-glimmer.cpp	the graph itself
src/models/models.h	declaration
gguf-py/gguf/{constants,tensor_mapping}.py	GGUF constants + tensor map
conversion/{__init__,muse_glimmer}.py	safetensors → GGUF converter

⚠️ convert_hf_to_gguf.py does not know MuseGlimmer. The converter is conversion/muse_glimmer.py, driven through conversion.get_model_class.

Stage 2 — vision projector (clip / mtmd)

File	Role
tools/mtmd/models/muse-glimmer.cpp	projector graph
tools/mtmd/clip-impl.h	PROJECTOR_TYPE_MUSE_GLIMMER
tools/mtmd/clip-model.h	hparams + LANCZOS enum
tools/mtmd/clip-graph.h	build_vit_opts + 7-arg overload
tools/mtmd/clip.cpp, models/models.h, CMakeLists.txt	wiring
tools/mtmd/mtmd-image.{cpp,h}	preprocessor; LANCZOS → bicubic_pillow fallback
tools/mtmd/mtmd.cpp	<|image_start|> / <|image_end|> markers
gguf-py/gguf/tensor_mapping.py	model.vision_tower.layers.{bid}.attn.{q,k,v,proj}, norm1/2, mlp.fc1/2, ln_post
gguf-py/gguf/constants.py	VisionProjectorType.MUSE_GLIMMER

This stage is what makes --mmproj work. Use the BF16 projector in this repo (mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors, clip.projector_type = muse-glimmer, merge 2, patch 14, image_size 896).

Stage 3 — chat parser

common/chat.cpp — common_chat_params_init_muse_glimmer, PEG_NATIVE.

Without it the model's to=self<|message|> control sequence is emitted into content. With it, content is clean: "The capital of Japan is Tokyo."

Build note

cmake's source GLOB is configure-time. After the patch adds src/models/muse-glimmer.cpp you must re-run the cmake -S . -B build-muse configure step, not just --build. -DLLAMA_BUILD_WEBUI=OFF avoids a node/npm requirement.

Applying to a different base commit

The patch header names commit 3edc3d3, and it applies cleanly to later revisions (verified on b41ce12). On trees where cohere2moe and bailing_hybrid model sources are absent, their factory cases in llama-model.cpp reference symbols that do not exist in that tree — build those two out, or apply on 3edc3d3 where their .cpp files are present. The muse-glimmer factory case and graph are independent of both.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (the muse-glimmer converter is not in
#    convert_hf_to_gguf.py; it ships as conversion/muse_glimmer.py in the patch)
python -c "from conversion import get_model_class; ..."   # driver, see patch README

# 2. measure the real BPW before committing
llama-quantize --dry-run muse-glimmer-30B-BF16.gguf /tmp/x.gguf Q8_0_ROCMFPX 8

# 3. quantize with the patched ROCmFPX build (only it has ggml types 100-106)
llama-quantize --output-tensor-type q8_0 --token-embedding-type q8_0 \
  muse-glimmer-30B-BF16.gguf muse-glimmer-30B-Q8_0_ROCMFPX.gguf Q8_0_ROCMFPX 16

Source: meta-models/Muse-Glimmer-30B, revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68 (fetched and verified, not assumed) → muse-glimmer-30B-BF16.gguf 55,725,514,112 bytes. Quantized from BF16 only — this is not a requantization of a smaller file.

--output-tensor-type and --token-embedding-type are both set explicitly so the output head and token embeddings land at q8_0 rather than inheriting the block type.

Not yet measured

Listed so nobody mistakes absence for a pass:

Test	Status
Perplexity / KL divergence vs BF16	❓ not measured
Long-context behaviour beyond 4096	❓ not measured
Vision beyond single-colour placement (OCR, charts, documents)	❓ not measured
MMLU-Pro, GPQA, GSM8K, HumanEval+	❓ not run
Multi-step agentic loop	❓ not run
Sustained multi-thousand-token generation	❓ not measured
Independent reproduction	❓ none yet

Known issues

1. Stock llama.cpp cannot load these files. muse-glimmer is not an upstream architecture and ROCmFP4/ROCmFPX are not upstream tensor types. Both come from the patch in this repo.
2. -fa off is required for the vision path on gfx1151. Text-only serving runs fine with -fa on.
3. Images below 28×28 px collapse to a single merge token. The preprocessor snaps to patch 14 × merge 2, so an 8×8 input carries no usable spatial signal and the model reports the dominant colour of the padded canvas. Feed images at 256×256 or larger.
4. Vulkan / CUDA / CPU cannot load these files — ROCmFP4/ROCmFPX are ROCm-only formats.

Independent results

None yet. If you run this build, please open a discussion with hardware, GPU, ROCm version, runtime commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent reproductions will be listed separately from author benchmarks and carry more weight.

License and attribution

Base model and its licence are the Muse team's. ROCmFP4 / ROCmFPX quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains the quantised weights, the architecture port and the measurements above.

Acknowledgements

ROCmFPX — maintained by charlie12345 The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every file here was produced with its llama-quantize and runs on its runtime. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm — the compute platform these builds target, on gfx1151 / Radeon 8060S.

Muse — the base model and its licence are theirs. This repository contributes the architecture port, quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.