Model card, license and build method

LICENSE

@@ -0,0 +1,16 @@
+Qwen Community License 1.0
+
+Copyright (c) 2026 Qwen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software, including the model weights, parameters, configuration files, inference code and associated documentation files (collectively, the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, sell, deploy, host, fine-tune, and create derivative works from (collectively, "Use" or "Using") copies of the Software; and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. If the Software (or any derivative works thereof) is Used for any of the licensee's commercial products or services that have more than 100,000,000 monthly active users or US$ 20,000,000 (or equivalent in other currencies) monthly revenue, respective model name must be prominently displayed on the user interface of such product or service; and,
+
+2. If the licensee or any of its affiliates conducts a Model as a Service or AI Work Assistant business, the licensee shall obtain a separate license from Qwen before Using the Software or its derivative works for any commercial purpose. The foregoing requirement shall not apply to the licensee's internal Use of the Software, provided that such Use does not make the Software, its outputs, or its underlying model capabilities available to any third party.
+
+"Model as a Service" means giving a third party access to language model inference or fine-tuning (e.g., via API or a hosted endpoint) in a manner that allows such third parties to exercise meaningful control over the inputs, parameters, or training data. This does not include the mere relaying of requests to models hosted by other third parties. 
+“AI Work Assistant” means an independent AI-powered product primarily designed for AI-assisted coding or office productivity (e.g., Qoder and QwenWork). It does not include: (a) a single-purpose AI tool (such as an AI translation tool); (b) an AI assistant primarily designed for a domain other than coding or office productivity (such as Taobao AI Shopping Assistant or AMap AI Chat); or (c) an AI assistant that is a feature of a product whose primary purpose is not AI-assisted coding or office productivity.
+
+THE SOFTWARE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL QWEN, ITS AFFILIATES OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. THE USE OF THE SOFTWARE MUST COMPLY WITH APPLICABLE LAWS AND REGULATIONS, AND MUST NOT INFRINGE THE INTELLECTUAL PROPERTY RIGHTS OF ANY THIRD PARTY.
+
+For any questions regarding this license, please contact model-business@notice.qwencloud.com.

README.md

@@ -0,0 +1,123 @@
+---
+license: other
+license_name: qwen-community-1.0
+license_link: LICENSE
+base_model:
+- Qwen/Qwen3.8-Flash-Next
+base_model_relation: quantized
+pipeline_tag: text-generation
+library_name: llama.cpp
+tags:
+- gguf
+- llama.cpp
+- qwen4exp
+- strix-halo
+- unified-memory
+---
+
+# Qwen3.8-Flash-Next-Q4_K_XL-DN4
+
+This is unsloth's UD-Q4_K_XL with one change: the 48 expert down projections are IQ4_NL instead of Q5_1. That takes the file from 111.3 GB to 102.5 GB. On the same text it measures as the same model.
+
+I built it because I run this model on a 128 GB AMD Strix Halo (Ryzen AI MAX+ 395, Radeon 8060S) under Windows with llama.cpp on Vulkan. On that machine one process can put about 78 GiB on the GPU. UD-Q4_K_XL does not fit that unless you page or move expert layers to the CPU, and the next size down, UD-IQ4_XS, is a worse model. Nobody ships the size in between, so I made it. If your machine has room for something between 94 GB and 111 GB, this is for you.
+
+Every number below was measured on that machine. The logs are in the method notes.
+
+## What changed
+
+| tensor group | size | UD-Q4_K_XL | this file |
+|---|---|---|---|
+| expert gate and up (`ffn_gate_exps`, `ffn_up_exps`) | 44.4 GB | Q4_K, layer 2 at Q5_K | same |
+| expert down (`ffn_down_exps`, 48 layers) | 27.1 GB to 22.6 GB | Q5_1, 5 layers at Q8_0 | IQ4_NL |
+| n-gram table (per-layer embeddings) | 28.8 GB | IQ4_NL | same |
+| attention, linear attention, hyper-connections, shared expert, embeddings, output | 5.5 GB | Q8_0 | same |
+| total | | 111.3 GB | 102.5 GB |
+
+1,224 tensors in four shards, same split layout as the source. 48 tensors differ. Every other byte is unsloth's.
+
+Why the down projections, and why IQ4_NL: `ffn_down_exps` is 640 values wide. That is not a
+multiple of 256, so none of the K-quants or I-quants with 256-value blocks can touch it. unsloth had to leave it at Q5_1, a 32-block type at 6 bits, which makes it the one big tensor group in the file with more bits than it needs. The 32-block family also has IQ4_NL at 4.5 bits. Moving the downs there saves 8.9 GB and changes nothing else. The same rule is why the 28.8 GB table (rows 160 wide) cannot get any smaller.
+
+## Quality
+
+Same engine, same text (wikitext-2 test, 24 chunks of 2048 tokens, `-b 2048 -ub 1024 -fa on`), same machine.
+
+| file | perplexity | KL divergence vs UD-Q4_K_XL | same top-1 token |
+|---|---|---|---|
+| unsloth UD-Q4_K_XL, 111.3 GB | 2.9117 ± 0.0363 | reference | reference |
+| this file, 102.5 GB | 2.9165 ± 0.0363 | 0.028 ± 0.001 | 94.5 % |
+| unsloth UD-IQ4_XS, 93.7 GB | 2.998 ± 0.037 | 0.089 | 90.75 % |
+
+Perplexity moves 0.17 %, which is inside the error bar. The KL divergence is the number that
+matters: median 0.005, 99th percentile 0.34, one token in eighteen changes its top pick. IQ4_XS changes one in eleven and its tail is three times worse.
+
+Two checks on that number. The reference had to run with eight expert layers on the CPU, because 111 GB does not fit my GPU. Scoring this file the same way gives 0.0277 and 94.8 %, so the CPU kernels add nothing and the 0.028 is the quantization itself. And for scale: the q4_0 KV cache that I run every day puts 0.026 between this file and itself, and I have never noticed it.
+
+## The imatrix made no difference
+
+The downs here were requantized from unsloth's Q5_1 without an importance matrix. I expected that to be the weak point, so I tried it: the same 48 tensors requantized with unsloth's published `imatrix_unsloth.gguf_file`, everything else identical.
+
+| downs requantized | KL divergence vs UD-Q4_K_XL | same top-1 |
+|---|---|---|
+| without the imatrix (this file) | 0.0282 ± 0.0007 | 94.50 % |
+| with the imatrix | 0.0277 ± 0.0007 | 94.60 % |
+
+Inside one error bar on every statistic except the single worst token. From a 6-bit source the IQ4_NL grid sets the error and the importance weighting has nothing left to steer. I deleted the imatrix build and kept this one. I guess a rung quantized from the BF16 checkpoint with the imatrix would still be the right way to make this file, though.
+
+## Running it on a 128 GB Strix Halo, Windows, Vulkan
+
+This is what serves on my machine every day, at 262144 context, the whole transformer on the GPU, no expert layers on the CPU:
+
+```
+llama-server -m Qwen3.8-Flash-Next-Q4_K_XL-DN4-00001-of-00004.gguf \
+  -c 262144 -ngl 99 -fa on -ctk q4_0 -ctv q4_0 -ub 512 -np 1 \
+  -lm mmap --lazy-mode on --cache-ram 0 --ctx-checkpoints 8 --jinja \
+  -md mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-type draft-mtp,ngram-mod \
+  --spec-draft-n-max 4 --spec-draft-p-min 0.75 \
+  --spec-ngram-mod-n-min 16 --spec-ngram-mod-n-max 128 --spec-ngram-mod-n-match 48
+```
+
+Memory mode: dynamic settings with the shared memory slider at its 124 GB maximum. The driver then lets one process address about 78 GiB on the GPU. This file lands at 77.9 to 78.5 GiB with the q4_0 cache and the Q4_K_M draft head. If it ever dies at warm-up, `-ub 256` frees 1.7 GiB. I tested that fallback and it works.
+
+The table streams from the file. `-lm mmap --lazy-mode on` keeps the 28.8 GB n-gram table out of committed memory and reads rows as needed. It measured the same as a resident table. `mlock` cannot pin a region this big on Windows, so do not bother.
+
+The draft head is unsloth's `mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf`. Those are Qwen's own MTP weights, exported by unsloth. The head borrows embeddings from the main model by shape, so it pairs with this file unchanged. It is worth about 50 % more speed on fresh text. The n-gram drafter adds about 13 % on edits.
+
+The KV cache at q4_0 is the biggest quality cost in this setup: 0.026 of KL divergence against an f16 cache at 2048 tokens, 0.020 at 16k, and it does not grow with depth. The value side is the sensitive one. q8_0 on both sides costs 0.012 but needs the 1.7 GiB from `-ub 256`. I tried keys at q4_0 with values at q8_0. It fits, it scores 0.019, and it is a trap: on this Vulkan build the mixed types leave the fast attention path, and decode falls to 17.0 / 13.2 / 6.8 tokens per second in the under-25k, 25 to 50k and 50 to 100k bands, against 19.8 / 18.8 / 17.1 for q4_0 on both sides. A short bench cannot see this. Keep the cache types the same on both sides.
+
+Speed on my machine. Prefill about 180 tokens per second at shallow depth, 120 with the MTP head on a 22k prompt. Decode 28 to 32 on fresh text with the head, 45 to 50 on edits with both drafters, 11 to 13 past 200k tokens of context. On fresh text this file runs about 20 % slower than the source did in a fixed 96 GB carve-out, because IQ4_NL kernels cost more per step on this driver than Q5_1. That is the one price.
+
+Vision. unsloth's `mmproj` for this model works with it. On this driver run the projector on the CPU with `--no-mmproj-offload`.
+
+Engine used throughout: unsloth's llama.cpp fork build `b10715` with PR 27836 and unslothai#142 for MTP. Mainline llama.cpp loads the file the same way, as long as your build knows the `qwen4exp` architecture.
+
+## Files
+
+| file | bytes | sha256 |
+|---|---|---|
+| `Qwen3.8-Flash-Next-Q4_K_XL-DN4-00001-of-00004.gguf` | 10,946,624 | 4448186216b3af4cc558bbce2c3213f01608f8f8b2e5267a9767971dd3ec8082 |
+| `Qwen3.8-Flash-Next-Q4_K_XL-DN4-00002-of-00004.gguf` | 47,447,858,336 | 9904da25dd2dc97c1756ac88e0d56ff30012b65e890161b90b14b378a7d3d883 |
+| `Qwen3.8-Flash-Next-Q4_K_XL-DN4-00003-of-00004.gguf` | 44,552,691,904 | 2685325f5dfab3e4abad9f9519d1236b31b112b2bd7e5a3ff0afefd34e91b4cd |
+| `Qwen3.8-Flash-Next-Q4_K_XL-DN4-00004-of-00004.gguf` | 10,462,690,720 | e656c8d08713915c9005ee2a631e8c02efa6c5914cc5c492c97e2a1c5ff82f92 |
+
+Total 102,474,187,584 bytes, 102.47 GB, 95.43 GiB. Shard 1 holds the metadata and the split
+header only, as in the source.
+
+## How I built it, and how to build the next one
+
+`method/` next to this card has the three scripts (plain Python, no numpy) and the exact commands. The short version:
+
+1. `gguf_tensor_map.py` reads a GGUF's real composition from its header alone, from a local file or straight from a Hugging Face URL. Judge a quant by this map, never by its name.
+2. `gguf_extract.py` pulls the 48 down tensors out of the four source shards into one loadable GGUF. 29 GB, three minutes.
+3. `llama-quantize --allow-requantize --tensor-type ffn_down_exps=iq4_nl downs.gguf downs-iq4nl.gguf IQ4_NL`, about 25 minutes. The explicit `--tensor-type` is not optional. With the `IQ4_NL` file type alone, llama-quantize picks q5_K for the downs, hits the 256-block rule, and silently falls back to q5_1. You get the same file back and a success message. Read the log.
+4. `gguf_splice.py` writes a new shard set. Every tensor is copied byte for byte except the 48 replacements, with offsets recomputed. Eleven minutes. The originals are never touched.
+
+The same recipe works on any tensor whose width is not a multiple of 256, on any model.
+
+## Credits and license
+
+Qwen made the model. It is under the Qwen Community License 1.0, included here as `LICENSE`, as the license asks for derivatives.
+
+unsloth made UD-Q4_K_XL, which is every tensor in this file but 48, and exported the MTP head. Their choice to keep attention, the linear attention blocks and the hyper-connections at Q8_0 is what makes this model's 4-bit quants good. My measurements say those small tensors decide quality more than the expert bits do.
+
+I built and measured this on my own machine. I am not affiliated with Qwen or unsloth.

SHA256SUMS

@@ -0,0 +1,4 @@
+4448186216b3af4cc558bbce2c3213f01608f8f8b2e5267a9767971dd3ec8082 *Qwen3.8-Flash-Next-Q4_K_XL-DN4-00001-of-00004.gguf
+9904da25dd2dc97c1756ac88e0d56ff30012b65e890161b90b14b378a7d3d883 *Qwen3.8-Flash-Next-Q4_K_XL-DN4-00002-of-00004.gguf
+2685325f5dfab3e4abad9f9519d1236b31b112b2bd7e5a3ff0afefd34e91b4cd *Qwen3.8-Flash-Next-Q4_K_XL-DN4-00003-of-00004.gguf
+e656c8d08713915c9005ee2a631e8c02efa6c5914cc5c492c97e2a1c5ff82f92 *Qwen3.8-Flash-Next-Q4_K_XL-DN4-00004-of-00004.gguf

method/METHOD.md

@@ -0,0 +1,86 @@
+# How Qwen3.8-Flash-Next-Q4_K_XL-DN4 was built
+
+Three pure-Python scripts (Python 3.10+, no numpy, no gguf package) and one llama.cpp tool.
+Everything here is byte-exact and reversible: the source shards are never modified.
+
+Requirements: `llama-quantize` from any llama.cpp build that knows the `qwen4exp` architecture
+(unsloth's fork `b10715` was used), about 35 GB of scratch space, and the four
+`unsloth/Qwen3.8-Flash-Next-GGUF` `UD-Q4_K_XL` shards.
+
+## 0. Look before you touch: the tensor map
+
+```
+python gguf_tensor_map.py Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+python gguf_tensor_map.py hf:unsloth/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+```
+
+Reads the header only (about 32 MB), locally or over HTTP range requests, and prints every tensor
+group with its type and size, the expert gate/up/down types per layer band, and the KV-cache cost
+per token for this architecture. This is how the down projections were found to be Q5_1 while
+everything around them was Q4_K, and how the 640-wide shape (not a multiple of 256) explained why.
+
+## 1. Extract the tensors to requantize
+
+```
+python gguf_extract.py Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf downs.gguf "ffn_down_exps"
+```
+
+Writes one loadable GGUF holding only the tensors whose name matches the regex, carrying the first
+shard's full metadata with the `split.*` keys rewritten for a single file, so `llama-quantize`
+accepts it. 48 tensors, 29 GB, three minutes on NVMe.
+
+## 2. Requantize, with the explicit tensor type
+
+```
+llama-quantize --allow-requantize --tensor-type ffn_down_exps=iq4_nl downs.gguf downs-iq4nl.gguf IQ4_NL
+```
+
+20 to 26 minutes on 16 Zen 5 cores. **`--tensor-type ffn_down_exps=iq4_nl` is not optional.** With only
+the `IQ4_NL` file type, llama-quantize's per-tensor heuristic chooses q5_K for down projections,
+finds that 640 is not divisible by 256, and falls back to q5_1, the type the tensors already had.
+The tool reports success and the output is byte-for-byte the input. Read the per-tensor lines in
+the log: every `ffn_down_exps` line must end in `iq4_nl`, and the output must be about 22.6 GB.
+
+`--allow-requantize` is needed because the source is already quantized. A 6-bit Q5_1 source is
+close enough to lossless that the extra step costs little; do not use this recipe to requantize a
+4-bit tensor into another 4-bit type, where the errors compound.
+
+## 3. Splice the replacements into a new shard set
+
+```
+python gguf_splice.py Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf downs-iq4nl.gguf Qwen3.8-Flash-Next-Q4_K_XL-DN4
+```
+
+Writes `Qwen3.8-Flash-Next-Q4_K_XL-DN4-0000N-of-00004.gguf` next to the source shards. Every tensor
+is copied byte for byte unless a tensor of the same name exists in the replacement file, in which
+case the replacement's shape, type and data are used and the offsets are recomputed; shard 1
+(metadata only) is copied under the new name; `split.*` keys are regenerated. 11 minutes.
+
+Verify with the tensor map on the new first shard: the tensor count must match
+`split.tensors.count` (1,224), the down projections must read IQ4_NL on every layer, and nothing
+else may have changed.
+
+## 4. Measure before you believe it
+
+```
+llama-perplexity -m <REF>.gguf  -f wiki.test.raw -c 2048 -b 2048 -ub 1024 --chunks 24 -ngl 99 -fa on --kl-divergence-base base.kld
+llama-perplexity -m <CAND>.gguf -f wiki.test.raw -c 2048 -b 2048 -ub 1024 --chunks 24 -ngl 99 -fa on --kl-divergence-base base.kld --kl-divergence
+```
+
+The reference run writes every scored token's logits (12.2 GB for 24 × 1024 scored tokens); the
+candidate run reads them back and reports perplexity, mean and percentile KL divergence, and the
+share of tokens whose top-1 prediction is unchanged. Perplexity alone is too blunt for a change
+this small; the KLD is the number that says whether the file is the same model. Corpus: wikitext-2
+raw test set (`https://huggingface.co/datasets/ggml-org/ci/resolve/main/wikitext-2-raw-v1.zip`).
+
+If the reference does not fit on your GPU, `--n-cpu-moe N --no-host -lm mmap -lzm on` runs it
+with the first N expert layers on the CPU at roughly 30 % lower prefill speed; that is how the
+111 GB reference was scored on the 128 GB build machine.
+
+## Why this generalises
+
+Any tensor whose row width is not a multiple of 256 is limited to the 32-block types (Q4_0, Q4_1,
+Q5_0, Q5_1, Q8_0, IQ4_NL), and quantizers tend to leave such tensors at a fat 32-block type rather
+than reason about them. Wherever a large tensor group sits at Q5_1 or Q5_0 for that reason alone,
+IQ4_NL is a 25 % cut that costs about what a Q5 to Q4 step costs anywhere else. That is usually
+nothing you can measure when the tensor feeds an addition rather than a gate.

method/gguf_extract.py

@@ -0,0 +1,113 @@
+#!/usr/bin/env python
+"""Extract selected tensors from a (split) GGUF into a single, loadable GGUF. Pure Python, no numpy.
+
+The output carries the full metadata of the first shard (so llama-quantize can load it), with the
+split.* keys rewritten to describe a single file, followed by only the selected tensors.
+
+Usage:
+  python gguf_extract.py <first shard or single .gguf> <output.gguf> <regex on tensor name>
+Example (the 2026-09-03 build, all expert down projections):
+  python gguf_extract.py Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf downs.gguf "ffn_down_exps"
+
+Written 2026-09-03 for the "Q4_K_XL with IQ4_NL downs" build (see studies/). See gguf_splice.py for the way back.
+"""
+import glob, os, re, struct, sys
+
+ALIGN = 32
+T = {0: 'B', 1: 'b', 2: 'H', 3: 'h', 4: 'I', 5: 'i', 6: 'f', 7: '?', 10: 'Q', 11: 'q', 12: 'd'}
+# bytes per block, elements per block (for exact tensor byte sizes)
+BLOCK = {0: (4, 1), 1: (2, 1), 30: (2, 1), 2: (18, 32), 3: (20, 32), 6: (22, 32), 7: (24, 32), 8: (34, 32),
+         10: (84, 256), 11: (110, 256), 12: (144, 256), 13: (176, 256), 14: (210, 256), 20: (18, 32),
+         23: (136, 256), 21: (110, 256), 22: (82, 256), 18: (66, 256), 16: (66, 256), 17: (74, 256),
+         19: (50, 256), 29: (56, 256)}
+
+
+def nbytes(shape, ty):
+    n = 1
+    for d in shape: n *= d
+    bs, be = BLOCK[ty]
+    assert n % be == 0, (shape, ty)
+    return n // be * bs
+
+
+class Reader:
+    def __init__(self, path):
+        self.path = path; self.f = open(path, 'rb'); self.size = os.path.getsize(path)
+        assert self.f.read(4) == b'GGUF'
+        self.version = self.rd('I'); self.n_tensors = self.rd('Q'); self.n_kv = self.rd('Q')
+        self.kv = []          # (key, type, raw_bytes_of_value)
+        for _ in range(self.n_kv):
+            key = self.rstr(); ty = self.rd('I'); start = self.f.tell(); self.skip_val(ty); end = self.f.tell()
+            self.f.seek(start); raw = self.f.read(end - start); self.kv.append((key, ty, raw))
+        self.tensors = []     # (name, shape, type, offset)
+        for _ in range(self.n_tensors):
+            name = self.rstr(); nd = self.rd('I'); shape = [self.rd('Q') for _ in range(nd)]
+            ty = self.rd('I'); off = self.rd('Q'); self.tensors.append((name, shape, ty, off))
+        self.data_start = (self.f.tell() + ALIGN - 1) // ALIGN * ALIGN
+
+    def rd(self, fmt): return struct.unpack('<' + fmt, self.f.read(struct.calcsize(fmt)))[0]
+    def rstr(self): n = self.rd('Q'); return self.f.read(n).decode('utf-8', 'replace')
+    def skip_val(self, ty):
+        if ty == 8: n = self.rd('Q'); self.f.seek(n, 1)
+        elif ty == 9:
+            et = self.rd('I'); n = self.rd('Q')
+            for _ in range(n): self.skip_val(et)
+        else: self.f.seek(struct.calcsize(T[ty]), 1)
+
+    def read_tensor(self, name, shape, ty, off):
+        self.f.seek(self.data_start + off); return self.f.read(nbytes(shape, ty))
+
+
+def wstr(s):
+    b = s.encode('utf-8'); return struct.pack('<Q', len(b)) + b
+
+
+def patched_kv(kv, n_tensors_out):
+    out = []
+    for key, ty, raw in kv:
+        if key == 'split.no': raw = struct.pack('<H', 0)
+        elif key == 'split.count': raw = struct.pack('<H', 1)
+        elif key == 'split.tensors.count': raw = struct.pack('<i', n_tensors_out)
+        out.append((key, ty, raw))
+    return out
+
+
+def shards_of(first):
+    m = re.match(r'(.*)-(\d{5})-of-(\d{5})\.gguf$', first)
+    if not m: return [first]
+    base, _, count = m.groups()
+    return [f'{base}-{i:05d}-of-{count}.gguf' for i in range(1, int(count) + 1)]
+
+
+def main():
+    first, out, pattern = sys.argv[1], sys.argv[2], re.compile(sys.argv[3])
+    shards = [Reader(p) for p in shards_of(first)]
+    meta = shards[0].kv
+    selected = [(r, t) for r in shards for t in r.tensors if pattern.search(t[0])]
+    print(f'{len(selected)} tensors selected from {len(shards)} shard(s)')
+    kv = patched_kv(meta, len(selected))
+    # header
+    hdr = b'GGUF' + struct.pack('<IQQ', 3, len(selected), len(kv))
+    for key, ty, raw in kv: hdr += wstr(key) + struct.pack('<I', ty) + raw
+    infos = b''; off = 0; layout = []
+    for r, (name, shape, ty, src_off) in selected:
+        sz = nbytes(shape, ty); layout.append((r, name, shape, ty, src_off, off, sz))
+        infos += wstr(name) + struct.pack('<I', len(shape)) + b''.join(struct.pack('<Q', d) for d in shape) + struct.pack('<IQ', ty, off)
+        off += (sz + ALIGN - 1) // ALIGN * ALIGN
+    head = hdr + infos
+    pad = (ALIGN - len(head) % ALIGN) % ALIGN
+    total = 0
+    with open(out, 'wb') as o:
+        o.write(head + b'\x00' * pad)
+        for r, name, shape, ty, src_off, dst_off, sz in layout:
+            r.f.seek(r.data_start + src_off)
+            remaining = sz
+            while remaining:
+                chunk = r.f.read(min(remaining, 64 << 20)); o.write(chunk); remaining -= len(chunk)
+            o.write(b'\x00' * ((ALIGN - sz % ALIGN) % ALIGN)); total += sz
+            print(f'  {name:40s} type {ty:2d} {sz / 1e6:9.1f} MB', file=sys.stderr)
+    print(f'wrote {out}: {total / 1e9:.2f} GB of tensor data')
+
+
+if __name__ == '__main__':
+    main()

method/gguf_splice.py

@@ -0,0 +1,75 @@
+#!/usr/bin/env python
+"""Splice replacement tensors into a split GGUF, writing a new shard set. Pure Python, no numpy.
+
+Every tensor of the original shards is copied byte-for-byte unless a tensor of the same name exists
+in the replacement GGUF, in which case the replacement's shape/type/data is used instead. Shard 1
+(metadata only in gguf-split layouts) is copied verbatim under the new name; shards with tensors get
+their split.* keys regenerated and tensor offsets recomputed. The originals are never modified.
+
+Usage:
+  python gguf_splice.py <original first shard> <replacement.gguf> <output base name>
+  -> writes <output base name>-0000N-of-0000M.gguf next to the original shards
+Example (2026-09-03):
+  python gguf_splice.py Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf build-downs-iq4nl.gguf Qwen3.8-Flash-Next-UD-Q4_K_XL-dn4
+
+Verify afterwards with gguf_tensor_map.py on the new first shard (tensor count must match split.tensors.count).
+"""
+import os, re, shutil, struct, sys
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from gguf_extract import Reader, nbytes, wstr, shards_of, ALIGN
+
+
+def write_shard(out_path, kv, tensors, sources):
+    """kv: list of (key, type, raw); tensors: list of (name, shape, type); sources: name -> (Reader, src_off)."""
+    hdr = b'GGUF' + struct.pack('<IQQ', 3, len(tensors), len(kv))
+    for key, ty, raw in kv: hdr += wstr(key) + struct.pack('<I', ty) + raw
+    infos = b''; off = 0; layout = []
+    for name, shape, ty in tensors:
+        sz = nbytes(shape, ty); layout.append((name, shape, ty, off, sz))
+        infos += wstr(name) + struct.pack('<I', len(shape)) + b''.join(struct.pack('<Q', d) for d in shape) + struct.pack('<IQ', ty, off)
+        off += (sz + ALIGN - 1) // ALIGN * ALIGN
+    head = hdr + infos
+    pad = (ALIGN - len(head) % ALIGN) % ALIGN
+    with open(out_path, 'wb') as o:
+        o.write(head + b'\x00' * pad)
+        for name, shape, ty, dst_off, sz in layout:
+            r, src_off = sources[name]
+            r.f.seek(r.data_start + src_off); remaining = sz
+            while remaining:
+                chunk = r.f.read(min(remaining, 64 << 20)); o.write(chunk); remaining -= len(chunk)
+            o.write(b'\x00' * ((ALIGN - sz % ALIGN) % ALIGN))
+    return off
+
+
+def main():
+    first, repl_path, out_base = sys.argv[1], sys.argv[2], sys.argv[3]
+    shard_paths = shards_of(first); n = len(shard_paths)
+    repl = Reader(repl_path)
+    repl_map = {t[0]: t for t in repl.tensors}
+    print(f'replacement carries {len(repl_map)} tensors')
+    total = 0; replaced = 0; out_dir = os.path.dirname(first) or '.'
+    for i, p in enumerate(shard_paths, 1):
+        out_path = os.path.join(out_dir, f'{out_base}-{i:05d}-of-{n:05d}.gguf')
+        r = Reader(p)
+        if r.n_tensors == 0:
+            r.f.close(); shutil.copyfile(p, out_path); print(f'shard {i}: metadata only, copied'); continue
+        kv = []
+        for key, ty, raw in r.kv:
+            kv.append((key, ty, raw))   # split.no/count/tensors.count unchanged: same shard numbering, same total
+        tensors = []; sources = {}
+        for name, shape, ty, off in r.tensors:
+            if name in repl_map:
+                _, rshape, rty, roff = repl_map[name]
+                assert rshape == shape, (name, shape, rshape)
+                tensors.append((name, rshape, rty)); sources[name] = (repl, roff); replaced += 1
+            else:
+                tensors.append((name, shape, ty)); sources[name] = (r, off)
+        written = write_shard(out_path, kv, tensors, sources); total += written
+        print(f'shard {i}: {len(tensors)} tensors, {written / 1e9:.2f} GB -> {os.path.basename(out_path)}')
+        r.f.close()
+    print(f'done: {replaced} tensors replaced, {total / 1e9:.2f} GB of tensor data written')
+    if replaced != len(repl_map): print(f'WARNING: {len(repl_map) - replaced} replacement tensors were not found in the original')
+
+
+if __name__ == '__main__':
+    main()

method/gguf_tensor_map.py

@@ -0,0 +1,181 @@
+#!/usr/bin/env python
+"""Map where the bytes of a GGUF go, without loading it.
+
+Reads only the header (KV metadata + tensor infos) of every shard, so it works on a local
+file or on a Hugging Face repo by range-fetching the first MBs of each shard. Prints:
+  - bytes per category (experts, n-gram/PLE table, embeddings, attention, rest) by quant type
+  - expert gate/up/down quant type per layer band
+  - nextn (MTP) tensors and nextn_shared_target_tensors, if present
+  - the KV cache: which layers hold one (full attention / sliding window / recurrent), bytes per
+    token, and the cache size at 262k context for f16, q8_0 and q4_0 -- plus the GPU-resident sum
+    (weights excluding a host-side PLE table + KV), which is the number to hold against the memory
+    budget in platform.md before downloading anything
+
+Usage:
+  python gguf_tensor_map.py <local .gguf (any shard; all shards in the dir are found)>
+  python gguf_tensor_map.py hf:<org>/<repo> [<subfolder-or-name-filter>]
+
+Written 2026-09-03 for the Qwen3.8-Flash-Next quant study (see studies/). Shard size for the
+last tensor comes from the file size, so remote mode needs the HF tree API (no auth needed).
+"""
+import collections, glob, json, os, re, struct, sys, urllib.request
+
+T = {0: 'B', 1: 'b', 2: 'H', 3: 'h', 4: 'I', 5: 'i', 6: 'f', 7: '?', 10: 'Q', 11: 'q', 12: 'd'}
+TYPES = {0: 'F32', 1: 'F16', 2: 'Q4_0', 3: 'Q4_1', 6: 'Q5_0', 7: 'Q5_1', 8: 'Q8_0', 9: 'Q8_1', 10: 'Q2_K', 11: 'Q3_K',
+         12: 'Q4_K', 13: 'Q5_K', 14: 'Q6_K', 15: 'Q8_K', 16: 'IQ2_XXS', 17: 'IQ2_XS', 18: 'IQ3_XXS', 19: 'IQ1_S',
+         20: 'IQ4_NL', 21: 'IQ3_S', 22: 'IQ2_S', 23: 'IQ4_XS', 24: 'I8', 25: 'I16', 26: 'I32', 27: 'I64', 28: 'F64',
+         29: 'IQ1_M', 30: 'BF16', 34: 'TQ1_0', 35: 'TQ2_0', 39: 'MXFP4'}
+HEAD_BYTES_FIRST, HEAD_BYTES_OTHER = 32 << 20, 2 << 20   # first shard carries the tokenizer
+
+
+def parse_header(blob, fsize):
+    f = memoryview(blob); pos = 0
+    def rd(fmt):
+        nonlocal pos
+        v = struct.unpack_from('<' + fmt, f, pos)[0]; pos += struct.calcsize(fmt); return v
+    def rstr():
+        nonlocal pos
+        n = rd('Q'); s = bytes(f[pos:pos + n]).decode('utf-8', 'replace'); pos += n; return s
+    def rval(t):
+        if t == 8: return rstr()
+        if t == 9:
+            et = rd('I'); n = rd('Q')
+            return [rval(et) for _ in range(n)]   # full arrays: per-layer KV heads and SWA patterns live here
+        return rd(T[t])
+    assert bytes(f[0:4]) == b'GGUF', 'not a GGUF'
+    pos = 4; rd('I'); nt = rd('Q'); nkv = rd('Q')
+    kv = {}
+    for _ in range(nkv):
+        k = rstr(); t = rd('I'); kv[k] = rval(t)
+    ts = []
+    for _ in range(nt):
+        name = rstr(); nd = rd('I'); shape = [rd('Q') for _ in range(nd)]; ty = rd('I'); off = rd('Q')
+        ts.append((name, shape, TYPES.get(ty, str(ty)), off))
+    align = kv.get('general.alignment') or 32
+    data_start = (pos + align - 1) // align * align
+    ts.sort(key=lambda x: x[3])
+    out = []
+    for i, (name, shape, ty, off) in enumerate(ts):
+        nxt = ts[i + 1][3] if i + 1 < len(ts) else fsize - data_start
+        out.append((name, shape, ty, nxt - off))
+    return kv, out
+
+
+def local_shards(path):
+    d = os.path.dirname(path) or '.'
+    base = re.sub(r'-\d{5}-of-\d{5}\.gguf$', '', os.path.basename(path))
+    files = sorted(glob.glob(os.path.join(d, base + '-*-of-*.gguf'))) or [path]
+    for p in files:
+        with open(p, 'rb') as fh:
+            yield p, fh.read(HEAD_BYTES_FIRST), os.path.getsize(p)
+
+
+def hf_shards(repo, flt):
+    tree = json.load(urllib.request.urlopen(f'https://huggingface.co/api/models/{repo}/tree/main?recursive=true'))
+    files = sorted(e['path'] for e in tree if e['type'] == 'file' and e['path'].endswith('.gguf') and (flt or '') in e['path'])
+    sizes = {e['path']: e['size'] for e in tree if e['type'] == 'file'}
+    for i, p in enumerate(files):
+        req = urllib.request.Request(f'https://huggingface.co/{repo}/resolve/main/{p}',
+                                     headers={'Range': f'bytes=0-{(HEAD_BYTES_FIRST if i == 0 else HEAD_BYTES_OTHER) - 1}'})
+        yield p, urllib.request.urlopen(req).read(), sizes[p]
+
+
+def category(name):
+    if name == 'per_layer_token_embd.weight': return 'n-gram/PLE table'
+    if name in ('token_embd.weight', 'output.weight'): return name
+    if not name.startswith('blk.'): return 'other top-level'
+    if '_exps' in name: return 'experts'
+    if 'shexp' in name: return 'shared expert'
+    if 'attn_' in name or 'indexer' in name: return 'attention'
+    if 'ssm_' in name: return 'linear attention (ssm)'
+    if 'hc_' in name: return 'hyper-connections'
+    if 'nextn' in name: return 'nextn (MTP)'
+    return 'other per-layer'
+
+
+KV_BYTES = {'f16': 2.0, 'q8_0': 34 / 32, 'q4_0': 18 / 32}   # bytes per cached value (block = 32 values + scale)
+KV_CTX = 262144
+
+
+def kv_cache_report(kv, arch, table_bytes, total_bytes):
+    """Estimate the KV cache from the header: per token and at 262k, for the cache types that matter here.
+
+    Layer kinds: full attention (pays the whole context), sliding-window (llama.cpp iSWA keeps only
+    window + micro-batch tokens), recurrent / linear attention (a fixed state, no per-token cache).
+    Verified 2026-09-04 against qwen4exp: 12 full layers x 2 KV heads x K256/V256 at q4_0 = 1728 MiB,
+    exactly the 'Vulkan0 KV buffer size' the server reports at 262144 context.
+    """
+    n = kv.get(f'{arch}.block_count')
+    if not n:
+        return
+    g = lambda k, d=None: kv.get(f'{arch}.{k}', d)
+    n_head, n_embd = g('attention.head_count'), g('embedding_length')
+    hd = n_embd // n_head if n_embd and n_head else None
+    heads = g('attention.head_count_kv', n_head)
+    heads = heads if isinstance(heads, list) else [heads] * n
+    k_len, v_len = g('attention.key_length', hd), g('attention.value_length', hd)
+    k_swa, v_swa = g('attention.key_length_swa', k_len), g('attention.value_length_swa', v_len)
+    swa, pat = g('attention.sliding_window', 0), g('attention.sliding_window_pattern')
+    if isinstance(pat, list):
+        is_swa = [bool(x) for x in pat]
+    elif isinstance(pat, int) and pat > 0 and swa:
+        is_swa = [(i + 1) % pat != 0 for i in range(n)]
+    else:
+        is_swa = [False] * n
+    interval = g('full_attention_interval')            # hybrid families: every Nth layer is attention, the rest recurrent
+    is_rec = [bool(interval and (i + 1) % interval != 0) or not heads[i] for i in range(n)]
+    full = [i for i in range(n) if not is_rec[i] and not is_swa[i]]
+    win = [i for i in range(n) if not is_rec[i] and is_swa[i]]
+    if k_len is None or v_len is None:
+        print("  KV cache: cannot estimate (no head/embedding sizes in header)"); return
+    vals_full = sum((k_len + v_len) * heads[i] for i in full)          # cached values per token, full layers
+    vals_win = sum((k_swa + v_swa) * heads[i] for i in win)
+    kinds = f"{len(full)} full-attention"
+    if win: kinds += f", {len(win)} sliding-window (window {swa})"
+    if n - len(full) - len(win): kinds += f", {n - len(full) - len(win)} recurrent/no-KV"
+    hs = sorted(set(heads[i] for i in full + win))
+    print(f"  KV cache: {kinds} layers; KV heads {hs} x K{k_len}/V{v_len}"
+          + (f" (window layers K{k_swa}/V{v_swa})" if win and (k_swa, v_swa) != (k_len, v_len) else ""))
+    weights = total_bytes - table_bytes
+    print(f"  GPU-resident weights {weights / 2**30:.1f} GiB" + (f" (the {table_bytes / 2**30:.1f} GiB PLE table stays in host RAM)" if table_bytes else "")
+          + f"; at {KV_CTX // 1024}k context the cache adds:")
+    for t, b in KV_BYTES.items():
+        cache = vals_full * b * KV_CTX + vals_win * b * min(KV_CTX, swa + 512)
+        print(f"    {t:5s} {vals_full * b / 1024:6.1f} KiB/token  ->  {cache / 2**30:6.2f} GiB cache  ->  {(weights + cache) / 2**30:6.1f} GiB on the GPU before compute buffers and any draft head")
+    if g('attention.indexer.head_count'):
+        print("    note: this architecture also allocates an attention-indexer cache the estimate leaves out "
+              "(qwen4exp measured 648 MiB extra at q4_0, 262144 ctx)")
+
+
+def main():
+    arg = sys.argv[1]
+    shards = hf_shards(arg[3:], sys.argv[2] if len(sys.argv) > 2 else '') if arg.startswith('hf:') else local_shards(arg)
+    agg = collections.defaultdict(collections.Counter); per = collections.defaultdict(dict); kvs = {}; total = 0
+    for p, blob, fsize in shards:
+        kv, ts = parse_header(blob, fsize); kvs.update(kv)
+        for name, shape, ty, sz in ts:
+            agg[category(name)][ty] += sz; total += sz
+            if name.startswith('blk.') and '_exps' in name:
+                per[int(name.split('.')[1])][name.split('.')[2].replace('ffn_', '').replace('_exps', '')] = ty
+    arch = kvs.get('general.architecture')
+    print(f"{arg}\n  arch={arch} name={kvs.get('general.name')} blocks={kvs.get(f'{arch}.block_count')} "
+          f"nextn={kvs.get(f'{arch}.nextn_predict_layers', 0)} shared_target={kvs.get(f'{arch}.nextn_shared_target_tensors')}")
+    print(f"  total {total / 1e9:.2f} GB ({total / 2**30:.2f} GiB)")
+    for c, cnt in sorted(agg.items(), key=lambda x: -sum(x[1].values())):
+        s = sum(cnt.values())
+        print(f"  {s / 1e9:7.2f} GB  {c:24s} " + ", ".join(f"{t}:{v / 1e9:.1f}" for t, v in cnt.most_common()))
+    if per:
+        print("  expert gate / up / down by layer band:")
+        runs, prev, start = [], None, 0
+        for b in sorted(per):
+            sig = (per[b].get('gate'), per[b].get('up'), per[b].get('down'))
+            if sig != prev:
+                if prev: runs.append((start, b - 1, prev))
+                prev, start = sig, b
+        runs.append((start, max(per), prev))
+        for a, b, (g, u, d) in runs: print(f"    layers {a:2d}-{b:2d}: {g} / {u} / {d}")
+    kv_cache_report(kvs, arch, sum(agg['n-gram/PLE table'].values()), total)
+
+
+if __name__ == '__main__':
+    main()