update model and documentation

.gitattributes

@@ -1,36 +1,3 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
+*.ninfer filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
-thinkingcap-qwen3.8-27b.ninfer filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,3 @@
+*.tmp
+*.part
+__pycache__/

MODEL_FILE.txt

@@ -0,0 +1,8 @@
+Add the model binary to the repository root before pushing:
+
+thinkingcap-qwen3.8-27b-d48.ninfer
+
+Expected exact size:
+16705529348 bytes
+
+The ZIP intentionally does not contain the model binary.

PROVENANCE.md

@@ -0,0 +1,75 @@
+# Provenance
+
+## Target
+
+- Model family: ThinkingCap-Qwen3.8-27B
+- NInfer variant: D48
+- Base recipe: `qwen3_8_27b`
+- Components: text
+- Additional override: 48 selected `mlp/down` projections to NVFP4
+- Activation policy for those overrides: `AllowA4`
+- Activation input divisor used in the current experiment: `1.0`
+
+## Converter / runtime development state
+
+Base checkout commit:
+
+```text
+9e163eee4b8acec21ab0ac765107b6a3f287b217
+```
+
+Recorded local `git diff` SHA256 from the experimental checkout:
+
+```text
+2e526a1eb2333992c146fde7324d7f615dc3e4a8dae1bc01f54357aaeed2befa
+```
+
+Converter image used in the experiment:
+
+```text
+ninfer:a2-convert
+```
+
+Perplexity image used in the experiment:
+
+```text
+ninfer-lab/weekly-cal-down-probe-ppl:9e163eee-local-20260922
+```
+
+Benchmark image used in the experiment:
+
+```text
+ninfer:a2-bench
+```
+
+## Model construction rule
+
+All NInfer artifacts are generated directly from the original BF16/FP16 checkpoint. Existing `.ninfer` artifacts are not used as quantization sources.
+
+## Expected D48 artifact
+
+Filename:
+
+```text
+thinkingcap-qwen3.8-27b-d48.ninfer
+```
+
+Exact file size:
+
+```text
+16705529348 bytes
+```
+
+Runtime weight capacity observed during benchmark:
+
+```text
+16692422656 bytes
+```
+
+Workspace capacity observed during benchmark:
+
+```text
+159981568 bytes
+```
+
+The model binary itself is intentionally not included in this package.

README.md

@@ -1,48 +1,211 @@
 ---
-license: apache-2.0
-language:
-- en
-base_model:
-- bottlecapai/ThinkingCap-Qwen3.8-27B
+pipeline_tag: text-generation
 tags:
-- Blackwell
-- Ninfer
-- ThinkingCap
-- Qwen3.8
-- quantization
-- text-generation
+  - qwen
+  - ninfer
+  - quantization
+  - nvfp4
+  - blackwell
+  - text-generation
 ---
-# ThinkingCap-Qwen3.8-27B — NInfer
 
-NInfer conversions of **ThinkingCap-Qwen3.8-27B**, optimized for NVIDIA Blackwell GPUs.
+# ThinkingCap-Qwen3.8-27B — NInfer D48 NVFP4
 
-This repository currently contains two target-model variants:
+Experimental **NInfer** conversion of **ThinkingCap-Qwen3.8-27B**, optimized for NVIDIA Blackwell GPUs with a quality-first mixed-precision strategy.
 
-| Variant | File | Size | Description |
-|---|---|---:|---|
-| Baseline | `thinkingcap-qwen3.8-27b.ninfer` | 17,106,573,056 bytes | Standard NInfer `qwen3_8_27b` conversion |
-| **D48** | `thinkingcap-qwen3.8-27b-d48.ninfer` | **16,705,529,348 bytes** | Quality-first optimized variant with 48 selected MLP Down projections converted to NVFP4 |
+The current recommended artifact is the **D48 anchor**.
 
-## Recommended variant
+It starts from NInfer's standard `qwen3_8_27b` recipe and adds model-specific NVFP4 quantization to selected MLP Down projections. The D48 layers were selected from a complete 64-layer single-change sensitivity sweep and then validated cumulatively with full perplexity and runtime benchmarks.
 
-For current use, **D48 is the recommended quality-first optimized anchor**.
+## Artifact
 
-Compared with the standard NInfer conversion, D48:
+Place the model file in the repository root with this name:
 
-- reduces the artifact by approximately **401 MB**
-- reduces model size by approximately **2.34%**
-- improves prefill throughput substantially relative to the unoptimized baseline architecture
-- shows no measurable overall perplexity regression on the development corpus used during optimization
-- keeps the most sensitive MLP Down layers at the precision chosen by the standard NInfer recipe
+```text
+thinkingcap-qwen3.8-27b-d48.ninfer
+```
+
+| Property | Value |
+|---|---:|
+| Model | ThinkingCap-Qwen3.8-27B |
+| Runtime format | NInfer |
+| Variant | D48 |
+| Artifact size | 16,705,529,348 bytes |
+| Runtime weight capacity | 16,692,422,656 bytes |
+| Workspace | 159,981,568 bytes |
+| Target GPU | NVIDIA RTX PRO 4000 Blackwell 24 GB |
+| Base NInfer recipe | `qwen3_8_27b` |
+| Additional optimization | 48 selected MLP Down projections → NVFP4 |
+
+The corresponding standard NInfer baseline artifact was **17,106,573,056 bytes**. D48 reduces the artifact by **401,043,708 bytes (~382.5 MiB / 2.34%)**.
+
+## What D48 means
+
+D48 does **not** mean “layers 0–47”. Every one of the 64 MLP Down projections was tested individually. The 48 most tolerant layers were selected for the cumulative D48 build.
+
+### NVFP4 MLP Down layers
+
+```text
+0, 1, 4, 5, 6, 7, 9,
+13, 14, 15, 16, 17, 19, 20, 22, 23, 24,
+28, 29, 30, 31,
+33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
+53, 54, 55, 56, 58, 59, 60, 61, 62
+```
+
+### Protected MLP Down layers
+
+```text
+2, 3, 8, 10, 11, 12, 18, 21,
+25, 26, 27, 32,
+51, 52, 57, 63
+```
+
+These remain at the precision selected by the standard NInfer recipe.
+
+## Quantization strategy
+
+New artifacts are generated directly from the original BF16/FP16 ThinkingCap checkpoint. Existing `.ninfer` files are **not** requantized.
+
+```text
+ThinkingCap BF16/FP16 Safetensors
+        │
+        ▼
+NInfer qwen3_8_27b recipe
+        │
+        ▼
+standard mixed-precision NInfer baseline
+        │
+        ▼
+selected MLP Down projections → NVFP4
+        │
+        ▼
+ThinkingCap D48 NInfer
+```
+
+D48 is still a mixed-precision model. Only the selected MLP Down projections receive the additional NVFP4 override; all other tensors continue to use the formats chosen by the standard NInfer recipe.
+
+## Full perplexity validation
+
+Development corpus configuration:
+
+```text
+context:        4096
+stride:         2048
+KV cache:       BF16
+scored tokens:  782,535
+streams:        12
+```
+
+The corpus contains English long-form text, English reference text, and code. It is a **development/sensitivity corpus**, not an untouched final holdout.
+
+| Domain | Standard NInfer baseline | D48 |
+|---|---:|---:|
+| English long-form | 7.918420 | 7.944685 |
+| English reference | 6.264029 | 6.219786 |
+| Code | 1.651905 | 1.654696 |
+| **Overall** | **4.351060** | **4.348019** |
+
+Overall D48 delta versus the baseline is approximately **−0.070% PPL** on this development corpus.
+
+The negative aggregate delta must not be interpreted as evidence that quantization improves the underlying model. The individual domains move in different directions; the result is best read as **no measurable aggregate quality loss on this development corpus at the tested resolution**.
+
+## Runtime benchmark
 
-The optimization is deliberately conservative.
+Measured on an **NVIDIA RTX PRO 4000 Blackwell 24 GB** with FP8 KV cache, prefill chunk 1024, 2 warmups and 5 measured repetitions.
 
-The goal is not minimum file size.
+| Test | Throughput |
+|---|---:|
+| PP512 | 1293.37 tok/s |
+| PP2048 | 1333.02 tok/s |
+| PP8192 | 1310.12 tok/s |
+| TG256 | 32.46 tok/s |
+| TG1024 | 32.03 tok/s |
+| PP2048 + TG512 — prefill | 1323.95 tok/s |
+| PP2048 + TG512 — decode | 31.92 tok/s |
+| PP8192 + TG512 — prefill | 1297.68 tok/s |
+| PP8192 + TG512 — decode | 31.43 tok/s |
 
-The priority is:
+These values are intended for comparisons within the same NInfer/hardware/software setup. They should not be treated as directly comparable to unrelated runtimes or benchmark methodologies.
+
+## Why D48 is the current anchor
+
+More aggressive cumulative candidates were also tested:
+
+| Variant | Extra NVFP4 Down layers | Full PPL | Artifact size |
+|---|---:|---:|---:|
+| Standard baseline | 0 | 4.351060 | 17,106,573,056 B |
+| **D48** | **48** | **4.348019** | **16,705,529,348 B** |
+| D50 | 50 | 4.350115 | 16,688,818,692 B |
+| D53 | 53 | 4.355314 | 16,663,752,708 B |
+
+D50 was only about **16 MiB smaller** than D48 and measured roughly **+0.31% mean prefill / −0.12% mean decode** versus D48 in the same benchmark setup. D53 produced a clearer prefill gain, but also a clearer perplexity regression.
+
+For the current **quality-first** target, D48 is therefore the working Down anchor.
+
+## Current optimization status
+
+```text
+1. Standard NInfer conversion
+2. Full 64-layer MLP Down sensitivity sweep
+3. D48 quality-first Down anchor        ← this release
+4. ThinkingCap-specific IMatrix
+5. Gate / Up sensitivity analysis
+6. Combined mixed-precision optimization
+7. Final task-level quality validation
+8. MTP and/or DFlash2 integration
+```
+
+### Not yet included
+
+This D48 artifact does **not** yet include:
+
+- ThinkingCap-specific IMatrix optimization
+- Agentic-workload importance calibration
+- dedicated Gate optimization
+- dedicated Up optimization
+- final Gate/Up + Down joint optimization
+- MTP speculative decoding model
+- DFlash2 speculative decoding model
+- final task-level evaluation suite
+
+## Planned IMatrix work
+
+A ThinkingCap-specific importance matrix is planned from the original BF16/FP16 weights using an agentic/code-oriented calibration corpus.
+
+The IMatrix will be used to prioritize Gate/Up candidates, identify important tensors/channels, reduce the search space, and cross-check the existing empirical sensitivity results. It will **not** replace direct single-change and cumulative-group quality measurements.
+
+## Reproducibility
+
+Converter checkout base commit:
+
+```text
+9e163eee4b8acec21ab0ac765107b6a3f287b217
+```
+
+The local build also contains experimental changes required for the NVFP4 path, so exact compatibility can depend on the corresponding NInfer runtime implementation.
+
+See `PROVENANCE.md`, `quantization/d48-layers.txt`, and `benchmarks/` for compact experiment metadata.
+
+## Intended use
+
+This release is primarily intended for:
+
+- ThinkingCap-Qwen3.8-27B experiments with NInfer
+- NVIDIA Blackwell inference
+- mixed-precision / NVFP4 research
+- quality/performance trade-off analysis
+- further model-specific optimization work
+
+The optimization priority is:
 
 ```text
 1. Preserve model quality
-2. Improve inference throughput
-3. Reduce VRAM / model footprint
-4. Keep the configuration reproducible
+2. Improve real inference throughput
+3. Reduce persistent model/VRAM footprint
+4. Keep experiments reproducible
+```
+
+## License and upstream terms
+
+This repository does not redefine the licenses of the upstream ThinkingCap model, its base model, NInfer, or NVIDIA components. Users should review and comply with the applicable upstream licenses and terms before redistribution or use.

benchmarks/d48-runtime.csv

@@ -0,0 +1,10 @@
+test,metric,tok_per_s
+pp512,prefill,1293.37
+pp2048,prefill,1333.02
+pp8192,prefill,1310.12
+tg256,decode,32.46
+tg1024,decode,32.03
+pp2048+tg512,prefill,1323.95
+pp2048+tg512,decode,31.92
+pp8192+tg512,prefill,1297.68
+pp8192+tg512,decode,31.43

benchmarks/full-ppl.csv

@@ -0,0 +1,5 @@
+variant,english_long_form,english_reference,ninfer_code,overall,scored_tokens,context,stride,kv_dtype
+baseline,7.918420,6.264029,1.651905,4.3510600961173225,782535,4096,2048,bf16
+d48,7.944685,6.219786,1.654696,4.348019,782535,4096,2048,bf16
+d50,7.949046,6.223909,1.655082,4.350115,782535,4096,2048,bf16
+d53,7.953046,6.243357,1.655006,4.355314,782535,4096,2048,bf16

quantization/d48-layers.txt

@@ -0,0 +1,69 @@
+ThinkingCap-Qwen3.8-27B NInfer D48
+
+Selected MLP Down layers converted to NVFP4 (48):
+0
+1
+4
+5
+6
+7
+9
+13
+14
+15
+16
+17
+19
+20
+22
+23
+24
+28
+29
+30
+31
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+53
+54
+55
+56
+58
+59
+60
+61
+62
+
+Protected MLP Down layers (16):
+2
+3
+8
+10
+11
+12
+18
+21
+25
+26
+27
+32
+51
+52
+57
+63

thinkingcap-d48.ninfer

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdc304dca679849512e52c075194185193788ccaaf009148a9116f30eac69d36
+size 16705529348

verify-model.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+set -u
+MODEL="${1:-thinkingcap-qwen3.8-27b-d48.ninfer}"
+EXPECTED=16705529348
+
+if [ ! -f "$MODEL" ]; then
+  echo "ERROR: model not found: $MODEL" >&2
+  exit 1
+fi
+
+ACTUAL=$(stat -c '%s' "$MODEL")
+echo "file:  $MODEL"
+echo "bytes: $ACTUAL"
+
+if [ "$ACTUAL" -ne "$EXPECTED" ]; then
+  echo "ERROR: expected $EXPECTED bytes" >&2
+  exit 1
+fi
+
+echo "size check: PASS"
+echo
+sha256sum "$MODEL"