Schestex commited on
Commit
4d0cffa
·
verified ·
1 Parent(s): 1e02fcf

update readme.md

Browse files
Files changed (1) hide show
  1. README.md +227 -78
README.md CHANGED
@@ -1,101 +1,144 @@
1
  ---
2
- license: other
3
- license_name: polyform-small-business-1.0.0
4
- license_link: LICENSE
5
  base_model: bottlecapai/ThinkingCap-Qwen3.8-27B
6
  base_model_relation: quantized
7
  library_name: ninfer
8
  pipeline_tag: text-generation
 
 
 
9
  tags:
10
- - ninfer
11
- - thinkingcap
12
  - qwen3_8
 
13
  - blackwell
14
  - sm120a
15
- - quantized
16
- - speculative-decoding
17
  - mtp
 
18
  - indexed88
 
 
 
 
19
  - text-generation
20
  ---
21
 
22
- # ThinkingCap Qwen3.8-27B — NInfer releases
23
 
24
- NInfer-format conversions of [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B), based on [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), for NVIDIA Blackwell inference.
25
 
26
- **These artifacts are native `.ninfer` files, not Safetensors, GGUF, or Transformers checkpoints.** They require a compatible [NInfer](https://github.com/Xtravaganz/ninfer) runtime and are not intended for direct loading with `transformers.AutoModel` or vLLM.
27
 
28
- ## Releases
29
 
30
- | Release | Public vocabulary | Physical rows | MTP proposal rows | Status |
31
- | --- | ---: | ---: | ---: | --- |
32
- | **Indexed88 v6.4 / 135K / R2** | **135,000** | **135,168** | **88,064** | Locally deployed; tagged container reports `canary` |
33
- | **Original V1 / 147K** | 147,590 | 147,712 | 131,072 | Previously published, separately versioned |
34
 
35
- **The 135K and 147K releases are different artifacts. Do not mix their tokenizers, proposal rankings, runtime settings, or conversion sidecars.** The original V1 remains in [`combined-147590/`](./combined-147590/).
 
 
 
36
 
37
- ## Indexed88 v6.4 — 135K / R2
38
 
39
- This release uses one physical Q8-backed MTP output head and evaluates only 88,064 selected vocabulary rows for speculative proposals. It is **not** a second output head or a second complete head projection.
40
 
41
- ### Release components
42
 
43
- | Component | Description |
44
  | --- | --- |
45
- | Model artifact | `indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer` |
46
- | Selected-row IDs | `indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le` |
47
- | Chat template | `indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja` |
48
- | Checksums | `indexed88-v6.4-135k-r2/SHA256SUMS` |
49
- | Release metadata | `indexed88-v6.4-135k-r2/RELEASE.json` |
50
 
51
- The public model filename above is a **distribution name** for the local artifact `thinkingcap-135000-q8-repack-EXPERIMENTAL.ninfer`. Renaming/copying it does not change the model bytes. Verify its SHA-256 before publication and record the checksum in `SHA256SUMS`; a public model SHA-256 has not been asserted here.
 
 
 
 
 
 
 
 
 
 
52
 
53
- Known source facts:
54
 
55
- | Item | Value |
56
- | --- | --- |
57
- | Local model artifact size | **15,820,439,556 bytes** (approximately 14.73 GiB) |
58
- | Model vocabulary | 135,000 public / 135,168 physical rows |
59
- | Indexed proposal selection | 88,064 original token IDs |
60
- | Selection file SHA-256 | `b578828c0b517ba47abe90a130fda7cde5a56d94add1afeac7623e7c8f6234b4` |
61
- | Sharp chat-template SHA-256 | `cdff39fb26b60dc90faa292e726655c6b21f62db497846e02e4c4bbab942a84a` |
62
- | Local deployment image tag | `thinkingcap-v1:indexed88-v6.4-135k-r2` |
63
- | Local deployment image label | `org.thinkingcap.release.status=canary` |
64
 
65
- **Important:** The Docker tag is a local runtime/deployment identifier, not a Hugging Face model filename and not evidence of a published OCI image. This repository distributes model assets; a container image must be distributed separately through an OCI registry if desired.
66
 
67
- ### Requirements
68
 
69
- - A NInfer build that includes **135K Indexed88 v6.4 direct Q8 MTP support**. A generic upstream NInfer binary is not guaranteed to load or run this release correctly.
70
- - Linux with an NVIDIA Blackwell GPU supported by that build (tested on an RTX PRO 4000 Blackwell with 24 GB VRAM, CUDA architecture `sm_120a`).
71
- - Compatible CUDA runtime/driver and sufficient device memory. The referenced local builds used CUDA 13.3, `sm_120a`, Release mode.
72
- - The exact 135K ranking and chat template supplied with this release.
73
 
74
- ### Download
75
 
76
- After the release directory has been uploaded:
 
 
77
 
78
  ```bash
79
  hf download Schestex/ThinkingCap-Qwen3.8-27B-NInfer \
80
  --include 'indexed88-v6.4-135k-r2/*' \
81
  --local-dir ./thinkingcap-ninfer
82
 
83
- cd thinkingcap-ninfer
84
- (cd indexed88-v6.4-135k-r2 && sha256sum --check SHA256SUMS)
85
  ```
86
 
87
- ### Example: native NInfer server
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
- This command assumes you have **built a compatible 135K Indexed88 v6.4 NInfer server**. On a host with multiple GPUs, `--device 0` refers to the first GPU visible to the process.
90
 
91
  ```bash
92
- export CUDA_VISIBLE_DEVICES=0 # change to the intended physical GPU index if needed
93
  export NINFER_MTP_INDEXED_SHADOW=1
94
  export NINFER_MTP_INDEXED_MODE=indexed96
95
- export NINFER_MTP_INDEXED_RANKING="$PWD/indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le"
 
 
96
 
97
- ./ninfer-serve \
98
- "$PWD/indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer" \
99
  --host 127.0.0.1 --port 8080 \
100
  --model-id thinkingcap-135k-indexed88 \
101
  --device 0 \
@@ -104,47 +147,153 @@ export NINFER_MTP_INDEXED_RANKING="$PWD/indexed88-v6.4-135k-r2/selected_88064_13
104
  --max-context 8192 --kv-capacity 8192 \
105
  --max-concurrency 1 --prefill-chunk 512 \
106
  --preserve-thinking \
107
- --chat-template "$PWD/indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja"
108
  ```
109
 
110
- The historical environment variable `NINFER_MTP_INDEXED_SHADOW=1` enables the Indexed-MTP configuration. **`NINFER_MTP_INDEXED_MODE=indexed96` selects the direct path** in the tested runtime, notwithstanding the older internal name. The runtime's `indexed96` enum uses 88,064 selected rows for this release. In direct mode the selected Q8 proposal is authoritative and the full MTP proposal projection is skipped. No external Q5/Q6 head is required.
111
 
112
- Do not set `NINFER_MTP_INDEXED_MODE=shadow` if you want the one-projection Indexed88 direct configuration.
113
 
114
- ### Validation and performance scope
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115
 
116
- The 135K Indexed88 configuration was evaluated on **one RTX PRO 4000 Blackwell GPU (24 GB)** using native NInfer builds, `--spec mtp --draft-tokens 5`, NVFP4 KV cache, `temperature=0`, and three consecutive requests capped at 3,500 output tokens each. Each kernel variant used the same model, ranking, and template.
117
 
118
- | Indexed88 Q8 kernel tested | Turn 1 | Turn 2 | Turn 3 | Acceptance across the test |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  | --- | ---: | ---: | ---: | ---: |
120
- | R0 reference | 95.34 tok/s | 94.75 tok/s | 90.44 tok/s | 55.33% |
121
- | Experimental R2 (row-ID hoisting) | 95.19 tok/s | 94.66 tok/s | 90.42 tok/s | 55.33% |
122
- | Experimental R3 (hoisting + occupancy) | 95.30 tok/s | 94.80 tok/s | 90.43 tok/s | 55.33% |
123
 
124
- All three experimental builds produced **identical complete response hashes for each corresponding turn**, and matched draft/acceptance counts. The R2/R3 optimizations were **experimental isolated builds, not changes to the local production image**. These results are not a standardized model-quality benchmark or a guarantee of performance on other hardware, prompts, contexts, or decoding settings. The model and the actual published image/build should be matched using separately recorded hashes before making reproducibility claims.
125
 
126
- ### Limitations
 
 
 
 
 
 
127
 
128
- - The selected 88,064-row proposal vocabulary is a performance/coverage trade-off; it is **not** a reduction of the public tokenizer to 88,064 IDs. The model's public vocabulary is 135,000.
129
- - Generated content can be inaccurate, unsafe, or unsuitable for unsupervised code/agent execution. Validate outputs before use.
130
- - This release is NInfer-runtime-specific, not a drop-in HF Transformers distribution.
131
- - CUDA Graph and runtime behavior depend on the exact NInfer build. The tested direct `indexed96` path supports CUDA Graph integration; other versions may differ.
132
- - Release status is still marked `canary` in the known local image metadata; do not interpret this page as a universal production-readiness or support guarantee.
133
 
134
- ## Provenance and attribution
135
 
136
- - **Base model:** [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B).
137
- - **ThinkingCap fine-tune:** [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B).
138
- - **NInfer:** [Xtravaganz/ninfer](https://github.com/Xtravaganz/ninfer) (the repository linked by the previous release; a matching 135K Indexed88 runtime commit/tag has not yet been publicly verified here).
139
 
140
- This is an independently packaged NInfer conversion and runtime configuration of the upstream ThinkingCap model; it should not be presented as a new pretraining or fine-tuning run.
 
 
 
 
 
141
 
142
- ## License
143
 
144
- ThinkingCap is licensed under **PolyForm Small Business License 1.0.0**, with an additional personal, noncommercial use permission from BottleCap AI. The underlying Qwen materials carry **Apache-2.0** requirements. See the original [ThinkingCap LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/LICENSE) and [NOTICE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/NOTICE).
 
 
145
 
146
- Keep the upstream license texts, copyright notices, and attribution files when redistributing these derived weights. This release does not grant broader commercial rights than the upstream license.
147
 
148
- ---
149
 
150
- For historical 147K V1 artifacts and results, retain the existing [`combined-147590/`](./combined-147590/), `provenance/`, and `benchmarks/` directories until older references have been verified and archived cleanly.
 
1
  ---
 
 
 
2
  base_model: bottlecapai/ThinkingCap-Qwen3.8-27B
3
  base_model_relation: quantized
4
  library_name: ninfer
5
  pipeline_tag: text-generation
6
+ license: other
7
+ license_name: polyform-small-business-1.0.0
8
+ license_link: LICENSE
9
  tags:
 
 
10
  - qwen3_8
11
+ - ninfer
12
  - blackwell
13
  - sm120a
 
 
14
  - mtp
15
+ - speculative-decoding
16
  - indexed88
17
+ - vocabulary-prune
18
+ - coding
19
+ - agentic
20
+ - nvfp4
21
  - text-generation
22
  ---
23
 
24
+ # ThinkingCap Qwen3.8-27B for NInfer — 135K Indexed88 & 147K V1
25
 
26
+ > **Current release: ThinkingCap 135K / Indexed88 v6.4 R2.** The original 147,590-vocabulary V1 is preserved as a separate, frozen release.
27
 
28
+ NInfer-format exports of [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B), derived from [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B). These releases target specialized coding and agent-style workloads using NVIDIA Blackwell hardware.
29
 
30
+ **Important:** A `.ninfer` file is **not** a GGUF, Safetensors, or standard Hugging Face Transformers checkpoint. Use the matching [NInfer runtime](https://github.com/Xtravaganz/ninfer). This repository contains *model artifacts*, not a downloadable Docker image.
31
 
32
+ ## Release overview
 
 
 
33
 
34
+ | Release | Public tokenizer | Physical rows | MTP proposal rows | Runtime | Status |
35
+ | --- | ---: | ---: | ---: | --- | --- |
36
+ | **135K Indexed88 v6.4 R2** | **135,000** | **135,168** | **88,064** | Indexed88 direct Q8, MTP5 | Current release; local deployment image labeled `canary` |
37
+ | **Combined 147K V1** | 147,590 | 147,712 | 131,072 | Original V1 MTP5 | Frozen / historically production validated |
38
 
39
+ **Do not mix** the model, tokenizer, physical-vocabulary layout, selection IDs, or runtime release across these versions. The `R2` suffix of the 135K release denotes the *Indexed88 ranking/release revision*; it does **not** mean the experimental R2 row-ID-hoisting kernel.
40
 
41
+ ## 135K Indexed88 v6.4 R2 — current release
42
 
43
+ ### Artifacts
44
 
45
+ | Item | Repository path |
46
  | --- | --- |
47
+ | NInfer model | [`indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer`](./indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer) |
48
+ | Sorted selected token IDs | [`indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le`](./indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le) |
49
+ | Sharp chat template | [`indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja`](./indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja) |
50
+ | SHA-256 checksums | [`indexed88-v6.4-135k-r2/SHA256SUMS`](./indexed88-v6.4-135k-r2/SHA256SUMS) |
51
+ | Release manifest | [`indexed88-v6.4-135k-r2/RELEASE.json`](./indexed88-v6.4-135k-r2/RELEASE.json) |
52
 
53
+ | Property | 135K value |
54
+ | --- | --- |
55
+ | Public vocabulary size | 135,000 |
56
+ | Physical vocabulary rows | 135,168 |
57
+ | Physical Q8 MTP output head | 135,168 × 5,120; Q8 G32 with FP16 scales |
58
+ | Indexed proposal rows | 88,064 (original physical token IDs, not a second tokenizer) |
59
+ | Model artifact size | 15,820,439,556 bytes (≈14.73 GiB) |
60
+ | Ranking SHA-256 | `b578828c0b517ba47abe90a130fda7cde5a56d94add1afeac7623e7c8f6234b4` |
61
+ | Sharp Jinja SHA-256 | `cdff39fb26b60dc90faa292e726655c6b21f62db497846e02e4c4bbab942a84a` |
62
+ | Model SHA-256 | **See uploaded `SHA256SUMS` and `RELEASE.json`**; no unchecked digest is asserted here |
63
+ | Hugging Face upload | [Commit `1e02fcfe5bdd2658be3067e11c4df6b684d5cabe`](https://huggingface.co/Schestex/ThinkingCap-Qwen3.8-27B-NInfer/commit/1e02fcfe5bdd2658be3067e11c4df6b684d5cabe) |
64
 
65
+ The published model file is a distribution copy of the locally named `thinkingcap-135000-q8-repack-EXPERIMENTAL.ninfer`. Its Q8 output head is **one part** of the stored model; the filename does not mean that every layer uses Q8.
66
 
67
+ ### Why Indexed88?
68
+
69
+ The MTP proposal projection uses **one physical Q8 output head**. It projects only the 88,064 selected vocabulary rows, performs argmax in the selected space, and remaps proposals to the original token IDs. There is **no second full MTP-head projection** in the authoritative direct path.
70
+
71
+ The original model vocabulary remains 135,000 public tokens. **88,064 is a proposal-selection size, not the tokenizer size.** The proposal subset is a coverage/performance trade-off and may not contain every token that the full model can produce.
72
+
73
+ ### Vocabulary specialization: English, German, and code
 
 
74
 
75
+ The project aims to progressively specialize the model toward **English, German, programming syntax, structured formats, and coding/agent tasks**.
76
 
77
+ The current 135K vocabulary is an intermediate milestone. It still retains some tokens associated with other European languages. Rather than applying a single radical multilingual cut, vocabulary reduction proceeds in carefully checked stages, protecting special tokens, tokenizer dependencies, code-related tokens, and practical behavior. Each successive cut is intended to be validated before the next one.
78
 
79
+ The **long-term direction** is an increasingly focused English + German + code-oriented vocabulary. That is **a goal, not a claim that all other-language tokens have already been removed**. Vocabulary membership is not the same as a model's knowledge: reducing token coverage does not prove that multilingual knowledge was erased from the transformer weights, nor does it guarantee language quality.
 
 
 
80
 
81
+ Other languages may still function, but multilingual coverage is not the primary optimization target and can change across releases. English and German are the main intended human languages, and specialized coding/agent use is the main workload focus; this is not a formal language-quality benchmark.
82
 
83
+ ### Download and checksum verification
84
+
85
+ Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) if needed and download the complete release directory:
86
 
87
  ```bash
88
  hf download Schestex/ThinkingCap-Qwen3.8-27B-NInfer \
89
  --include 'indexed88-v6.4-135k-r2/*' \
90
  --local-dir ./thinkingcap-ninfer
91
 
92
+ cd thinkingcap-ninfer/indexed88-v6.4-135k-r2
93
+ sha256sum --check SHA256SUMS
94
  ```
95
 
96
+ This verifies the model file, ranked ID file, and template against the manifest produced during local release staging.
97
+
98
+ ### Compatible NInfer source and build
99
+
100
+ The 135K Indexed88 runtime is maintained separately at [Xtravaganz/ninfer](https://github.com/Xtravaganz/ninfer). The matching release tag is:
101
+
102
+ `thinkingcap-indexed88-v6.4-135k-r2`
103
+
104
+ [GitHub source at the release tag](https://github.com/Xtravaganz/ninfer/tree/thinkingcap-indexed88-v6.4-135k-r2) · [GitHub release page](https://github.com/Xtravaganz/ninfer/releases/tag/thinkingcap-indexed88-v6.4-135k-r2)
105
+
106
+ The release tag and GitHub Release page were checked as publicly accessible on November 1, 2026. Generic upstream NInfer builds may not recognize this 135K Indexed88 format.
107
+
108
+ Tested environment: Linux x86-64, NVIDIA RTX PRO 4000 Blackwell (24 GB), CUDA 13.3, `sm_120a`, Release build. Other GPUs/toolkits may require changes and have not been validated here.
109
+
110
+ Example source build (after the tag has been published):
111
+
112
+ ```bash
113
+ git clone https://github.com/Xtravaganz/ninfer.git
114
+ cd ninfer
115
+ git checkout thinkingcap-indexed88-v6.4-135k-r2
116
+ cmake -S . -B build -G Ninja \
117
+ -DCMAKE_BUILD_TYPE=Release \
118
+ -DCMAKE_CUDA_ARCHITECTURES=120a \
119
+ -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc
120
+ cmake --build build --target ninfer-serve --parallel 6
121
+ ```
122
+
123
+ This release's cleaned source snapshot compiled successfully and produced the expected Indexed88 Q8 **R0 reference kernel** resource signature: **80 registers/thread, 17,920 B shared memory/CTA, and zero reported local memory**. The isolated release-build `ninfer-serve` executable SHA-256 was:
124
+
125
+ `b826e03375e41e73ba9417f51097f42f4e79886c1bb357aeea096e5a43ce3446`
126
+
127
+ The local container tag `thinkingcap-v1:indexed88-v6.4-135k-r2` is **not a published image download**. Its recorded status label is `canary`. The cleaned release binary was compiled after source cleanup; the end-to-end benchmark described below was obtained with an earlier, separately isolated R0 build. The hashes are not claimed to be identical.
128
+
129
+ ### Example: single-head Indexed88 MTP5 server
130
 
131
+ Run these commands from a directory containing the downloaded `indexed88-v6.4-135k-r2/` folder, with the compiled `ninfer-serve` path adjusted as appropriate:
132
 
133
  ```bash
 
134
  export NINFER_MTP_INDEXED_SHADOW=1
135
  export NINFER_MTP_INDEXED_MODE=indexed96
136
+ export NINFER_MTP_INDEXED_RANKING="$(realpath indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le)"
137
+ # CUDA_VISIBLE_DEVICES may be set to select a physical GPU;
138
+ # --device 0 then means the first visible GPU.
139
 
140
+ ./build/apps/ninfer-serve \
141
+ "$(realpath indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer)" \
142
  --host 127.0.0.1 --port 8080 \
143
  --model-id thinkingcap-135k-indexed88 \
144
  --device 0 \
 
147
  --max-context 8192 --kv-capacity 8192 \
148
  --max-concurrency 1 --prefill-chunk 512 \
149
  --preserve-thinking \
150
+ --chat-template "$(realpath indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja)"
151
  ```
152
 
153
+ The environment variable `NINFER_MTP_INDEXED_SHADOW=1` is a historical enable switch. **`NINFER_MTP_INDEXED_MODE=indexed96` selects the direct, single-projection path** in this release, even though `indexed96` is the legacy internal mode name for a selection of **88,064 rows**. Do not set the mode to `shadow` for the single-head configuration. No additional Q5/Q6 proposal head is required.
154
 
155
+ A small deterministic HTTP smoke test:
156
 
157
+ ```bash
158
+ curl http://127.0.0.1:8080/v1/chat/completions \
159
+ -H 'Content-Type: application/json' \
160
+ -d '{"model":"thinkingcap-135k-indexed88","messages":[{"role":"user","content":"Write a minimal Python hello-world example."}],"temperature":0,"max_tokens":96}'
161
+ ```
162
+
163
+ ### Indexed88 MTP5 validation
164
+
165
+ An isolated comparison ran R0 (the release reference kernel) against experimental R2 (row-ID hoisting) and R3 (hoisting plus altered minimum-block occupancy), using the **same** 135K model, 88,064-ID ranking, Sharp template, NVFP4 KV cache, GPU1, and five draft tokens. R2/R3 are **not part of this published R0 release**.
166
+
167
+ Each variant generated three consecutive requests of 3,500 output tokens (10,500 tokens per variant). Response hashes matched across variants at corresponding turns, as did draft/accepted counts.
168
+
169
+ | Kernel | Turn 1 tok/s | Turn 2 tok/s | Turn 3 tok/s | Steady turns 2+3 tok/s | Draft acceptance |
170
+ | --- | ---: | ---: | ---: | ---: | ---: |
171
+ | **R0 / released kernel baseline** | **95.34** | **94.75** | **90.44** | **92.55** | **55.33%** |
172
+ | R2 / experimental | 95.19 | 94.66 | 90.42 | 92.49 | 55.33% |
173
+ | R3 / experimental | 95.30 | 94.80 | 90.43 | 92.56 | 55.33% |
174
+
175
+ These are observed wall-clock generation throughputs for a particular test, not generalized model speeds, independent capability scores, or proof of a production-equivalent binary. R0 and R3 were effectively tied end-to-end despite R3's higher kernel occupancy. The acceptance rate changed across turns; it should not be interpreted as a universal acceptance rate for prompts or contexts.
176
+
177
+ ## 147K original V1 — frozen historical release
178
+
179
+ The earlier model remains separately available at [`combined-147590/`](./combined-147590/). Its **147,590 public / 147,712 physical tokens** and **131,072 proposal rows** are independent of the 135K release. V1 was converted directly from the BF16 source checkpoint rather than requantized from another `.ninfer` file.
180
+
181
+ ### Exact V1 artifact
182
+
183
+ ```text
184
+ combined-147590/thinkingcap-v1-upstream-d48dx3p5-oattndx6-ogdndx1-igdndx1-iattndx1-g64dx1-mtpgateupdx15p40-combined-147590.ninfer
185
+ ```
186
+
187
+ | Field | V1 value |
188
+ | --- | --- |
189
+ | File size | 16,308,912,900 bytes / 15.19 GiB |
190
+ | Model SHA-256 | `d030303f73b658399810a1be097c3b682610c47df8ea1cd24eb0c1eb53255b27` |
191
+ | Conversion sidecar | Same filename plus `.conversion.json` |
192
+ | Sidecar SHA-256 | `62eff698b11f1d393b518cf62937f918dc131e6893f65ec9cb57d5c5d8a9843b` |
193
+ | Source tag | [`thinkingcap-upstream-v1-2026-10-05`](https://github.com/Xtravaganz/ninfer/tree/thinkingcap-upstream-v1-2026-10-05) |
194
+ | Validated local production image | `ninfer-upstream-combined-147k:v1` |
195
+ | Build | CUDA 13.1.2 / SM120a / Release |
196
+ | Prefill chunk | **2048** (V1-specific) |
197
+
198
+ #### V1 activation divisor profile
199
+
200
+ | Family | Input divisor |
201
+ | --- | ---: |
202
+ | D48 — MLP down | 3.5 |
203
+ | O_GDN — GDN output | 1.0 |
204
+ | O_ATTN — attention output | 6.0 |
205
+ | I_GDN — GDN input | 1.0 |
206
+ | I_ATTN — attention input | 1.0 |
207
+ | G64 — MLP gate/up | 1.0 |
208
+ | MTP GateUp | 15.40 |
209
+
210
+ Frozen layer-family selections: **D48, O44, I53, G64**. The corresponding activation divisors are baked into the V1 model; no runtime DX override environment variables are required for V1.
211
+
212
+ #### V1 runtime settings
213
+
214
+ ```text
215
+ kv-dtype = nvfp4
216
+ spec = mtp
217
+ draft-tokens = 5
218
+ lm-head-draft = enabled
219
+ prefill-chunk = 2048
220
+ ```
221
 
222
+ These are **V1 settings**, not blanket instructions for the 135K Indexed88 direct release.
223
 
224
+ #### V1 source and binary provenance
225
+
226
+ | Item | Commit or SHA-256 |
227
+ | --- | --- |
228
+ | Published V1 master | `ba8c4e726834c5e00cf5f68255ea6430c66e5319` |
229
+ | Exact V1 tag commit | `883d0e434ecb54cb6d6a9274c6a1a680988fab9d` |
230
+ | Compact-vocabulary Q8 work | `b8685532` |
231
+ | NVFP4 runtime divisor hooks | `883d0e43` |
232
+ | V1 `ninfer` binary SHA-256 | `ee41f5be3a4f6b77148420b1bda8cf8fe12479ad8de9be5f9bcc63539557e7e3` |
233
+ | V1 `ninfer-serve` binary SHA-256 | `2137f5c7d3d54343e5769735ca5103585b53d872ffb8422d3758b45bdf75058d` |
234
+
235
+ The V1 production binary was previously checked as byte-identical to the build from its validated worktree. That statement does not apply to the 135K source cleanup.
236
+
237
+ #### V1 frozen equivalence (runtime-tuned vs baked artifact)
238
+
239
+ | Workload | Tuned acceptance | Baked V1 acceptance |
240
+ | --- | ---: | ---: |
241
+ | Mixed 8K | 0.7509293680 | 0.7509293680 |
242
+ | Mixed 32K | 0.6134185304 | 0.6134185304 |
243
+ | Mixed 48K | 0.5483870968 | 0.5483870968 |
244
+ | Mixed 110K | 0.5864197531 | 0.5864197531 |
245
+
246
+ **V1 baked equivalence: PASS** in those four validation workloads.
247
+
248
+ #### V1 previous-production vs frozen-V1 comparison
249
+
250
+ This was an **end-to-end stack comparison** with changes to runtime, calibration and prefill chunk; it is not an isolated kernel or quantization comparison.
251
+
252
+ | Context | Old acc. | V1 acc. | Old decode tok/s | V1 decode tok/s | Decode gain | Old prefill tok/s | V1 prefill tok/s | Prefill gain |
253
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
254
+ | Mixed 8K | 54.02% | 75.09% | 82.15 | 116.99 | +42.41% | 3390.4 | 3664.2 | +8.07% |
255
+ | Mixed 32K | 40.91% | 61.34% | 64.00 | 93.16 | +45.56% | 2866.7 | 3027.2 | +5.60% |
256
+ | Mixed 48K | 55.33% | 54.84% | 75.78 | 83.33 | +9.96% | 2551.6 | 2707.2 | +6.10% |
257
+ | Mixed 110K | 54.52% | 58.64% | 67.55 | 76.57 | +13.35% | 1798.4 | 1919.2 | +6.72% |
258
+
259
+ #### V1 prefill chunk 1024 vs 2048
260
+
261
+ | Context | Acc. 1024 | Acc. 2048 | Decode gain | Prefill gain |
262
  | --- | ---: | ---: | ---: | ---: |
263
+ | Mixed 32K | 55.29% | 61.34% | +6.28% | +4.89% |
264
+ | Mixed 110K | 43.53% | 58.64% | +22.45% | +4.79% |
 
265
 
266
+ #### V1 vocabulary reduction and validation
267
 
268
+ | Property | Tokens |
269
+ | --- | ---: |
270
+ | Source public vocabulary | 248,077 |
271
+ | V1 public vocabulary | 147,590 |
272
+ | V1 physical vocabulary | 147,712 |
273
+ | Removed | 100,487 (40.51%) |
274
+ | Padding rows | 122 |
275
 
276
+ V1 used conservative Unicode/script classification, workload protection, BPE dependency closure, special/added token protection, tokenizer parity checks, teacher-output audits, and runtime A/B validation. The 131,072-row V1 MTP proposal head was independent of the tokenizer cut.
 
 
 
 
277
 
278
+ The V1 tuning corpus mixed coding and agent-style contexts of approximately 8,192, 32,768, 49,152 and 112,640 tokens. Results on these selected corpora are **not** claims about universal performance or acceptance.
279
 
280
+ ## Limitations and responsible use
 
 
281
 
282
+ - This is a **specialized experimental-to-production inference distribution**. Performance and language coverage vary with workloads, hardware, and runtime versions.
283
+ - English, German and code are the project priorities. Remaining other-language token coverage is not promised stable, and the model can still generate text in additional languages.
284
+ - The 88,064 selected IDs are for **MTP proposals only**; the full model still has 135,000 public tokens.
285
+ - The model can generate inaccurate, insecure or nonworking code. Review and test generated code and tool commands before executing them.
286
+ - Do not confuse the 135K direct-Q8 MTP configuration with the older V1 NVFP4 indexed-head configuration.
287
+ - Model weights are distributed through Hugging Face; the local Docker tag is not a public container registry address.
288
 
289
+ ## Provenance and license
290
 
291
+ - **Upstream base:** [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B).
292
+ - **ThinkingCap fine-tune:** [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B).
293
+ - **NInfer source:** [Xtravaganz/ninfer](https://github.com/Xtravaganz/ninfer); NInfer code under Apache License 2.0, with separate licenses for included dependencies.
294
 
295
+ The ThinkingCap derivative model is subject to **PolyForm Small Business License 1.0.0**, plus BottleCap AI's additional personal noncommercial-use grant. Underlying Qwen materials have Apache-2.0 requirements. Read the upstream [ThinkingCap LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/LICENSE) and [NOTICE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/NOTICE) and retain applicable redistribution notices. A separate commercial license may be required for organizations outside the upstream license terms. This project does not expand the upstream usage permissions.
296
 
297
+ ## Support
298
 
299
+ [Support the project on Buy Me a Coffee](https://buymeacoffee.com/Schestex). Independent work on model optimization, quantization, fine-tuning experiments and efficient CUDA inference is supported by voluntary contributions. Support does not confer commercial rights to the model or promise development outcomes.