Blame for README.md

---
base_model: bottlecapai/ThinkingCap-Qwen3.8-27B
base_model_relation: quantized
library_name: ninfer
pipeline_tag: text-generation
license: other
license_name: polyform-small-business-1.0.0
license_link: LICENSE
tags:
  - qwen3_8
  - ninfer
  - blackwell
  - sm120a
  - mtp
  - speculative-decoding
  - indexed88
  - vocabulary-prune
  - coding
  - agentic
  - nvfp4
  - text-generation
---

# ThinkingCap Qwen3.8-27B for NInfer — 135K Indexed88 & 147K V1

> **Current release: ThinkingCap 135K / Indexed88 v6.4 R2.** The original 147,590-vocabulary V1 is preserved as a separate, frozen release.

NInfer-format exports of [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B), derived from [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B). These releases target specialized coding and agent-style workloads using NVIDIA Blackwell hardware.

**Important:** A `.ninfer` file is **not** a GGUF, Safetensors, or standard Hugging Face Transformers checkpoint. Use the matching [NInfer runtime](https://github.com/Xtravaganz/ninfer). This repository contains *model artifacts*, not a downloadable Docker image.

## Release overview

| Release | Public tokenizer | Physical rows | MTP proposal rows | Runtime | Status |
| --- | ---: | ---: | ---: | --- | --- |
| **135K Indexed88 v6.4 R2** | **135,000** | **135,168** | **88,064** | Indexed88 direct Q8, MTP5 | Current release; local deployment image labeled `canary` |
| **Combined 147K V1** | 147,590 | 147,712 | 131,072 | Original V1 MTP5 | Frozen / historically production validated |

**Do not mix** the model, tokenizer, physical-vocabulary layout, selection IDs, or runtime release across these versions. The `R2` suffix of the 135K release denotes the *Indexed88 ranking/release revision*; it does **not** mean the experimental R2 row-ID-hoisting kernel.

## 135K Indexed88 v6.4 R2 — current release

### Artifacts

| Item | Repository path |
| --- | --- |
| NInfer model | [`indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer`](./indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer) |
| Sorted selected token IDs | [`indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le`](./indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le) |
| Sharp chat template | [`indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja`](./indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja) |
| SHA-256 checksums | [`indexed88-v6.4-135k-r2/SHA256SUMS`](./indexed88-v6.4-135k-r2/SHA256SUMS) |
| Release manifest | [`indexed88-v6.4-135k-r2/RELEASE.json`](./indexed88-v6.4-135k-r2/RELEASE.json) |

| Property | 135K value |
| --- | --- |
| Public vocabulary size | 135,000 |
| Physical vocabulary rows | 135,168 |
| Physical Q8 MTP output head | 135,168 × 5,120; Q8 G32 with FP16 scales |
| Indexed proposal rows | 88,064 (original physical token IDs, not a second tokenizer) |
| Model artifact size | 15,820,439,556 bytes (≈14.73 GiB) |
| Ranking SHA-256 | `b578828c0b517ba47abe90a130fda7cde5a56d94add1afeac7623e7c8f6234b4` |
| Sharp Jinja SHA-256 | `cdff39fb26b60dc90faa292e726655c6b21f62db497846e02e4c4bbab942a84a` |
| Model SHA-256 | **See uploaded `SHA256SUMS` and `RELEASE.json`**; no unchecked digest is asserted here |
| Hugging Face upload | [Commit `1e02fcfe5bdd2658be3067e11c4df6b684d5cabe`](https://huggingface.co/Schestex/ThinkingCap-Qwen3.8-27B-NInfer/commit/1e02fcfe5bdd2658be3067e11c4df6b684d5cabe) |

The published model file is a distribution copy of the locally named `thinkingcap-135000-q8-repack-EXPERIMENTAL.ninfer`. Its Q8 output head is **one part** of the stored model; the filename does not mean that every layer uses Q8.

### Why Indexed88?

The MTP proposal projection uses **one physical Q8 output head**. It projects only the 88,064 selected vocabulary rows, performs argmax in the selected space, and remaps proposals to the original token IDs. There is **no second full MTP-head projection** in the authoritative direct path.

The original model vocabulary remains 135,000 public tokens. **88,064 is a proposal-selection size, not the tokenizer size.** The proposal subset is a coverage/performance trade-off and may not contain every token that the full model can produce.

### Vocabulary specialization: English, German, and code

The project aims to progressively specialize the model toward **English, German, programming syntax, structured formats, and coding/agent tasks**.

The current 135K vocabulary is an intermediate milestone. It still retains some tokens associated with other European languages. Rather than applying a single radical multilingual cut, vocabulary reduction proceeds in carefully checked stages, protecting special tokens, tokenizer dependencies, code-related tokens, and practical behavior. Each successive cut is intended to be validated before the next one.

The **long-term direction** is an increasingly focused English + German + code-oriented vocabulary. That is **a goal, not a claim that all other-language tokens have already been removed**. Vocabulary membership is not the same as a model's knowledge: reducing token coverage does not prove that multilingual knowledge was erased from the transformer weights, nor does it guarantee language quality.

Other languages may still function, but multilingual coverage is not the primary optimization target and can change across releases. English and German are the main intended human languages, and specialized coding/agent use is the main workload focus; this is not a formal language-quality benchmark.

### Download and checksum verification

Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) if needed and download the complete release directory:

```bash
hf download Schestex/ThinkingCap-Qwen3.8-27B-NInfer \
  --include 'indexed88-v6.4-135k-r2/*' \
  --local-dir ./thinkingcap-ninfer

cd thinkingcap-ninfer/indexed88-v6.4-135k-r2
sha256sum --check SHA256SUMS
```

This verifies the model file, ranked ID file, and template against the manifest produced during local release staging.

### Compatible NInfer source and build

The 135K Indexed88 runtime is maintained separately at [Xtravaganz/ninfer](https://github.com/Xtravaganz/ninfer). The matching release tag is:

`thinkingcap-indexed88-v6.4-135k-r2`

[GitHub source at the release tag](https://github.com/Xtravaganz/ninfer/tree/thinkingcap-indexed88-v6.4-135k-r2) · [GitHub release page](https://github.com/Xtravaganz/ninfer/releases/tag/thinkingcap-indexed88-v6.4-135k-r2)

The release tag and GitHub Release page were checked as publicly accessible on November 1, 2026. Generic upstream NInfer builds may not recognize this 135K Indexed88 format.

Tested environment: Linux x86-64, NVIDIA RTX PRO 4000 Blackwell (24 GB), CUDA 13.3, `sm_120a`, Release build. Other GPUs/toolkits may require changes and have not been validated here.

Example source build (after the tag has been published):

```bash
git clone https://github.com/Xtravaganz/ninfer.git
cd ninfer
git checkout thinkingcap-indexed88-v6.4-135k-r2
cmake -S . -B build -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CUDA_ARCHITECTURES=120a \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc
cmake --build build --target ninfer-serve --parallel 6
```

This release's cleaned source snapshot compiled successfully and produced the expected Indexed88 Q8 **R0 reference kernel** resource signature: **80 registers/thread, 17,920 B shared memory/CTA, and zero reported local memory**. The isolated release-build `ninfer-serve` executable SHA-256 was:

`b826e03375e41e73ba9417f51097f42f4e79886c1bb357aeea096e5a43ce3446`

The local container tag `thinkingcap-v1:indexed88-v6.4-135k-r2` is **not a published image download**. Its recorded status label is `canary`. The cleaned release binary was compiled after source cleanup; the end-to-end benchmark described below was obtained with an earlier, separately isolated R0 build. The hashes are not claimed to be identical.

### Example: single-head Indexed88 MTP5 server

Run these commands from a directory containing the downloaded `indexed88-v6.4-135k-r2/` folder, with the compiled `ninfer-serve` path adjusted as appropriate:

```bash
export NINFER_MTP_INDEXED_SHADOW=1
export NINFER_MTP_INDEXED_MODE=indexed96
export NINFER_MTP_INDEXED_RANKING="$(realpath indexed88-v6.4-135k-r2/selected_88064_135k_v64_40_R2.i32le)"
# CUDA_VISIBLE_DEVICES may be set to select a physical GPU;
# --device 0 then means the first visible GPU.

./build/apps/ninfer-serve \
  "$(realpath indexed88-v6.4-135k-r2/thinkingcap-135000-q8-repack.ninfer)" \
  --host 127.0.0.1 --port 8080 \
  --model-id thinkingcap-135k-indexed88 \
  --device 0 \
  --spec mtp --draft-tokens 5 \
  --kv-dtype nvfp4 \
  --max-context 8192 --kv-capacity 8192 \
  --max-concurrency 1 --prefill-chunk 512 \
  --preserve-thinking \
  --chat-template "$(realpath indexed88-v6.4-135k-r2/qwen3_8_sharp.jinja)"
```

The environment variable `NINFER_MTP_INDEXED_SHADOW=1` is a historical enable switch. **`NINFER_MTP_INDEXED_MODE=indexed96` selects the direct, single-projection path** in this release, even though `indexed96` is the legacy internal mode name for a selection of **88,064 rows**. Do not set the mode to `shadow` for the single-head configuration. No additional Q5/Q6 proposal head is required.

A small deterministic HTTP smoke test:

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"thinkingcap-135k-indexed88","messages":[{"role":"user","content":"Write a minimal Python hello-world example."}],"temperature":0,"max_tokens":96}'
```

### Indexed88 MTP5 validation

An isolated comparison ran R0 (the release reference kernel) against experimental R2 (row-ID hoisting) and R3 (hoisting plus altered minimum-block occupancy), using the **same** 135K model, 88,064-ID ranking, Sharp template, NVFP4 KV cache, GPU1, and five draft tokens. R2/R3 are **not part of this published R0 release**.

Each variant generated three consecutive requests of 3,500 output tokens (10,500 tokens per variant). Response hashes matched across variants at corresponding turns, as did draft/accepted counts.

| Kernel | Turn 1 tok/s | Turn 2 tok/s | Turn 3 tok/s | Steady turns 2+3 tok/s | Draft acceptance |
| --- | ---: | ---: | ---: | ---: | ---: |
| **R0 / released kernel baseline** | **95.34** | **94.75** | **90.44** | **92.55** | **55.33%** |
| R2 / experimental | 95.19 | 94.66 | 90.42 | 92.49 | 55.33% |
| R3 / experimental | 95.30 | 94.80 | 90.43 | 92.56 | 55.33% |

These are observed wall-clock generation throughputs for a particular test, not generalized model speeds, independent capability scores, or proof of a production-equivalent binary. R0 and R3 were effectively tied end-to-end despite R3's higher kernel occupancy. The acceptance rate changed across turns; it should not be interpreted as a universal acceptance rate for prompts or contexts.

## 147K original V1 — frozen historical release

The earlier model remains separately available at [`combined-147590/`](./combined-147590/). Its **147,590 public / 147,712 physical tokens** and **131,072 proposal rows** are independent of the 135K release. V1 was converted directly from the BF16 source checkpoint rather than requantized from another `.ninfer` file.

### Exact V1 artifact

```text
combined-147590/thinkingcap-v1-upstream-d48dx3p5-oattndx6-ogdndx1-igdndx1-iattndx1-g64dx1-mtpgateupdx15p40-combined-147590.ninfer
```

| Field | V1 value |
| --- | --- |
| File size | 16,308,912,900 bytes / 15.19 GiB |
| Model SHA-256 | `d030303f73b658399810a1be097c3b682610c47df8ea1cd24eb0c1eb53255b27` |
| Conversion sidecar | Same filename plus `.conversion.json` |
| Sidecar SHA-256 | `62eff698b11f1d393b518cf62937f918dc131e6893f65ec9cb57d5c5d8a9843b` |
| Source tag | [`thinkingcap-upstream-v1-2026-10-05`](https://github.com/Xtravaganz/ninfer/tree/thinkingcap-upstream-v1-2026-10-05) |
| Validated local production image | `ninfer-upstream-combined-147k:v1` |
| Build | CUDA 13.1.2 / SM120a / Release |
| Prefill chunk | **2048** (V1-specific) |

#### V1 activation divisor profile

| Family | Input divisor |
| --- | ---: |
| D48 — MLP down | 3.5 |
| O_GDN — GDN output | 1.0 |
| O_ATTN — attention output | 6.0 |
| I_GDN — GDN input | 1.0 |
| I_ATTN — attention input | 1.0 |
| G64 — MLP gate/up | 1.0 |
| MTP GateUp | 15.40 |

Frozen layer-family selections: **D48, O44, I53, G64**. The corresponding activation divisors are baked into the V1 model; no runtime DX override environment variables are required for V1.

#### V1 runtime settings

```text
kv-dtype      = nvfp4
spec          = mtp
draft-tokens  = 5
lm-head-draft = enabled
prefill-chunk = 2048
```

These are **V1 settings**, not blanket instructions for the 135K Indexed88 direct release.

#### V1 source and binary provenance

| Item | Commit or SHA-256 |
| --- | --- |
| Published V1 master | `ba8c4e726834c5e00cf5f68255ea6430c66e5319` |
| Exact V1 tag commit | `883d0e434ecb54cb6d6a9274c6a1a680988fab9d` |
| Compact-vocabulary Q8 work | `b8685532` |
| NVFP4 runtime divisor hooks | `883d0e43` |
| V1 `ninfer` binary SHA-256 | `ee41f5be3a4f6b77148420b1bda8cf8fe12479ad8de9be5f9bcc63539557e7e3` |
| V1 `ninfer-serve` binary SHA-256 | `2137f5c7d3d54343e5769735ca5103585b53d872ffb8422d3758b45bdf75058d` |

The V1 production binary was previously checked as byte-identical to the build from its validated worktree. That statement does not apply to the 135K source cleanup.

#### V1 frozen equivalence (runtime-tuned vs baked artifact)

| Workload | Tuned acceptance | Baked V1 acceptance |
| --- | ---: | ---: |
| Mixed 8K | 0.7509293680 | 0.7509293680 |
| Mixed 32K | 0.6134185304 | 0.6134185304 |
| Mixed 48K | 0.5483870968 | 0.5483870968 |
| Mixed 110K | 0.5864197531 | 0.5864197531 |

**V1 baked equivalence: PASS** in those four validation workloads.

#### V1 previous-production vs frozen-V1 comparison

This was an **end-to-end stack comparison** with changes to runtime, calibration and prefill chunk; it is not an isolated kernel or quantization comparison.

| Context | Old acc. | V1 acc. | Old decode tok/s | V1 decode tok/s | Decode gain | Old prefill tok/s | V1 prefill tok/s | Prefill gain |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Mixed 8K | 54.02% | 75.09% | 82.15 | 116.99 | +42.41% | 3390.4 | 3664.2 | +8.07% |
| Mixed 32K | 40.91% | 61.34% | 64.00 | 93.16 | +45.56% | 2866.7 | 3027.2 | +5.60% |
| Mixed 48K | 55.33% | 54.84% | 75.78 | 83.33 | +9.96% | 2551.6 | 2707.2 | +6.10% |
| Mixed 110K | 54.52% | 58.64% | 67.55 | 76.57 | +13.35% | 1798.4 | 1919.2 | +6.72% |

#### V1 prefill chunk 1024 vs 2048

| Context | Acc. 1024 | Acc. 2048 | Decode gain | Prefill gain |
| --- | ---: | ---: | ---: | ---: |
| Mixed 32K | 55.29% | 61.34% | +6.28% | +4.89% |
| Mixed 110K | 43.53% | 58.64% | +22.45% | +4.79% |

#### V1 vocabulary reduction and validation

| Property | Tokens |
| --- | ---: |
| Source public vocabulary | 248,077 |
| V1 public vocabulary | 147,590 |
| V1 physical vocabulary | 147,712 |
| Removed | 100,487 (40.51%) |
| Padding rows | 122 |

V1 used conservative Unicode/script classification, workload protection, BPE dependency closure, special/added token protection, tokenizer parity checks, teacher-output audits, and runtime A/B validation. The 131,072-row V1 MTP proposal head was independent of the tokenizer cut.

The V1 tuning corpus mixed coding and agent-style contexts of approximately 8,192, 32,768, 49,152 and 112,640 tokens. Results on these selected corpora are **not** claims about universal performance or acceptance.

## Limitations and responsible use

- This is a **specialized experimental-to-production inference distribution**. Performance and language coverage vary with workloads, hardware, and runtime versions.
- English, German and code are the project priorities. Remaining other-language token coverage is not promised stable, and the model can still generate text in additional languages.
- The 88,064 selected IDs are for **MTP proposals only**; the full model still has 135,000 public tokens.
- The model can generate inaccurate, insecure or nonworking code. Review and test generated code and tool commands before executing them.
- Do not confuse the 135K direct-Q8 MTP configuration with the older V1 NVFP4 indexed-head configuration.
- Model weights are distributed through Hugging Face; the local Docker tag is not a public container registry address.

## Provenance and license

- **Upstream base:** [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B).
- **ThinkingCap fine-tune:** [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B).
- **NInfer source:** [Xtravaganz/ninfer](https://github.com/Xtravaganz/ninfer); NInfer code under Apache License 2.0, with separate licenses for included dependencies.

The ThinkingCap derivative model is subject to **PolyForm Small Business License 1.0.0**, plus BottleCap AI's additional personal noncommercial-use grant. Underlying Qwen materials have Apache-2.0 requirements. Read the upstream [ThinkingCap LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/LICENSE) and [NOTICE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B/blob/main/NOTICE) and retain applicable redistribution notices. A separate commercial license may be required for organizations outside the upstream license terms. This project does not expand the upstream usage permissions.

## Support

[Support the project on Buy Me a Coffee](https://buymeacoffee.com/Schestex). Independent work on model optimization, quantization, fine-tuning experiments and efficient CUDA inference is supported by voluntary contributions. Support does not confer commercial rights to the model or promise development outcomes.