Text Generation
NInfer
qwen.83
qwen3_8
blackwell
sm120a
mtp
speculative-decoding
indexed88
vocabulary-prune
coding
agentic
nvfp4
conversational
Instructions to use Schestex/ThinkingCap-Qwen3.8-27B-NInfer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NInfer
How to use Schestex/ThinkingCap-Qwen3.8-27B-NInfer with NInfer:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -335,10 +335,37 @@ I44 is designed to retain much of that performance benefit while excluding the i
|
|
| 335 |
### Current I44 benchmark status
|
| 336 |
|
| 337 |
```text
|
| 338 |
-
D48 + O44 + I44 performance
|
| 339 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 340 |
|
| 341 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 342 |
|
| 343 |
---
|
| 344 |
|
|
|
|
| 335 |
### Current I44 benchmark status
|
| 336 |
|
| 337 |
```text
|
| 338 |
+
## D48 + O44 + I44 performance
|
| 339 |
+
|
| 340 |
+
The I44 candidate was benchmarked against D48 + O44 and the performance-only I64 ceiling on the same physical RTX PRO 4000 Blackwell GPU.
|
| 341 |
+
|
| 342 |
+
| Test | Metric | D48 + O44 | I44 | I44 gain | I64 ceiling | Ceiling retained |
|
| 343 |
+
|---|---|---:|---:|---:|---:|---:|
|
| 344 |
+
| pp512 | prefill | 840.81 | 973.42 | +15.77% | 1047.02 | 64.3% |
|
| 345 |
+
| pp2048 | prefill | 850.04 | 998.64 | +17.48% | 1086.12 | 62.9% |
|
| 346 |
+
| pp8192 | prefill | 790.06 | 945.86 | +19.72% | 1029.31 | 65.1% |
|
| 347 |
+
| tg256 | decode | 19.06 | 20.52 | +7.68% | 21.19 | 68.5% |
|
| 348 |
+
| tg1024 | decode | 18.72 | 19.78 | +5.64% | 20.33 | 65.5% |
|
| 349 |
+
| pp2048+tg512 | prefill | 765.80 | 899.07 | +17.40% | 988.97 | 59.7% |
|
| 350 |
+
| pp2048+tg512 | decode | 18.70 | 19.68 | +5.23% | 20.37 | 58.7% |
|
| 351 |
+
| pp8192+tg512 | prefill | 754.14 | 886.13 | +17.50% | 964.10 | 62.9% |
|
| 352 |
+
| pp8192+tg512 | decode | 18.52 | 19.40 | +4.74% | 20.08 | 56.3% |
|
| 353 |
+
|
| 354 |
+
Summary:
|
| 355 |
|
| 356 |
+
- **Mean I44 prefill gain:** +17.576%
|
| 357 |
+
- **Mean I44 decode gain:** +5.822%
|
| 358 |
+
- **Mean prefill I64 ceiling retained:** 63.0%
|
| 359 |
+
- **Mean decode I64 ceiling retained:** 62.2%
|
| 360 |
+
|
| 361 |
+
Artifact details:
|
| 362 |
+
|
| 363 |
+
```text
|
| 364 |
+
artifact bytes = 16,371,739,140
|
| 365 |
+
runtime weights = 16,358,539,264
|
| 366 |
+
workspace = 159,981,568
|
| 367 |
+
SHA256 = 09babb880c5d0ea112005ec6701ab7e6a074683318c0ac47cefbe60b80d55040
|
| 368 |
+
```
|
| 369 |
|
| 370 |
---
|
| 371 |
|