Commit ·
d44fa51
1
Parent(s): db0fa29
docs: add an experimental BF16 uncensored option (#1)
Browse files- docs: add an experimental BF16 uncensored option (73936dfd442fac196ab1108dbabbb9e43335e992)
README.md
CHANGED
|
@@ -25,6 +25,8 @@ tags:
|
|
| 25 |
|
| 26 |
Measured on 17 and 18 October 2026 on one pair of DGX Spark. Every number on this page comes from a saved run in the [GitHub recipe](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark). The engine is [my TensorFold fork](https://github.com/Elevatormusic/TensorFold) (branch `deepseek-v41-tp2`).
|
| 27 |
|
|
|
|
|
|
|
| 28 |
---
|
| 29 |
|
| 30 |
## At a glance
|
|
@@ -74,6 +76,18 @@ The Engram tables of the 1017 release are new. Do not use the tables of the olde
|
|
| 74 |
|
| 75 |
---
|
| 76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
## What you need
|
| 78 |
|
| 79 |
- **Two NVIDIA DGX Spark**, joined by one QSFP cable between their ConnectX-7 network ports.
|
|
@@ -205,5 +219,6 @@ The scripts, tests and documents in the recipe are Apache-2.0. The model, the ch
|
|
| 205 |
| Recipe scripts, tests and documents | Apache-2.0 |
|
| 206 |
| DeepSeek-V4.1-Flash-1017 (the model) | MIT |
|
| 207 |
| LibertAIDAI EXL3 2.9 bpw weights | MIT |
|
|
|
|
| 208 |
| TensorFold and the V4.1 engine | Apache-2.0 |
|
| 209 |
| MiaAI-Lab vLLM kit and chat template | AGPL-3.0 |
|
|
|
|
| 25 |
|
| 26 |
Measured on 17 and 18 October 2026 on one pair of DGX Spark. Every number on this page comes from a saved run in the [GitHub recipe](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark). The engine is [my TensorFold fork](https://github.com/Elevatormusic/TensorFold) (branch `deepseek-v41-tp2`).
|
| 27 |
|
| 28 |
+
The speed results and the full 1M-token request test below apply to the base recipe. They do not apply to the optional uncensored BF16 overrides.
|
| 29 |
+
|
| 30 |
---
|
| 31 |
|
| 32 |
## At a glance
|
|
|
|
| 76 |
|
| 77 |
---
|
| 78 |
|
| 79 |
+
## Optional uncensored BF16 overrides (experimental)
|
| 80 |
+
|
| 81 |
+
This option adds 54 original dense BF16 matrices from the uncensored source to the base EXL3 model. It replaces `wo_b` and shared-expert `w2` in layers 10 through 36. It is not an all-BF16 checkpoint.
|
| 82 |
+
|
| 83 |
+
The source is [dealignai/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-UNCENSORED](https://huggingface.co/dealignai/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-UNCENSORED/tree/89af9bf193575b7eeeb4f1dd85463ec24ea5bb4c), pinned to `89af9bf193575b7eeeb4f1dd85463ec24ea5bb4c`. See [recipe PR #14](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/pull/14) and its [optional recipe instructions](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/00cf8f325fae83ab5805bb1d0a364ae696bde247/docs/UNCENSORED.md). This option needs the [engine change](https://github.com/Elevatormusic/TensorFold/pull/8) at `dccc4a997b7c18fc684781ee333273f8d902c510`. These pull requests are not merged. The default engine image does not support these overrides. The quick start below still uses the base model.
|
| 84 |
+
|
| 85 |
+
Real replies passed with 8K-token and 64K-token prompts and with four concurrent requests. The server was configured for a 1M-token context, but no full 1M-token request was tested with these overrides. Both nodes logged `NV_ERR_NO_MEMORY` warnings while the API requests completed. No OOM kill was observed. This does not establish that memory failures are fixed.
|
| 86 |
+
|
| 87 |
+
The 8 bpw override test produced a repetition loop. No decode-speed benefit was established, so BF16 is the documented option. These checks do not establish model quality or an uncensored benchmark score. See the [saved test limits](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/00cf8f325fae83ab5805bb1d0a364ae696bde247/runs/2026-10-19/uncensored-bf16/service-check.json). That summary records an earlier local engine, not GPU qualification of the new engine PR head or preparation wrapper.
|
| 88 |
+
|
| 89 |
+
---
|
| 90 |
+
|
| 91 |
## What you need
|
| 92 |
|
| 93 |
- **Two NVIDIA DGX Spark**, joined by one QSFP cable between their ConnectX-7 network ports.
|
|
|
|
| 219 |
| Recipe scripts, tests and documents | Apache-2.0 |
|
| 220 |
| DeepSeek-V4.1-Flash-1017 (the model) | MIT |
|
| 221 |
| LibertAIDAI EXL3 2.9 bpw weights | MIT |
|
| 222 |
+
| Optional dealignai BF16 overrides | [DeepSeek V4.1 Flash 1017 licence](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/blob/main/LICENSE) |
|
| 223 |
| TensorFold and the V4.1 engine | Apache-2.0 |
|
| 224 |
| MiaAI-Lab vLLM kit and chat template | AGPL-3.0 |
|