Elevatormusic commited on
Commit
d44fa51
·
1 Parent(s): db0fa29

docs: add an experimental BF16 uncensored option (#1)

Browse files

- docs: add an experimental BF16 uncensored option (73936dfd442fac196ab1108dbabbb9e43335e992)

Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -25,6 +25,8 @@ tags:
25
 
26
  Measured on 17 and 18 October 2026 on one pair of DGX Spark. Every number on this page comes from a saved run in the [GitHub recipe](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark). The engine is [my TensorFold fork](https://github.com/Elevatormusic/TensorFold) (branch `deepseek-v41-tp2`).
27
 
 
 
28
  ---
29
 
30
  ## At a glance
@@ -74,6 +76,18 @@ The Engram tables of the 1017 release are new. Do not use the tables of the olde
74
 
75
  ---
76
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  ## What you need
78
 
79
  - **Two NVIDIA DGX Spark**, joined by one QSFP cable between their ConnectX-7 network ports.
@@ -205,5 +219,6 @@ The scripts, tests and documents in the recipe are Apache-2.0. The model, the ch
205
  | Recipe scripts, tests and documents | Apache-2.0 |
206
  | DeepSeek-V4.1-Flash-1017 (the model) | MIT |
207
  | LibertAIDAI EXL3 2.9 bpw weights | MIT |
 
208
  | TensorFold and the V4.1 engine | Apache-2.0 |
209
  | MiaAI-Lab vLLM kit and chat template | AGPL-3.0 |
 
25
 
26
  Measured on 17 and 18 October 2026 on one pair of DGX Spark. Every number on this page comes from a saved run in the [GitHub recipe](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark). The engine is [my TensorFold fork](https://github.com/Elevatormusic/TensorFold) (branch `deepseek-v41-tp2`).
27
 
28
+ The speed results and the full 1M-token request test below apply to the base recipe. They do not apply to the optional uncensored BF16 overrides.
29
+
30
  ---
31
 
32
  ## At a glance
 
76
 
77
  ---
78
 
79
+ ## Optional uncensored BF16 overrides (experimental)
80
+
81
+ This option adds 54 original dense BF16 matrices from the uncensored source to the base EXL3 model. It replaces `wo_b` and shared-expert `w2` in layers 10 through 36. It is not an all-BF16 checkpoint.
82
+
83
+ The source is [dealignai/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-UNCENSORED](https://huggingface.co/dealignai/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-UNCENSORED/tree/89af9bf193575b7eeeb4f1dd85463ec24ea5bb4c), pinned to `89af9bf193575b7eeeb4f1dd85463ec24ea5bb4c`. See [recipe PR #14](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/pull/14) and its [optional recipe instructions](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/00cf8f325fae83ab5805bb1d0a364ae696bde247/docs/UNCENSORED.md). This option needs the [engine change](https://github.com/Elevatormusic/TensorFold/pull/8) at `dccc4a997b7c18fc684781ee333273f8d902c510`. These pull requests are not merged. The default engine image does not support these overrides. The quick start below still uses the base model.
84
+
85
+ Real replies passed with 8K-token and 64K-token prompts and with four concurrent requests. The server was configured for a 1M-token context, but no full 1M-token request was tested with these overrides. Both nodes logged `NV_ERR_NO_MEMORY` warnings while the API requests completed. No OOM kill was observed. This does not establish that memory failures are fixed.
86
+
87
+ The 8 bpw override test produced a repetition loop. No decode-speed benefit was established, so BF16 is the documented option. These checks do not establish model quality or an uncensored benchmark score. See the [saved test limits](https://github.com/Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark/blob/00cf8f325fae83ab5805bb1d0a364ae696bde247/runs/2026-10-19/uncensored-bf16/service-check.json). That summary records an earlier local engine, not GPU qualification of the new engine PR head or preparation wrapper.
88
+
89
+ ---
90
+
91
  ## What you need
92
 
93
  - **Two NVIDIA DGX Spark**, joined by one QSFP cable between their ConnectX-7 network ports.
 
219
  | Recipe scripts, tests and documents | Apache-2.0 |
220
  | DeepSeek-V4.1-Flash-1017 (the model) | MIT |
221
  | LibertAIDAI EXL3 2.9 bpw weights | MIT |
222
+ | Optional dealignai BF16 overrides | [DeepSeek V4.1 Flash 1017 licence](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/blob/main/LICENSE) |
223
  | TensorFold and the V4.1 engine | Apache-2.0 |
224
  | MiaAI-Lab vLLM kit and chat template | AGPL-3.0 |