Model card: no internal labels, no language note
Browse files
README.md
CHANGED
|
@@ -35,7 +35,7 @@ These are the weights we run DeepSeek V4.1 Flash 1017 with on one home PC. The m
|
|
| 35 |
- The experts are quantized with **GPTQ** on our own calibration texts, not with round-to-nearest. Every routed-expert layer is covered.
|
| 36 |
- More bits go to the most sensitive parts: the down projections, and the first and last four layers.
|
| 37 |
- Engram is kept at the official FP8 precision.
|
| 38 |
-
- We measure quality as the **distance to the full model**: the extra negative log-likelihood, in nat/token, of the full model's own (API) continuations, on three prompt sets
|
| 39 |
- `og` exact: 0.090 / 0.051 / 0.069.
|
| 40 |
- With the engine's default speed rules: 0.091 / 0.054 / 0.072.
|
| 41 |
- HumanEval: 160 / 164.
|
|
@@ -45,13 +45,13 @@ These are the weights we run DeepSeek V4.1 Flash 1017 with on one home PC. The m
|
|
| 45 |
|
| 46 |
1. Build the fork from the patches in the [GitHub repository](https://github.com/MasherDon/Siege-Ladder). It is tested on Windows with CUDA 13 on an RTX 50 card (sm_120).
|
| 47 |
2. Pass `-m .../DeepSeek-V4.1-Flash-MXFP4-engram-00001-of-00011.gguf`. The other parts load by themselves.
|
| 48 |
-
3. Set up the lens (
|
| 49 |
|
| 50 |
Our PC: RTX 5070 Ti 16 GB on PCIe 4.0 x16, Ryzen 5 7600, 64 GB DDR5, one PCIe 5.0 NVMe (plus an optional expert mirror on a second NVMe).
|
| 51 |
|
| 52 |
| | tok/s |
|
| 53 |
|---|---|
|
| 54 |
-
| decode, short context | 18.44–18.80 through the server (6 November); 21.30 teacher-forced
|
| 55 |
| reading a 246K-token document (256K context) | 1128 (6 November, mean of two runs) |
|
| 56 |
| reading a 479K-token document (512K context) | 1043 (6 November) |
|
| 57 |
| reading a 947K-token document (1M context) | 888 (6 November) |
|
|
|
|
| 35 |
- The experts are quantized with **GPTQ** on our own calibration texts, not with round-to-nearest. Every routed-expert layer is covered.
|
| 36 |
- More bits go to the most sensitive parts: the down projections, and the first and last four layers.
|
| 37 |
- Engram is kept at the official FP8 precision.
|
| 38 |
+
- We measure quality as the **distance to the full model**: the extra negative log-likelihood, in nat/token, of the full model's own (API) continuations, on three prompt sets. Smaller is better.
|
| 39 |
- `og` exact: 0.090 / 0.051 / 0.069.
|
| 40 |
- With the engine's default speed rules: 0.091 / 0.054 / 0.072.
|
| 41 |
- HumanEval: 160 / 164.
|
|
|
|
| 45 |
|
| 46 |
1. Build the fork from the patches in the [GitHub repository](https://github.com/MasherDon/Siege-Ladder). It is tested on Windows with CUDA 13 on an RTX 50 card (sm_120).
|
| 47 |
2. Pass `-m .../DeepSeek-V4.1-Flash-MXFP4-engram-00001-of-00011.gguf`. The other parts load by themselves.
|
| 48 |
+
3. Set up the lens (on by default). The engine looks for it in a folder named `g44` *next to* the model's folder, not inside it. After downloading this repository, move its `g44` folder one level up, or set `LLAMA_MOE_STREAM_G44` to the path of `1017-lens-d2.gguf`. Without the lens the engine still runs, but decodes slower.
|
| 49 |
|
| 50 |
Our PC: RTX 5070 Ti 16 GB on PCIe 4.0 x16, Ryzen 5 7600, 64 GB DDR5, one PCIe 5.0 NVMe (plus an optional expert mirror on a second NVMe).
|
| 51 |
|
| 52 |
| | tok/s |
|
| 53 |
|---|---|
|
| 54 |
+
| decode, short context | 18.44–18.80 through the server (6 November); 21.30 teacher-forced (4 November) |
|
| 55 |
| reading a 246K-token document (256K context) | 1128 (6 November, mean of two runs) |
|
| 56 |
| reading a 479K-token document (512K context) | 1043 (6 November) |
|
| 57 |
| reading a 947K-token document (1M context) | 888 (6 November) |
|