MasherDon commited on
Commit
ad1032e
·
verified ·
1 Parent(s): 4e1b024

Model card: no internal labels, no language note

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -35,7 +35,7 @@ These are the weights we run DeepSeek V4.1 Flash 1017 with on one home PC. The m
35
  - The experts are quantized with **GPTQ** on our own calibration texts, not with round-to-nearest. Every routed-expert layer is covered.
36
  - More bits go to the most sensitive parts: the down projections, and the first and last four layers.
37
  - Engram is kept at the official FP8 precision.
38
- - We measure quality as the **distance to the full model**: the extra negative log-likelihood, in nat/token, of the full model's own (API) continuations, on three prompt sets (general, Russian, router). Smaller is better.
39
  - `og` exact: 0.090 / 0.051 / 0.069.
40
  - With the engine's default speed rules: 0.091 / 0.054 / 0.072.
41
  - HumanEval: 160 / 164.
@@ -45,13 +45,13 @@ These are the weights we run DeepSeek V4.1 Flash 1017 with on one home PC. The m
45
 
46
  1. Build the fork from the patches in the [GitHub repository](https://github.com/MasherDon/Siege-Ladder). It is tested on Windows with CUDA 13 on an RTX 50 card (sm_120).
47
  2. Pass `-m .../DeepSeek-V4.1-Flash-MXFP4-engram-00001-of-00011.gguf`. The other parts load by themselves.
48
- 3. Set up the lens (G44 / G40, on by default). The engine looks for it in a folder named `g44` *next to* the model's folder, not inside it. After downloading this repository, move its `g44` folder one level up, or set `LLAMA_MOE_STREAM_G44` to the path of `1017-lens-d2.gguf`. Without the lens the engine still runs, but decodes slower.
49
 
50
  Our PC: RTX 5070 Ti 16 GB on PCIe 4.0 x16, Ryzen 5 7600, 64 GB DDR5, one PCIe 5.0 NVMe (plus an optional expert mirror on a second NVMe).
51
 
52
  | | tok/s |
53
  |---|---|
54
- | decode, short context | 18.44–18.80 through the server (6 November); 21.30 teacher-forced with G44 + G40 (4 November) |
55
  | reading a 246K-token document (256K context) | 1128 (6 November, mean of two runs) |
56
  | reading a 479K-token document (512K context) | 1043 (6 November) |
57
  | reading a 947K-token document (1M context) | 888 (6 November) |
 
35
  - The experts are quantized with **GPTQ** on our own calibration texts, not with round-to-nearest. Every routed-expert layer is covered.
36
  - More bits go to the most sensitive parts: the down projections, and the first and last four layers.
37
  - Engram is kept at the official FP8 precision.
38
+ - We measure quality as the **distance to the full model**: the extra negative log-likelihood, in nat/token, of the full model's own (API) continuations, on three prompt sets. Smaller is better.
39
  - `og` exact: 0.090 / 0.051 / 0.069.
40
  - With the engine's default speed rules: 0.091 / 0.054 / 0.072.
41
  - HumanEval: 160 / 164.
 
45
 
46
  1. Build the fork from the patches in the [GitHub repository](https://github.com/MasherDon/Siege-Ladder). It is tested on Windows with CUDA 13 on an RTX 50 card (sm_120).
47
  2. Pass `-m .../DeepSeek-V4.1-Flash-MXFP4-engram-00001-of-00011.gguf`. The other parts load by themselves.
48
+ 3. Set up the lens (on by default). The engine looks for it in a folder named `g44` *next to* the model's folder, not inside it. After downloading this repository, move its `g44` folder one level up, or set `LLAMA_MOE_STREAM_G44` to the path of `1017-lens-d2.gguf`. Without the lens the engine still runs, but decodes slower.
49
 
50
  Our PC: RTX 5070 Ti 16 GB on PCIe 4.0 x16, Ryzen 5 7600, 64 GB DDR5, one PCIe 5.0 NVMe (plus an optional expert mirror on a second NVMe).
51
 
52
  | | tok/s |
53
  |---|---|
54
+ | decode, short context | 18.44–18.80 through the server (6 November); 21.30 teacher-forced (4 November) |
55
  | reading a 246K-token document (256K context) | 1128 (6 November, mean of two runs) |
56
  | reading a 479K-token document (512K context) | 1043 (6 November) |
57
  | reading a 947K-token document (1M context) | 888 (6 November) |