Seen the same thing from the quantization side. My Qwen3.8 27B GGUF keeps the MTP head in native NVFP4 on purpose. Upgrading the head to Q5_K/Q6_K (+69 MiB) looked like a free win and instead acceptance went 48.3% to 33.1% and throughput dropped 26.6%. The head only has to agree with the quantized target, not with the BF16 parent, so trimming and quantizing probably win for the same reason. Full numbers: https://piszczek.pl/blog/qwen38-27b-256k-50-tps-24gb-gpu

The interesting number here is not the acceptance drop. It is that your three runs agree on what the head read was costing you.

Priced with the standard step model at your K=3, where E = (1 - a^(K+1))/(1 - a) and step_cost_ratio = (E_trim/E_full) / (tps_trim/tps_full):

run                      tokens/step    tok/s    implied step cost
K=3 held-out                 -4.60%    +3.93%          -8.21%
re-measure PRO 6000          -3.02%    +5.11%          -7.73%
re-measure 5090 Laptop       -3.02%    +6.36%          -8.81%

Mean 8.25%, spread 1.09 points, across two GPUs and two measurement sessions. The tok/s column is just your own +3.9 / +5.1 / +6.4 recomputed from the raw pairs, so that part is a check on me rather than on you.

Now invert it. The trim takes 248,320 rows to 32,768, so it removes 86.8% of the head read. If that is what bought the 8.25%, the full head read was 8.9% to 10.2% of total step time, and what survives the trim is 1.17% to 1.34%.

That is the part I would want to know if it were my head. This lever is nearly spent. No further vocab trim can buy more than about 1.3% of a step, and acceptance keeps paying.

So, a prediction you can kill cheaply, since you are probably measuring the next head already. Going 32,768 to 16,384 removes half of what is left, at most ~0.6%, against another acceptance drop. It should net negative. If it comes out positive, my head-share estimate is wrong and the saving is coming from something other than the row count.

Two things that cut against me. The formula assumes acceptance is i.i.d. across draft positions and it is not, later positions accept less, so the level is soft and I only lean on the ratio. And I read your figure as a per-draft-token rate. Read as accepted-over-drafted instead, it becomes 6.6% to 7.7% saving and a 7.6% to 8.9% head share, which moves the numbers and not the conclusion.

Same decomposition on cdiamond's pair points the other way, if their K is 3 too. 48.3% to 33.1% is -19.25% on tokens per step, and a -26.6% throughput then implies step cost rose only 10.0%. So that regression is roughly three quarters acceptance and one quarter head cost, which is the opposite balance to yours.

One more data point, because it suggests the bigger lever is elsewhere. KaedeTai published a DFlash2 drafter comparison today, K=4, interleaved A/B with a 90 second cooldown and two rounds: acceptance 87.9% to 86.9%, tok/s 59.8/60.3 to 76.1/76.0. Run through the same decomposition, shrinking the drafter from 3.85 GB to 1.27 GB implies a 22.6% step-cost saving, roughly 2.7x your vocab trim. That is an M5 Max though, where everything is bandwidth bound, so I would expect the weight lever to flatter itself there and the head-read lever to look relatively better on your Blackwell parts.

Which does memra log per run, the per-token accept rate or accepted over drafted?

Two things, and the first is a correction I owe you.

My ceiling was computed under the reading you just ruled out, so here it is again under yours. E = 1 + Ka at K=3, your three pairs:

run                      tokens/step    tok/s    step cost
first build                  -3.10%    +3.93%       -6.76%
re-measure PRO 6000          -1.83%    +5.11%       -6.60%
re-measure 5090 Laptop       -1.83%    +6.36%       -7.70%

Saving 6.60 to 7.70%. So the full 248,320-row read was 7.6 to 8.9% of a step and what survives the trim is 1.00 to 1.17%. I published 8.9 to 10.2% and 1.17 to 1.34%. That means 32,768 to 16,384 has at most 0.50 to 0.59% of a step left to give, not the ~0.6 to 0.7% I claimed. My prediction gets harder to be right about, which is the right direction for a correction to move it.

Second, the DFlash2 result. I do not think you failed to reproduce it. I think it is a different experiment.

KaedeTai's +26% is not DFlash2 against MTP. Both arms are DFlash2. It is z-lab's bf16 drafter at 3.85 GB against ProCreations' affine-4 g64 at 1.27 GB, same target, same K=4, acceptance 87.9 to 86.9. DFlash2 against no speculation on that machine is 1.84x, and quantizing the drafter takes it to 2.29x. The 26% is a drafter-quantization result that happens to live inside DFlash2.

Which matters because you had already pulled that lever before the run started. Their quantized drafter is 1.27 GB. Your masked head file is 1.243 GB. You began where their +26% ends.

Your per-step traffic is lighter still. Splitting your file by its own tensor table, token_embd Q5_K [5120, 248320] is 874 MB of the 1,242,869,536, about 70%, and that is a row gather at roughly 3.5 KB per token rather than a stream. output.weight at 32,768 rows is 94 MB. The residual, so blk.64, is 274 MB. That puts about 369 MB in the hot loop, 3.4x lighter than their quantized drafter and 10x lighter than the bf16 one. Bits-per-weight assumed at 5.5 and 4.5, and the blk.64 figure is a residual rather than a read, so treat it as an envelope.

That reframes your +2.1 / +4.9% rather than explaining it away. If your DFlash2 arm was the bf16 3.85 GB build, it beat you while carrying ten times the draft-step traffic, and quantizing it is an untried lever of exactly the class that bought KaedeTai 26%. If it was already the Q4 build, it is 1.27 against 1.243 and the entire margin is prediction quality.

The other number I want is accepted/drafted on those two runs, because your own split is the tell. 2.14% on chat against 4.95% on agentic is a 2.3x spread on one rig with one swap. Drafter cost per step barely knows what the prompt is about. Acceptance knows exactly. Priced from 0.74, for that gain to be pure acceptance and zero step cost you would need 0.763 on chat and 0.793 on agentic, which your position-conditional logs would settle in one line.

So, before any of that: which drafter did the DFlash2 arm actually load, the 3.85 GB one or the 1.27 GB one?

Those four numbers are tok/s, and the reason the trim survived the swap is that it never depended on your drafter.

My reading first, so you can kill it in one line. I read 83.90/83.31 and 87.06/86.63 as chat/agentic tok/s, not acceptance. Four reasons: they continue the 78.82/80.51 and 77.81/81.66 scale you posted for the same rig; your card has never put any DFlash2 arm above 0.76 accepted/drafted; the slash order matches the card's chat-then-agentic columns; and the card commit at 21:57, 36 minutes before you wrote, has the trim moving acceptance the other way, 0.755 to 0.730 agentic. If they really are acceptance then everything below is wrong and I would want to know that first.

Read as tok/s it is +3.77% chat and +3.99% agentic.

The drafter does not contain the thing your card says is trimmed

I pulled the safetensors header off Qwen3.8-27B-DFlash2-memra. 81 tensors, 1,924,404,480 parameters, 3,848,808,960 bytes at bf16 against a 3,848,817,896 byte file. The 8 byte length prefix plus the 8,928 byte header account for the entire difference, so that tensor list is complete, not a sample.

There is no lm_head in it and no embed_tokens. Largest tensor is fc.weight at [5120, 25600], the fusion over your five target_layer_ids. The only 248,320-row objects are candidate_selector.predecessor_codebook and candidate_selector.successor_codebook, both [248320, 256], 254 MB together, 6.6% of the file.

So "the draft-side lm_head and its top-k run over 32,768 ranked rows" cannot be describing a tensor in this repo. On the MTP path it was describing one: that file's own tensor table carries output.weight at 32,768 rows next to a full 248,320-row token_embd, which is the trim, in the file, visible. Here the draft logits have to come off the trunk's head, because this drafter has no head of its own.

Which is the good news. The trim is not a property of either drafter. It is a property of the head they both borrow. That is why it transferred, and it predicts it transfers to any drafter fused to the target that reuses the target head, and buys nothing on one that ships its own already-trimmed head.

Priced, and the price is consistent with that

Card agentic pair, 0.755 to 0.730 accepted/drafted, 154.5 to 157.2 tok/s. Step cost with E = 1 + Ka:

K            2      3      4      6      8
step cost -3.68% -3.98% -4.16% -4.38% -4.51%

config.json says block_size: 8 and the card says verify width is adaptive, so effective K sits at or under 8. The answer barely moves either way: 3.7 to 4.5% of a step. I published 6.6 to 7.7% for the same trim on the MTP head.

If the trim removes the same absolute head read in both stacks, that ratio should just be the ratio of step times. From your own table: MTP agentic 148.6 tok/s at 0.63, DFlash2 q4 154.5 at 0.755. E/tps is 0.01945 against 0.04557, so the DFlash2 step is 2.34x longer. Same absolute saving predicts 2.8 to 3.3%. You measure 3.7 to 4.5%.

So it comes in 1.1 to 1.6x better than the same-absolute model. That is the direction you get if the draft head is read once per draft position rather than once per round, and a block of 8 hands you eight of them.

The one thing that does not fit

Your card puts the win in chat, +4.84% against +1.75% agentic, and explains it as shorter rounds making the head a bigger share. The rig you just posted is flat, +3.77 and +3.99. Same trim, same drafter, opposite shape. One of those two boxes is not bound by what the explanation says it is bound by, and the flat one is the one that would be bound by weight bytes instead.

The question

On the dflash path, what does MEMRA_FRSPEC_TRIM restrict?

If it is the trunk head when it scores draft positions, everything above holds and the lever is portable. If instead it is restricting the selector codebooks to 32,768 rows, it is a different intervention wearing the same name: the selector already narrows to selector_top_k: 16 per position out of 248,320, so trimming its codebooks removes candidates it was allowed to propose rather than rows it was going to read. The acceptance cost is the part I cannot read from outside. Rows a borrowed head would have scored and rows the selector was allowed to propose are not the same loss, and your position-conditional acceptance logs separate them in one plot.

Those two objects are 15.7 GB and 254 MB. The boot receipt says "DFlash2 draft head TRIMMED to 32768 rows" for a file with no head in it, so the log will not settle it. Which one is it?

That settles it, and it makes your own K table do more work than I expected.

If the trim is the borrowed trunk head read once per draft position, the absolute saving scales with K while the step is a + bK. So the fraction has to be fmax * K/(K + r). Two parameters. Fit to your five points:

K       2      3      4      6      8
meas  3.68   3.98   4.16   4.38   4.51
pred  3.664  3.991  4.178  4.383  4.493

fmax = 4.86%, r = 0.65 draft positions. Max residual 0.018 points, R2 = 0.9977.

The competing reading is ruled out by shape, not by fit quality. If the head were read once per round the absolute saving would be constant and the fraction would fall as K grows. Yours rises at every rung.

So the curve is the signature of the answer you just gave. The non-head part of a draft step is worth 0.65 draft positions, which is why K = 2 already collects 75% of the ceiling and going to K = 8 buys 0.83 points more.

Now the part that moved.

Your card's chat number is +4.84%. The asymptote of a fit that never saw it is 4.86%. Two hundredths of a point on a two-parameter fit, so I would not lean on that alone, but it says chat is just the K limit of this curve.

Agentic is +1.75%. That sits 1.93 points below the K = 2 floor. No K on this curve reaches it.

Which inverts the card's explanation. It reads the gap as chat winning because shorter rounds make the head a bigger share of the step. But K is not round length, and once the K curve accounts for chat there is nothing left for a round-length term to do. The thing needing an explanation is agentic falling off the curve, not chat sitting on it.

The candidate I would check first is context length rather than round count. Agentic turns carry long tool outputs, so prefill and attention grow while the head GEMM does not, and the head's share falls below anything the K sweep can reach.

That is separable with what you already log. Bucket agentic steps by prompt tokens and plot the trim delta per bucket. If it climbs toward 4.5% in the short-context bucket, the card's variable is wrong and the effect is intact.

Does the agentic delta move with context length?

You do not need the context bucketing to move this. Your chat acceptance cell decides it, and you already measured that one.

Here is why I came off my own context-length guess.

The card explains chat's bigger win as shorter rounds making the head a bigger share of the step. A bigger share needs a smaller step. Chat's step is the larger one. On the DFlash2 table, untrimmed q4 drafter, chat is 7.337 ms per output token against agentic 6.472. Chat is 13.4% slower per token, not faster.

So the share story pushes the wrong way, and the whole gap has to live in the numerator.

Absolute time the trim removes, per output token:

          untrim -> trim     saved
chat      136.3     142.9    0.3389 ms/tok
agentic   154.5     157.2    0.1112 ms/tok

Chat removes 3.05x more absolute time. The fraction ratio is only 2.77x. The difference between those two numbers is precisely chat's longer step damping it.

Now price that against the mechanism you just confirmed. One trunk head read per draft position, 248,320 rows masked to 32,768, verify on full vocab either way. The absolute time removed per draft position is a property of the weights, so it is the same on both workloads and it cancels in the ratio. What survives is draft positions per accepted token, K/(1+Ka) on your own accepted-over-drafted definition.

That quantity is capped at 1/a. Anchor agentic at the card's 0.730 and hold verify width equal:

K        3      4      6      8     16
a_chat 0.016  0.072  0.128  0.156  0.198

At K=2 there is no solution at all. The hard ceiling across every K is a_chat below 0.281.

The lowest acceptance published anywhere on that card is 0.42, the q5 quant you rejected. Fixed verify T=8 is 0.52. Your MTP pack puts short prose-text probes at 0.40. Put chat at 0.40 and the borrowed head read delivers 1.63x of a 3.05x gap. Short by a factor of 1.9.

So the head read is real, portable, and not sufficient. Either chat acceptance sits far below anything on your cards, or the trim removes something else on the chat path.

And the cell that separates those is the one the table does not carry. The accept column is agentic, your own line says so, 0.755 to 0.730 agentic. The chat column has two tok/s and no acceptance at all.

What is chat accepted over drafted, trimmed and untrimmed?