This model versus Ostfralla/Qwen3.8-27B-NVFP4-NInfer
This model is 21.5GB vs Ostfralla's 18.3GB. Do you have any idea about the quality and speed differences between these two models?
i asked myself the same question, had the agent do some research for me. this one should have better quality and same speed but more vram footprint. apparently the way the weights are kept in higher precision where they matter most here.
Evaluation results are published
Ostfralla targeted similar quality to the group-wise int, plus a speedup from Blackwell's native NVFP4 capability. It looks like they achieved their goal as they show 800+ tps and a virtual tie at quality with the group-wise int. Ostfralla's artifact is about 2GB larger than the groupwise.
A caveat from Ostrfralla: Under greedy decoding this artifact truncated 5 of 164 HumanEval+ problems at a 50k token cap, against 3 for the groupwise-int artifact — it falls into reasoning loops slightly more often. With sampling enabled this does not occur.
The neroued artifact we're discussing (3.8 27b nvfp4 ninfer) follows Unsloth OptModel, essentially nvfp4 for MLP layers, but has the attention layers at FP8 resulting in noticeably higher quality than either groupwise or straight NVFP4, with speeds hitting 200 tps at smaller context sizes.