May we have a decently compromised version for 16GB VRAM?
Thanks for your wonderful work! Kindly ask that, may we have a decently compromised version for 16GB VRAM. There seems to be room between the very low (14.8 GB) and the low (15.6 GB)
Hi! I've just added a LOW-MINUS variant, standing in the middle between LOW and VERY-LOW. I tried to bump the tensors that matter the most (leaving MTP head the lowest to favor quality over squeezed performance for that version). Let me know if that works well for you.
LOW-MINUS is unfortunate naming because previously valid commands such as:
llama-server -hf esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF:LOW
Now grab LOW-MINUS variant.
I hear you. It turns out it's trickier than I thought to come up with a term between LOW and VERY-LOW and that it's couldn't be misunderstood as between LOW and MEDIUM instead 😄
I'll rename this to COMPACT-LOW . Change should be live in a bit.
Thanks! You're doing awesome work.
Thanks!