APEX quant version please
#7
by Tariel - opened
APEX quant is the SOTA quantization method for MoE models. It will be beneficial to have APEX-I-Quality, I-Balanced and I-Compact quants with Tiel-Coder-35B-A3B.imatrix.gguf.
APEX quants with Tiel-Coder-35B-A3B.imatrix.gguf and the Sharp chat template: https://www.modelscope.cn/models/Tariel/Tiel-Coder-35B-A3B-APEX-MTP-GGUF
Tariel changed discussion status to closed
Hi! :) Thank you for the interest, and for making a quant that others have requested! <3 According to APEX’s own published metrics, it looks slightly worse than Unsloth Dynamic, which is the quantization strategy I use and publish. So I still do not consider it State of The Art or recommend it to my users