request for qwen3.8 next
can we get a gsq rco quantization fot the next model
Yes, definitely. Once we’re done with Qwen3.8-27B, we’re going to work on GSQ-RCO quantization for the flash-next model as well. Thanks for the suggestion!
Yes, definitely. Once we’re done with Qwen3.8-27B, we’re going to work on GSQ-RCO quantization for the flash-next model as well. Thanks for the suggestion!
I can test it on 32GB VRAM 5090 and 128 GB RAM if that may be helpful.
That’d be awesome, thanks! I’d definitely be interested in seeing how it runs on your setup once our quants are ready.
An interesting idea, this will cause the ceo of chatgpt to collapse in shock as if he were seeing a nuclear bomb explode
Oh, by the way, I've noticed that some friends have cut off the non-code expert feature of qwen3.8-next and still managed to run coding tasks normally. Could we consider a fully functional iq3_s and a quantized version of iq3_s with only coding function cuts?
For our first release we’re aiming for a general-purpose ~38 GB model (excluding the N-gram table). That said this is a very interesting suggestion. Expert pruning is one of the areas where RCO can perform particularly well, and it’s something we haven’t explored yet.
38gb thats whithin the 3 ish bpw while still lossless on tasks thats crazy. i cant wait to try it on a new solution im working on for low vram machines !!! (experts multi external ssd streaming), im trying to see how much speed i can still get. but i think ill have to customize the backend for the model when it comes out.
For our first release we’re aiming for a general-purpose ~38 GB model (excluding the N-gram table). That said this is a very interesting suggestion. Expert pruning is one of the areas where RCO can perform particularly well, and it’s something we haven’t explored yet.
qwen3.8-27b and qwen3.8-next might both be specialized versions for the coding field. In fact, I only use them to write code, and most people probably do the same. So, eliminating other expert layers is not a problem. As for those who want to use them to write articles or make PPTS, Any other 9B small model can handle it.
By the way, in fact, relying on ai for everything is an extremely dangerous thing. It will gradually lose one's own thinking ability. People should still occasionally write articles and take notes by themselves to maintain the joy of creating something with their own hands. Otherwise, they will only become physical puppets of agents. Handing over the code to qwen3.8 was a choice made out of necessity to reduce the intensity of labor.
Thank you for your work. Good luck to you~
Hey bros, this is the expert-pruned version of Qwen3.8-Flash-Next:
https://huggingface.co/ranxianglei/Qwen3.8-Flash-Next-W4A16-Modular
https://github.com/ranxianglei/sglang/blob/ours/main/README.zh-CN.md
https://github.com/ranxianglei/sglang-expert-profile/blob/main/README.zh-CN.md
We hope to leverage the collective power of the entire community to create expert-layer profile descriptions for all expert layers in both Qwen3.8-27B and Qwen3.8-flash-next. This way, everyone can later freely combine the experts they need from these two models to create domain-specific models.
Then, combined with IQ3_S quantization, we can obtain minimal-size, domain-specialized quantized models that retain Qwen3.8's intelligence.
The author is also the creator of the billion-context project https://github.com/ranxianglei/billion-context/tree/master. Combined with this plugin, agents can maintain just 100K context per session while still handling long-conversation tasks.
The final goal is that both Qwen3.8-27B and Qwen3.8-Flash-Next can have 3-bit or 2-bit quantized, domain-specialized mini-models for each field, to assemble a team of expert agents for different tasks.
@anm2211 requesting this method for Gemma series as well (for prose especially for the E4B series) and Ling 3.0 series (less relative compute) and the other Qwen3.5/3.6 sizes for small models to experience the speed-up
I can test your models on my setup aswell. I have a RTX 3060 (12gb), RTX 5090 (32GB) and 192GB RAM.
i can test it too i have own pr too for my server i am currently using q8 and its smarts