I’ve been running ThinkingCap-Qwen3.8-Flash-Next Q4_K_M locally through XeStrata on Linux, on a Threadripper 3970X with 128 GB RAM and a Radeon RX 6900 XT with 16 GB VRAM. ThinkingCap’s shorter reasoning is particularly appealing for this kind of local setup.
Would you consider collaborating with ISTA-DASLab to release GSQ-RCO GGUFs of ThinkingCap, or exploring another method with a comparable quality-to-size trade-off?
75.8 GB (IQ3_XXS): 99.4% of the BF16 base’s average benchmark score across AIME25, GPQA-Diamond and LiveCodeBench v6.
83.6 GB (IQ3_S): approximately BF16-level scores on that suite.
For comparison, your current ThinkingCap GGUFs weigh 95.5 GB for IQ4_XS and 113.2 GB for the recommended Q4_K_M.
Those results wouldn’t automatically carry over to the fine-tuned weights, but combining a smaller footprint with ThinkingCap’s reported 38.5% average reduction in reasoning tokens would be compelling for local users like me.
A comparison showing total download size, resident memory requirements, benchmark-score retention and reasoning-token savings for each quantization under matched evaluation settings would be especially useful. It would help establish whether the smaller versions preserve both answer accuracy and ThinkingCap’s shorter reasoning.
Such a collab is not currently planned, but here is an IQ3_S RCO build for now. We also tried making the XXS variant, but that one already prolonged thinking traces too much compared to the full-precision checkpoint.
I've also added the usual Expected performance section to this model's GGUF cards. It took a bit longer in this case due to the size of the model and the amount of checkpoints to evaluate. Hope that provides the info you need.
