Thank You!
#1
by svarognsa - opened
Thank You for the effort.
This is the right direction for a next significant breakthrough in local LLMs inference. The instructions on the model page worked out of the box (I used 48GB MacBook). Maybe just to expand the instructions with a section how to use compiled llama.cpp with LlamaBarn.
Doubled speed with simplicity of usage just need clean and stable uncensoring and Your Qwen3.6 27B will be perfect solution.
Thanks. I am in the middle of re-converting and reuploading all quants due to the following changes:
- I have added a 7th fix to the jinja chat template, to autoclose unclosed think blocks when a tool call starts
- this time converting from bf16 > f16 > quant, instead of bf16 > q8 > quant, for slightly more precision
It should be finished in a few hours. The changes are minor, and you can always download the chat template manually from my other repo, but I would suggest you use the new quants once the upload is finished.
I will also update the readme with more details about how to set it as an anthropic endpoint, how to fir a larger context with turboquants, and which quant is best for various environments.
froggeric changed discussion status to closed