Spaces:

---

IsValorum

/

MiniPlus-NanoPlus-Requests

Like 3

Running

CyberTiel

1. Link: peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF-MTP (https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF-MTP)
2. Just like Tiel , agentic coding tasks, but stronger, less thinking(in my opinion) and uncensored
3. Miniplus like what you did on Tiel

The MiniPlus V2.1 and NanoPlus quantizations are already underway; they will be available within an hour at most—follow me to get notified.

Wow
Thanks for your hard works and very fast reply, really impressive, this is the fastest model request i've seen on hf (or maybe you had already worked on this model from before :))

I will definitely check and test for if the Apex Miniplus CyberTiel work good too

Request for next model.
Reason: I've personally used k2 and it's the closest thing to match or exceed Qwen intelligence:
IFM/K2-Horizon-MoVA-36B-A4B

Need of Speed) https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B

I personally tried the K2-Horizon, the smaller versions and that MoVA, and they didn't convince me because of the cost of the KV Cache.

I could barely get it to 8K context in Q4_0, not enough to properly test it, is it really any good?

For me, the fact that it had such an absurdly expensive KV cache was enough to completely rule it out. Did you guys test it in real-world use?

You were using Q4_0 KV?
How much VRAM on the rig?
I assume your models were tested on 24gb ram?

I tried it on an RTX 3090 and I simply didn't find that series of models appealing; the KV cache was too expensive.

Gotcha, no worries. Thanks for the responses, love your work.

Gotcha, no worries. Thanks for the responses, love your work.

If you find another model that you consider good, don't hesitate to let me know so I can quote it immediately.

are you interested in some thing called "Ternary-Bonsai-2-27B-gguf" (*edit:oh i tried result really bad, rather choose ornith 1.5)

Yeah, despite Prism-ML claiming over 90% retention, in practice they perform terribly. Ornith 1.5 is just way better.

among all 35b model, which provide you the best performance in agentic coding tasks ?
atm i'm using Tiel/CyberTiel

Tiel and CyberTiel are definitely great for that, but Occamy fits right in there as well.

While Tiel is very focused on direct syntax and fast tool execution, Occamy handles agentic coding equally well while offering a bit more balance on broader architectural context, complex debugging, and multi-file reasoning.

If you want to give it a spin to compare against your Tiel setup, here is the MiniPlus V2.1 release:
https://huggingface.co/IsValorum/Occamy-1.0-APEX-I-MiniPlus-V2.1-GGUF

Between Tiel/CyberTiel and Occamy, you're pretty much looking at the best options in the 35B tier for agentic workflows right now.

Yeah i see
But maybe i still go with tiel until qwen 4 35b a3b(just guess lol)
The author optimize this model kv cache really good so it take less vram/ram for large context amount
And it have mtp too
After apex miniplus exist so i change to mini plus when need quality, and comeback to iq3 xxs when need 262k context

Totally fair point. That DeltaNet KV cache efficiency plus MTP makes it a great daily driver for long context. Enjoy Tiel!

If you're dropping to flat IQ3_XXS just to fit the full 262K context, check out the NanoPlus release of Tiel:
https://huggingface.co/IsValorum/Tiel-Coder-35B-A3B-APEX-I-NanoPlus-GGUF

That's actually why I built NanoPlus: to give you that lower memory footprint for massive context while still maintaining Q4_K_M/L-class practical quality instead of the flat degradation of standard IQ3_XXS, keeping routers in F32 and output in Q6_K.

Yeah, but i think miniplus with 131k still work good, even can upto 180-190k if i reduce some prefill boost parameter and disable mtp

Thoughts on the new European Model? Aleph-Alpha/Kolibri-1 https://huggingface.co/Aleph-Alpha/Kolibri-1

78B with ~4B active, gets decent benchmark

Hi @userr99 ,

I looked into it as soon as it dropped today, but unfortunately it is physically impossible to make GGUFs for it right now.

The main blocker is that Aleph Alpha built a completely custom architecture from scratch (Kolibri1ForCausalLM). llama.cpp has zero support for it: the GGUF converter will simply crash because it cannot map the tensors, and the C++ engine has no kernels to execute its specific 4:1 sliding-window attention (4 SWA layers for every 1 GQA layer) or route across 384 micro-experts per layer (that is 19,200 expert FFN blocks across its 50 layers). It does not even run on standard vLLM without installing Aleph Alpha's own proprietary plugin package (aleph-alpha-inference).

On top of the architecture gap, the base weights are 145.5 GB in BF16. Even if someone in the community spends weeks writing a custom C++ PR to get kolibri1 into llama.cpp, a 78B MoE would still land around 32 to 35 GB in a MiniPlus quant, meaning it would not fit into a single 24 GB VRAM card anyway without spilling into RAM.

So for now, until upstream llama.cpp actually writes and merges full native support for the architecture, it is completely out of reach for GGUF.

i see something called "Xing4.0-29B-A4B" ,does it have any advantage when comparing to 35b models (coding, speed, weight,...)

Tiel seem still very op

And Cyber-Tiel is practically the same but Abliterated; I also have it under the MiniPlus and NanoPlus recipes if you want to try them.

Hello, thanks for your work! Have you tested your mini models at 512k–1M context, especially in llama.cpp? Does the quantization remain stable or accurate that deep into the context, or is 160k–256k currently the validated range?

Hello! Honestly, I have only tested and validated them up to 256K tokens, which is the range where I can verify stability and accuracy. Beyond that (512K to 1M), I haven't tested them directly.

Thanks for your work IsValorum, fan of your VAL-APEX Mini and Nano. I would really like to see https://huggingface.co/agentionai/Qwen3.8-27B-AP-GGUF with VAL-APEX Abliterated. Reason being is that it requires fewer thinking tokens for similar output, also outperforms ISTA and Unsloth. Agention seems to be among the models for its size I've come across. It also has adaptive speculative decoding. Would really like it to be abliterated and with VAL-Apex applied to see it be reduced in size further with it being uncensored.

Here is the full report from his work with substantiation of why I'd like specifically this model VAL-APEX abliterated: https://share.fieldworkledger.com/tTkQDHijE_Anj0yv79bsIw

Hi @smolboon ,

Thanks a lot for the support on the Mini and Nano builds!

I checked out the Fieldwork Ledger report you linked. It is actually a benchmark comparing different GGUF quantizers (Agention vs Unsloth, ISTA, etc.) on stock Qwen3.8-27B, rather than a new fine-tuned model.

Because Agention only released final .gguf files and there are no unquantized BF16 weights, it is physically impossible to abliterate their files (abliteration has to be calculated directly on raw 16-bit weights before quantizing).

That said, we already have models in the catalog that cover exactly what you are looking for:

• If you want an uncensored model that uses significantly fewer thinking tokens, check out Qwen3.8-27B-EfficientThink-Uncensored (available in both MiniPlus and NanoPlus). It was tuned specifically to cut down thinking bloat while keeping full reasoning.
• If you just want the stock 27B base fully abliterated, Huihui-Qwen3.8-27B-Abliterated is already available.
• Plus, our MiniPlus and NanoPlus V3 builds already include the native MTP speculative decoding head directly in the GGUF.

Give EfficientThink a spin, it is probably the exact fit for what you have in mind!

hi
i try your Qwen3.8-27B-EfficientThink-Uncensored NanoPlus V3 but got this error

Do you know how to fix

Good catch @nmh2k3 , and thanks for the screenshot!

That happened because the main GGUF has the standard 64 layers (0-63) while the MTP head was packaged separately in the draft file, but the conversion script left block_count = 65 in the header. llama.cpp saw that 65 and started looking for blk.64.* inside the main file, which obviously wasn't there.

I just pushed an update to both NanoPlus V3 and MiniPlus V3 fixing the header to block_count = 64.

If you already downloaded the ~11 GB file and don't feel like downloading it all over again, you can patch your local file in 2 seconds with this quick Python snippet:

import struct

filename = "Qwen3.8-27B-EfficientThink-Uncensored-VAL-APEX-I-NanoPlus-V3.gguf"

with open(filename, "r+b") as f:
    header = f.read(2048)
    idx = header.find(b"qwen35.block_count")
    if idx != -1:
        f.seek(idx + len(b"qwen35.block_count") + 4)
        f.write(struct.pack("<I", 64))
        print("Done! Fixed block_count to 64.")

After that it will load right away. And if you want to run MTP speculative decoding with it, just add --model-draft mtp-Qwen3.8-27B-Q4_0.gguf to your command!

I also got an error when using the model in MLX-Serve which is my go-to inference engine atm on Mac, likely TensorFold soon tho when their app drops.

Hi @smolboon !

Did you download the file before the hotfix I pushed earlier today?

If so, you likely ran into the exact same issue as @nmh2k3 above. The GGUF header had block_count = 65 instead of 64, which causes GGUF loaders (including mlx-serve's GGUF engine) to panic looking for layer 64 (blk.64.*). If that's what happened, running that quick 3-line Python snippet above on your local file will fix it immediately without having to redownload.

Also, just in case, make sure you're passing the --mlx-gguf flag when launching mlx-serve with single-file GGUFs on Apple Silicon.

If it's a different error entirely, could you paste the terminal output / log here? Happy to check it out!

After, I use MLX-Serve, it was just a repetitive output. I already deleted the model atm. I'll do it tomorrow when I get back from work. Thanks for your work! P.S. curious to your VAL-APEX version of DavidAU's Turbo Brilliance smallest model and it's performance benchmarking

Hi,

Thankyou for your great work for the custom-tensor quantization. It worked great so far for CyberTiel MiniPlus I'm using for long coding, it utilizes my GPU more unlike other CyberTiel quants and i can upped my context more.

Would you consider to adding Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated ? I think it has better if not fully uncensored version than the base (which is not uncensored and only trained on their uncensored base) or previous version that still introduced some rejections.

What do you think? Again, thanks for all your great work and time, much appreciated.

Also, I noticed that lfm2.5 24b a2b got QAD version of it now. But, I don't think it can be used or adopt or fit with your custom-tensor quants, isn't it? since its mostly only q4 tensor in the GGUF's metadata.

Hi @xtronema ,

Regarding huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated: yes, I will be quantizing it into both VAL-APEX-I-NanoPlus and VAL-APEX-I-MiniPlus-V3 builds with modular MTP support.

As for LFM2.5-24B-A2B QAD, that checkpoint is trained specifically to mitigate quantization loss directly on a classic, uniform Q4_0 format rather than standard modern quants like Q4_K_M or Q4_K_S. Because it was released strictly as a pre-compiled Q4_0 GGUF without raw BF16 weights, it cannot be adapted into custom mixed-precision recipes without destructive double quantization; furthermore, our existing APEX-I MiniPlus and NanoPlus V2.1 builds already achieve the overall perceived output quality of a full Q6_K while preserving key attention and state projections at higher precision.

Hello,

Ah I see, also I didn't realize it was mostly basic q4_0 haha. Regarding your builds, yeah I agree, it feels smart enough for me for my coding, at least it's CyberTiel in my use case. And that's nice to know if it can achieve comparable performance to q6_K which already close enough to q8_0 but without the cost, that's good enough for me.

Anyway, Thanks, IsValorum for the quick response and your great work, cheers.

Are you interested in qwen 3.8 flash next nanoplus?
I've tried qwen 3.8 fn iq2 xs gsq gco via strata (because i don't have enough 64gb ram for q3)
But the results seem... pretty bad compare to qwen 3.8 27b nanoplus or iq3s gsq.
I don't know it was because of iq2 weakness or it's actually whole 3.8 flashnext's problem.

Hi @nmh2k3 ,

Two things to separate: what the architecture does, and what the quantizer chose. Both models are mixtures of experts, so the comparison is like for like.

On Flash-Next, 96 percent of the weights sit in routed experts and 1.95 percent of them are read on any given token. There is almost unlimited room to cut there at no decode cost, and a budget search always finds it. On the 27B that same figure reads zero: no experts, every weight read on every token, nothing to hide the cuts behind, so the search has to take the bits out of the dense path itself. That is what the MoE rule forbids, and it is the whole reason a hand recipe earns its place on the 27B and not here.

Now, what happens if I port my MoE recipe to the Flash-Next tensor map:

Every structural decision lands in the same place. Norms, router, hyper-connections, indexer, PLE, and both ssm_alpha and ssm_beta at BF16 on both sides. And both down projections land on IQ4_NL: mine because that tensor is 640 wide and rejects every rung of my ladder, theirs because the search chose it.

The one real difference is the expert gate and up. Mine is flat IQ3_XXS on all 48 blocks; theirs drops 20 of those 48 to IQ2_S. On the 27B the same difference is cheap, because there the ladder is 64 percent of the parameters. Here it costs 5.35 percent, because the ladder is 96 percent of the file and the price of a flat floor is paid almost in full. At the IQ3_S footprint there is no room to add it, and that is why I do not ship a NanoPlus for this model.

About the low tiers. To compete with IQ2_XS I would have to build a MicroPlus, and it is not worth your bandwidth. Put the dense path at IQ4_XS, the cheapest my rule permits there, and at their 39.21 GB there are 36.6 GB left for 120.8B of expert parameters. That is 2.42 bits per weight, and 2.42 on that ladder is IQ2_S mixed with IQ2_XXS and IQ1_M. That is their IQ2_XS build, tensor for tensor. A MicroPlus would be a rename.

Run IQ3_S. That is where the shape matches.

IsValorum

Hello. Just found your models and I must say they are very impressive and I wanted to ask something, for context I am trying to build a harness that is essentially capable of frontier intelligence and work but using local small models (Astra/fable level performance in 8B dense to 40B MOE). I truly believe and have always believed that the AI industry's current approach of creating an omniscient model that knows everything and can zeroshot everything is completely wrong. If we had focused on creating models that is not all-knowing but rather has common sense and capable of gathering required context during inference, with deeper level of understanding for a specific harness and environment. I understand such model will not work with all harnesses but this will allow us saturate the weights for general intelligence and in combination with the tools, harnesses and especially the internet could in theory be able to perform as well as or even better than frontier models. At the cost of more tokens generated which is a non-issue as it is fully intended to be used for local generation. Now I wanted to ask you,, do you know of such models right now or any specific finetunes that is utilizing this approach?