ornith-ai

/

Ornith-1.5-35B-A3B

like 526

Follow

Ornith 3.19k

For me it doesn't fail tool calls, however it fails to deliver. Stuck in endless thinking process, never get anything done. Thinking process itself is legit, not doomloops, but it just can't get out of reasoning. Well it gets out of it, reads another file and than goes into reasoning again. Never actually applying its findings. I already see correct solution in model thought but it just keeps reasoning about it or possible other solutions. Tried to solve task which usually takes 60k tokens, Ornith 1.5 took 320k tokens (2 sessions) and was not able to solve it. Idk maybe we need to limit reasoning effort somehow as Qwen 3.8 requires... Promising but disappointment release.

This model seems to be pretty sensitive to KV cache quantisation, I had it failing tool calls constantly with Q4, after removing cache quantisation all the problems dissapeared

This is just the initial release, so like usual, they will probably release a post-training/adjustments/etc., but until then, this model fails at pretty much every task under every harness with various parameter tunes.

It will also do its own things on well-established files (e.g. I had a 40-container service Docker compose file, and this model made it 2 containers/services and the rest just disappeared). Don't risk your codebase without proper backup/versioning.

Not here to defend Ornith, but: which harness, which settings, which tasks? "Fails at pretty much every task under every harness" with zero repro details is just trust-me-bro. The benchmarks put it ahead of its predecessor and it's a strong model in practice, if your experience differs, the interesting question is what's different about your setup.

And on the compose file: you don't manage a 40-service Docker stack with a local 35B-A3B. That's ~3B active params being handed your production topology. When 38 services vanish, that's not a frontier-model story, that's the tool being wildly undersized for the job, and nobody reading the diff before it landed.

Exactly what @Kirin1997 wrote: Remove all kv cache quantization and try again. Qwen3.x and especially 3.5 (if this is based on 3.5..) are very sensitive to kv cache quantization!

This is just the initial release, so like usual, they will probably release a post-training/adjustments/etc., but until then, this model fails at pretty much every task under every harness with various parameter tunes.

It will also do its own things on well-established files (e.g. I had a 40-container service Docker compose file, and this model made it 2 containers/services and the rest just disappeared). Don't risk your codebase without proper backup/versioning.

is bro using cache-type-k/v q4_0? wdym fails lmao

I'm using Pi coding agent and never seen for 3 hours of testing this model used a single edit tool xD it only read and thinks. Oh and I don't quantize cache ever.

@Azula39 Try it without kv cache and or differnet weights quant. That could change things a lot.

Edit: ...without kv cache quantization ...

@Azula39 Try it without kv cache and or differnet weights quant. That could change things a lot.

I use Q5 model and don't quantize cache. Just tested write and edit tools, model can execute them if I tell it to do so. However on real task of finding and fixing a bug model is not using edit or write tools, wasting 100k+ of tokens for nothing. Never seen such behavior from 1.0 release. Maybe some kind of bug in template in this release idk...

I'm using Pi coding agent and never seen for 3 hours of testing this model used a single edit tool xD it only read and thinks. Oh and I don't quantize cache ever.

yaa it tends to think a lot

The KV cache I used was whatever the model's default is.

Also, it's internal benchmarks. Revealing benchmarks would be included in the next iteration of model releases, which would defeat the purpose. Internal benchmarks are very common in various AI discussions. I might give some information about it later, but it performed so poorly, so it's rather low priority, not that I (or most) really do that in general anyway.

Best 35B coding model for now is KAT-Coder-V2.5-Dev, but nobody talks about it...

KAT is not bad, but this is better, you just have to use APEX quants, they seems to be better.

Use smart quantized models like UD, AD or APEX to maximize quality over size. Do not quantize k cache too much. This model is actually good and in two harnesses I have, it never fails tool calls (at least never seen it yet).

Best 35B coding model for now is KAT-Coder-V2.5-Dev, but nobody talks about it...

I find the original by Qwen to be better than KAT Coder.

Fast like a hell, really good q4 quant performance. I have two 5060Ti 16GB and Qwen 3.8 which of course is great is to slow on my setup. KV q8_q8 - I dont know why but turboquant didn't work well and this is the most stable configuration. I use it as a Posit Assistant coding agent in RStudio.

Fast like a hell, really good q4 quant performance. I have two 5060Ti 16GB and Qwen 3.8 which of course is great is to slow on my setup. KV q8_q8 - I dont know why but turboquant didn't work well and this is the most stable configuration. I use it as a Posit Assistant coding agent in RStudio.

I cofigured Qwen 3.8 for the same GPU for my job buddy yesterday, 40-50 t\s decode, idk about prefill though but looks like above 500 t\s+. Use the best Unsloth 3 bit quant + 4 bit kv cache and you should be able to get around 70-80k ctx. Yes, quality may be lower than in a typical Qwen setup, but that's not what matters, what matters is that it's still levels above 35b and KAT in coding

Can confirm it misuses tools, only tested in bionic and cline, not exactly failed calls but mistakenly deleting a 800 line script because it used a replace_file tool instead of an edit_file tool a couple times. Also misremembers stuff a lot, couldn't remember what the cwd is even though its literally injected with the system prompt, then when asked what the user name of my profile is, it misspelled it, very weird. Q4, no quantization on the k/v cache.

Will have to stay on Kat-coder for speedy coding.

Can confirm it misuses tools, only tested in bionic and cline, not exactly failed calls but mistakenly deleting a 800 line script because it used a replace_file tool instead of an edit_file tool a couple times. Also misremembers stuff a lot, couldn't remember what the cwd is even though its literally injected with the system prompt, then when asked what the user name of my profile is, it misspelled it, very weird. Q4, no quantization on the k/v cache.

Will have to stay on Kat-coder for speedy coding.

that replace over edit means the tool definition/description isn't clear enough. could be the harness' fault

Insane reasoning token utilization.

I just did some testing, and in the first 3 prompts went full Qwen3.8-27b level times 3!, 20k tokens just on reasoning!!
i know i can limit it and even take it off but man 20k tokens for a Volcano html model test is more than Qwen3.8.

But the quality of the output is actually quite good, it failed to deliver but the whole architecture was really good, ran galaga prompt game on it and was less thinking, i think the harness of the agent.md / prompt matters heavily on this model, i went very specific and speed up and quality much more than Qwen.

Just came here from another discussion thread about MTP or speculative decoding. Essentially, I disabled it, and it seems better so far now.

If something is wrong, I will report back.

I've experience tool calls failed while running it on Openclaw, on Hermes however is so far flawless and much faster than Qwen3.8 on my AMD Strix