dawncr0w/crowqwen3.5-4b-agent-heretic-mlx-oq

CrowQwen3.5 4B Agent Heretic MLX oQ

Apple Silicon oQ builds for a compact Qwen3.5 coding and agent assistant.

Collection · GGUF

CrowQwen3.5 starts from Qwen/Qwen3.5-4B and is tuned toward local assistant work: coding notes, refactor plans, bug triage, shell-oriented workflows, and bilingual technical prompts.

Highlights

• Korean + code + English + agent SFT mix.
• Tool-call formatting support in the chat template.
• Heretic-edited final export selected by internal refusal/KL probe.
• 9 MLX/oMLX oQ variants for Apple Silicon.

Pick A Directory

Goal	Recommended
Balanced default	oQ4
More fidelity	oQ5, oQ6, oQ8
Smaller local runs	oQ2.7, oQ3, oQ3.5
Extra compact choices	oQ2, oQ2.5

Start with oQ4 if you are unsure.

Quick Start

huggingface-cli download \
  dawncr0w/crowqwen3.5-4b-agent-heretic-mlx-oq \
  --include "crowqwen3.5-4b-agent-heretic-oQ4/*" \
  --local-dir crowqwen-mlx-oq

from mlx_lm import load, generate

model, tokenizer = load("crowqwen-mlx-oq/crowqwen3.5-4b-agent-heretic-oQ4")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Draft a short CLI refactor plan."}],
    tokenize=False,
    add_generation_prompt=True,
)

print(generate(model, tokenizer, prompt=prompt, max_tokens=160))

Artifacts

Variant	Size	Smoke
oQ2	3.10 GiB	passed
oQ2.5	3.21 GiB	passed
oQ2.7	3.37 GiB	passed
oQ3	3.44 GiB	passed
oQ3.5	3.54 GiB	passed
oQ4	3.87 GiB	passed
oQ5	4.33 GiB	passed
oQ6	4.76 GiB	passed
oQ8	5.66 GiB	passed

BUILD_MANIFEST.json has the build and smoke details.

Note

Smoke checks verify local loading and short generation only. No benchmark scores are claimed.