nullsumlabs/qwen3.8-9b-coder

Qwen 3.8-9B-Coder

Custom small coding-focused LLM, packaged as GGUF for LM Studio and llama.cpp.

Qwen3.8-9B-Coder is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-9B architecture, further fine-tuned for coding, with vision restored via a Qwen3.5 projector.

LisyNeko -coder version Improvements

The original (empero-ai/Qwen3.8-9B) did not support image input. Building on it, the following enhancements were added:

• Vision Integration: Added image input capabilities
• Specialization: Coding-focused fine-tuning

Result: A custom model combining the Qwen3.5-9B architecture, Qwen3.8 knowledge distillation, multimodal vision integration, and specialized coding performance.

Empero 9b Improvements

Qwen3.8-9B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-9B architecture. The student was trained on ~70,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, code, general reasoning, instruction following, and tool use, quality-filtered before training.

The objective: bring the reasoning behavior of a frontier-scale teacher into a dense 9B that deploys on a single GPU.

Highlights

• Distilled chain-of-thought — every answer opens with a <think> block learned directly from Qwen3.8 2.4T A95B traces rather than synthetic self-generated reasoning.
• Mathematics and code emphasis — the trace mix is deliberately weighted toward hard math and competitive programming, the domains where distillation moves the needle most at this scale.
• Native function calling per Qwen3.5's specification — no wrapper or tool-specific fine-tune required.
• 262,144-token native context, inherited from the Qwen3.5 base.
• Full fine-tune — every parameter updated; not an adapter.

This is not a Transformers/safetensors repo. Tokenizer, config, and weights live inside the GGUF files. >Load it in LM Studio, llama.cpp, or Ollama — not transformers.pipeline. # Recommended settings for coding and terminal usage:

{
  "repeat_penalty": 1.05,
  "temperature": 0.7,
  "top_k": 20,
  "top_p": 0.8
}

Delete or rename the mmproj-qwen3.8-9b-coder file if you do not need Vision capability, save VRAM.

Running in LM Studio

Search for the model name in the application, qwen3.8-9b-coder

Manual Installation

1. Download this repo and drop the two .gguf files into LM Studio's models folder.
2. Select Qwen3.8-9B-Coder-Q4_K_M.
3. Keep mmproj-Qwen3.8-9B-Coder-BF16.gguf next to it for image input. Delete or move the mmproj file if you do not need vision (saves VRAM).

Running with llama.cpp

llama-server \
  -m Qwen3.8-9B-Coder-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-9B-Coder-BF16.gguf \
  -c 16384 \
  --jinja

Specs

• LLM size: ~5.8 GB (Q4_K_M)
• Vision projector: ~879 MB (BF16)
• Context window: 262,144 tokens
• Input: text, image (vision)

Architecture: qwen35
Type: Hybrid Causal Language Model (Linear SSM / Gated DeltaNet + Gated Attention)
File / Quantization Type: Q4_K_M (Total Size: ~5.8 GB)

Language Model Parameters & Dimensions

• Hidden Dimension (Embedding Length): 4,096
• Token Embedding: 248,320 (Vocab Size)
• LM Output: 248,320 (Unshared / Separate Head)
• Total Layer / Block Count: 33
• Hidden Layout: 8 × (3 × (SSM / Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)) + 1 × (Gated Attention → FFN with MTP)
  (Attention interval: every 4 layers on blocks 3, 7, 11, 15, 19, 23, 27, 31, and 32)

SSM / Linear Attention (Gated DeltaNet)

• Inner Size: 4,096
• State Size / Dimension: 128
• Time Step Rank: 32
• SSM Group Count: 16
• 1D Conv Kernel Size: 4

Gated Attention (Full Attention)

• Attention Heads: 16 for Q and 4 for KV (Grouped Query Attention)
• Head Dimension (Key / Value Length): 256
• Rotary Position Embedding (RoPE) Dimension: 64
• RoPE Dimension Sections: [11, 11, 10, 0]
• RoPE Base Frequency: 10,000,000 (1e+07)
• RMS Norm Epsilon: 1e-06

Feed Forward Network (FFN)

• Intermediate (Feed-Forward) Dimension: 12,288

Multi-Token Prediction (MTP) / Speculative Decoding

• Next-N Predict Layers: 1 (Integrated into Block 32)
• Projection Projection Dimension: [8192, 4096]

Context & Tokenizer

• Context Length: 262,144 tokens natively
• Base Tokenizer Model: GPT-2 BPE (pre-tokenizer: qwen35)
• BOS Token Added: False
• EOS Token ID: 248046
• Padding Token ID: 248044

Developed by Empero with vision and coding fine tuning by LisyNeko. 

Cloned from deployment on Ollama at: LisyNeko/qwen3.8-9b-coder

Repackaged on Huggingface by nullsumlabs for LM Studio compatibility.