Luck-Qwen3-4b-Code-FineTune-GGUF

This repository contains official GGUF quantizations for ahmetggg/Luck-Qwen3-4b-Code-FineTune (a 4.05B parameter QLoRA fine-tune optimized for agentic coding, tool use, multi-step reasoning, and calibrated uncertainty). Complete training logs, dataset details, and base weights are in the main repository.

======================================================================== SCORES

category base fine-tune delta

tool 93.8% 0.0% -93.8pp multistep 90.0% 0.0% -90.0pp calib_answer 100.0% 87.5% -12.5pp calib_idk 100.0% 10.0% -90.0pp reason 100.0% 100.0% +0.0pp code 87.5% 0.0% -87.5pp agent 100.0% 50.0% -50.0pp

---

FOCUS avg 95.9% 15.0% -80.9pp CALIBRATION 100.0% 48.8% -51.2pp MACRO avg 95.9% 35.4% -60.5pp

KARAR: KALDI

• FOCUS ortalama geriledi; fine-tune bu sette fayda göstermedi.
• IDK setinde iyileşme yok; belirsizlik kalibrasyonu bu sette görünmedi.
• Tool-call skoru düşmüş; şablon kayması olabilir.
• Kod pass oranı düşmüş — tavan/genel coding gerilemesi var, iddia ile çelişebilir.
• Bu SWE-bench / BFCL değildir. Hold-out mini set: tool format, ilk-aksiyon, R-Tuning tarzı IDK, kısa muhakeme, unit-test kod, repo-ajan muhakemesi.

elapsed: 12.1 min raw: /content/luck_eval_results/raw_results.json

## Prompt Template This model uses standard **ChatML** format (no thinking/reasoning tags): ```text <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user Write a Python function to check for prime numbers.<|im_end|> <|im_start|>assistant ```

Provided Quantizations

All files are located in the repository root, generated via llama.cpp's convert_hf_to_gguf.py and llama-quantize.

Quant	File Name	Size	Notes
F16	Luck-Qwen3-4B-F16.gguf	~8.1 GB	Unquantized full precision
Q8_0	Luck-Qwen3-4B-Q8_0.gguf	~4.3 GB	Extremely high fidelity
Q6_K	Luck-Qwen3-4B-Q6_K.gguf	~3.4 GB	Near-lossless quality
Q5_K_M	Luck-Qwen3-4B-Q5_K_M.gguf	~2.9 GB	High quality, low loss
Q5_K_S	Luck-Qwen3-4B-Q5_K_S.gguf	~2.7 GB	Slightly smaller Q5 variant
Q4_K_M	Luck-Qwen3-4B-Q4_K_M.gguf	~2.5 GB	Recommended: Optimal speed/quality balance
Q4_K_S	Luck-Qwen3-4B-Q4_K_S.gguf	~2.3 GB	Compact 4-bit quantization
IQ4_NL	Luck-Qwen3-4B-IQ4_NL.gguf	~2.2 GB	Non-linear 4-bit quantization
IQ4_XS	Luck-Qwen3-4B-IQ4_XS.gguf	~2.1 GB	Extra small 4-bit quantization
Q3_K_M	Luck-Qwen3-4B-Q3_K_M.gguf	~1.9 GB	Medium 3-bit quantization
Q3_K_S	Luck-Qwen3-4B-Q3_K_S.gguf	~1.8 GB	Small 3-bit quantization
IQ3_M	Luck-Qwen3-4B-IQ3_M.gguf	~1.7 GB	Advanced 3-bit quantization
IQ3_XS	Luck-Qwen3-4B-IQ3_XS.gguf	~1.5 GB	Extra small 3-bit quantization
Q2_K	Luck-Qwen3-4B-Q2_K.gguf	~1.4 GB	High compression, lower accuracy
IQ2_M	Luck-Qwen3-4B-IQ2_M.gguf	~1.3 GB	Extreme compression

Usage Instructions

Use with llama.cpp

Run directly using Hugging Face integration:

./llama-cli -hf ahmetggg/Luck-Qwen3-4b-Code-FineTune-GGUF:Q4_K_M -p "Write a Python script to parse JSON."

Use with Ollama

Run directly via the Hugging Face hub integration:

ollama run hf.co/ahmetggg/Luck-Qwen3-4b-Code-FineTune-GGUF:Q4_K_M

Use with LM Studio

Search for ahmetggg/Luck-Qwen3-4b-Code-FineTune-GGUF within the LM Studio search bar and download your desired quantization directly.

Note: Standard downstream benchmarks (SWE-bench, BFCL, etc.) have not been conducted yet; results will be published once available.