---
license: apache-2.0
base_model: Qwen/Qwen3-4B
tags:
- lora
- gguf
- tool-calling
- function-calling
- reproducible-training
- xyntetik-runner
---
# Qwen3-4B-Runner-ToolUse-Q4_K_M
> **Release: built to be downloaded and served with [Xyntetik Runner](https://github.com/Joakimpalm-Zen/xyntetik-runner).**
> - **Measured status against the parent, limit first:** off its training distribution this adapter is no better than the base. On 274 hand-written prompts over a 23-tool catalog, the base made 211 exact calls through the chat API and this adapter 200 ([the shifted set](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/evals/tooluse-shifted/README.md#set-v2-2026-09-24-the-redesign-and-can-the-instrument-fail)). On its own task, held-out exact tool-call score 0.690 for the base Q4_K_M, 1.000 with this adapter (29 prompts, greedy); byte-reproducible training with a provenance record.
> - **Parent model:** [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) at Q4_K_M
> - **Collection:** [Runner Releases](https://huggingface.co/collections/Joakimpalm-Zen/runner-releases-6aa98baaed03bba0e8a561ae), also in [LoRA Training on Quantized Weights](https://huggingface.co/collections/Joakimpalm-Zen/xyntetik-research-lora-training-on-quantized-weights-6aa98bb02859827a8d86618c)
> - **Evidence:** [training record, evals and the merge study](https://huggingface.co/datasets/Joakimpalm-Zen/Qwen3-4B-ToolUse-LoRA-training-record) (dataset)
> - **Runner compatibility:** `--train`, `--lora` and `--merge-lora` in Runner; merge to F16 or Q8_0, never to 4-bit (measured erasure). Every adapter feature, from continued training to per-request serving and signing: [the complete adapter guide](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/lora.md).
>
> - **Validation status (Runner support tiers):** each file's row in the Runner roster says which tier it is in, what is re-checked on a release and the date and version of the last run.
> - `merged-Q8_0.gguf`: [roster row](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/model-support.md#qwen3-4b-runner-tooluse-merged-q8_0)
> - `Qwen3-4B-ToolUse-LoRA.gguf`: [roster row](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/model-support.md#qwen3-4b-runner-tooluse-lora)
>
**Want this for your own data?** The adapter was trained through the 4-bit file it is served with; the [training walkthrough](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/train-lora-on-quantized-gguf.md) has the commands, and [the complete adapter guide](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/lora.md) explains every adapter feature.
[Request a model](https://github.com/Joakimpalm-Zen/xyntetik-runner/issues/new?template=model_request.yml) · [Report your result](https://github.com/Joakimpalm-Zen/xyntetik-runner/issues/new?template=result_report.yml) · [Community results](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/community-results.md) · [All files](https://huggingface.co/Joakimpalm-Zen)
If this file is useful to you, a contribution funds the hardware time behind the next one.
**What this is, and is not.** A LoRA adapter for one narrow, synthetic
tool-calling task (four tools, 158 generated training examples, 29 held-out
prompts), published to show the training mechanism and its reproducibility.
It is not a general tool-use upgrade for Qwen3-4B, and the training recipe
(316 steps at lr 1e-4 on 158 examples) is memorization-grade, right for a
format habit and wrong for a knowledge task. Measured off its task, it does
not help: on 274 hand-written prompts unlike its training, over 23 tools with
overlapping names, the base makes more exact calls through the chat API than
the adapter (211 to 200), and the adapter helps only where no tool fits
([the shifted set](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/evals/tooluse-shifted/README.md#set-v2-2026-09-24-the-redesign-and-can-the-instrument-fail)).
**This LoRA was trained directly through the quantized GGUF that serves it.**
No FP16 training copy. No Python training framework. The trainer's forward
pass *is* the inference forward pass: [Xyntetik Runner](https://github.com/Joakimpalm-Zen/xyntetik-runner)
(a single-binary C11 inference engine) trained this adapter against the
frozen `Q4_K_M` inference artifact on CPU, and the result is
**byte-reproducible**: run the same command on the same inputs with the same
build and you get this exact file, sha for sha. "Same build" is part of the
claim: an independent tester reproduced the bytes on a Tesla T4 and also
found that a binary rebuilt under a different instruction-set profile writes
different adapter bytes, so the reproduction is a property of the build, the
way Runner's receipt replay is.
What the mechanism covers, what it refuses and what it costs per step:
[the manual's adaptation section](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/MANUAL.md#adaptation).
## TL;DR
**Off its task the adapter does not help:** 274 hand-written prompts over 23
tools, exact calls 211 for the base and 200 with the adapter through the chat
API (170 and 176 on the training template, inside noise;
[the shifted set](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/evals/tooluse-shifted/README.md#set-v2-2026-09-24-the-redesign-and-can-the-instrument-fail)). **On its task:**
| | json parses | right tool | schema args | exact call |
|---|---|---|---|---|
| Qwen3-4B Q4_K_M, base | 1.000 | 0.724 | 0.724 | 0.690 |
| **+ this adapter** | **1.000** | **1.000** | **1.000** | **1.000** |
Held-out tool-calling eval (29 prompts, greedy, temperature 0): pick the
right tool among four schema-distinct tools, emit schema-shaped JSON args,
refuse requests no available tool fits, no prose around the JSON.
## Reproducibility record (measured, not asserted)
Everything below is from the machine-written provenance record
(`Qwen3-4B-ToolUse-LoRA.gguf.train.json`, committed beside the adapter):
| | sha256 |
|---|---|
| base (`Qwen_Qwen3-4B-Q4_K_M.gguf`, [bartowski's conversion](https://huggingface.co/bartowski/Qwen_Qwen3-4B-GGUF)) | `fbe1d5edd4ce802ae3ae7c7e4ab7d09789d697fdac1fc7929f8df4ca3c41bae3` |
| dataset (`data/train.jsonl`, deterministic generator) | `4c3feca6afa9d776b0e5d08afb6ac4b134bee93a94548dd934a74b2c944d6a5a` |
| **this adapter** | `ea38f80c33d381c9aa62c874920d2fa058dd307e7186e9b7c347d782f35b88e4` |
Config: rank 8 on every projection (attention q/k/v/output + FFN
gate/up/down), alpha 16, AdamW (0.9/0.999, wd 0.01), lr 1e-4, 316 steps
(2 epochs of 158 examples), seed 0, ctx 128. Loss 0.676 to 0.000006.
**Determinism was verified on this exact base before training the release
adapter**: two independent 40-step runs with identical inputs produced
byte-identical adapter files
(`35a19a2435fd717875929a098019250c7b7b5c3e1b669abec87e20b3fa1916f1`, both).
Runner gates this property in its test suite (`same data + same seed →
byte-identical adapter`); the engine design notes are in
[docs/adaptation-engine.md](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/adaptation-engine.md).
## Use it
```sh
# serve the base with the adapter (on the CPU here; on CUDA and Metal an
# offloaded block applies the adapter on the device: `runner -m ... --adapt-info`
# says which backend of your build carries adapter kernels)
runner -m Qwen_Qwen3-4B-Q4_K_M.gguf --lora Qwen3-4B-ToolUse-LoRA.gguf --serve --gpu off
# or reproduce the adapter from scratch: the same sha256 on the same build
python3 scripts/make-tooluse-data.py data/
runner -m Qwen_Qwen3-4B-Q4_K_M.gguf --train data/train.jsonl \
--train-steps 316 --lr 1e-4 --train-out adapter.gguf
```
Any Runner release serves it ([releases](https://github.com/Joakimpalm-Zen/xyntetik-runner/releases));
this adapter was produced at commit `3f7cd50` (the v0.2.0 training code),
and the adapter bytes are gated identical across later binaries, thread
counts and the CUDA assist on the same instruction-set profile. The adapter
GGUF uses the llama.cpp adapter naming convention.
## More you can do with this adapter
Each item below is explained, with its limits, in
[the complete adapter guide](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/docs/lora.md).
- **Serve it per request.** `runner -m Qwen_Qwen3-4B-Q4_K_M.gguf --serve --gpu off
--adapter tools=Qwen3-4B-ToolUse-LoRA.gguf` loads it once; a request naming
`"model": "Qwen_Qwen3-4B-Q4_K_M.gguf:tools"` gets the adapter, and one without
the suffix gets the bare base, side by side on parallel slots.
- **Evaluate it.** `--score -f heldout.txt`, with and without `--lora`, gives
teacher-forced logprobs, perplexity and top-1 rate for both.
- **Train it further.** `--lora Qwen3-4B-ToolUse-LoRA.gguf --train more.jsonl
--train-out next.gguf` continues from this adapter. Since Runner 1.4.2 the new
record names this file as `from_adapter`, so `--lineage next.gguf` walks back
through it to the base and the data.
- **Train on exact tokens, and see what a step trains on** (Runner 1.4.2). A
training line can give `prompt_tokens` and `completion_tokens` instead of
text, and `--train-logprobs FILE` reports the logprob of every target token
under the forward each step trains through.
- **Preferences and rewards.** `--train-dpo` trains from chosen and rejected
answers against the frozen base, and `scripts/train-grpo-lite.py` runs a
reinforcement loop whose every round checks that the trainer scores the
sampled rollouts bit for bit as the server did (Runner 1.4.2).
- **Sign it and require the signature.** `runner --sign-model
Qwen3-4B-ToolUse-LoRA.gguf --model-key key.pem` writes an OpenSSF Model Signing
bundle; a server started with `--model-pubkey key.pub --require-signed-model`
then refuses the adapter if its bytes or signature do not verify.
- **Trace it.** `runner --lineage Qwen3-4B-ToolUse-LoRA.gguf` checks the
adapter against its `.train.json` record and names the base and the data it
was trained from, by sha256 (each re-hashed where the file is on disk).
## The three-precision study (`study/`)
The same task, same seed, same data, same config, trained through **three
precisions of the same base**: does the quantization you train *through*
change what you learn?
| trained through | eval on own base | eval on Q4_K_M base | adapter sha256 (first 16) |
|---|---|---|---|
| bf16 | 1.000 exact | 1.000 exact | `24d5f02122bc7436` |
| Q8_0 | 1.000 exact | 1.000 exact | `a2c70991d7421bab` |
| Q4_K_M | 1.000 exact | n/a | `eda3c58491f9c654` |
Weight-space divergence between the learned adapters (all-parameter cosine
and relative L2):
| pair | cosine | rel L2 |
|---|---|---|
| bf16 vs Q8_0 | **0.99983** | 0.019 |
| bf16 vs Q4_K_M | **0.99259** | 0.122 |
| Q8_0 vs Q4_K_M | 0.99276 | 0.120 |
Read: training through Q8_0 learns essentially the same adapter as training
through bf16 (2% relative difference). Training through the 4-bit base
learns a **measurably different adapter** (12% relative divergence), so
"fine-tune FP16, then quantize for deployment" and "adapt the deployed
quant directly" are *not the same operation*; yet on this task's
**supervised, wide-margin** decisions the difference costs nothing:
every adapter scores 1.000, and the bf16- and Q8-trained adapters drop
onto the Q4_K_M serving base at 1.000 too. Read that scope literally:
it is **not** free everywhere, and the two refinements below show where
the cost actually shows up. All three study adapters and their
provenance records are in `study/`.
**Follow-up: the divergence is behavioral** (`evals/logprob-study/`).
Prompted by an independent reproduction of this repo's determinism claims
(HF forum; the same tester verified adapter byte-equality on a Tesla T4 and
correctly proposed logprobs as the next instrument): all 29 held-out
prompts with gold completions, scored through the same Q4_K_M serving
base under each study adapter, 3,195 positions.
| pair | weight rel L2 | mean \|Δlogprob\| | positions >1 nat |
|---|---|---|---|
| bf16-ad vs Q8-ad | 0.019 | 0.020 | 0% |
| bf16-ad vs Q4-ad | 0.122 | **0.146** | 2.2% |
| base vs any adapter | n/a | 0.32 to 0.40 | 6 to 8% |
The Q4-trained adapter differs from the bf16-trained one by ~40% of the
entire adapter effect in logprob space; the 6x weight-space gap maps to a
7.4x logprob gap.
**Refinement (2026-08-25, from the same independent reviewer who
reproduced the determinism claims):** splitting those positions by
prompt vs gold completion changes what the aggregate measures. On the
supervised completion region the three adapters nearly coincide
(bf16-vs-Q4 mean |dlogprob| ~0.0003 nat, no position near 1 nat,
verified from the published files); almost all of the 0.146 aggregate
lives in the UNSUPERVISED prompt region. So the honest reading is:
the precision path leaves a real function-level fingerprint, largely
outside what the completion loss supervised, while the supervised
decisions stay wide-margin. Those are two different observables. The
same reviewer then supplied the missing piece: on deliberately
ambiguous requests the fingerprint DOES cross a decision boundary. On
"tell me what README.md says and translate it", the bf16- and
Q8-trained adapters call read_file (margins ~0.49 nat) while the
Q4-trained adapter confidently answers none (margin 2.17 nat), and the
split survives full deterministic JSON generation. One constructed
case, an existence proof rather than a prevalence estimate, and
exactly the "where does weight divergence become behavioral" answer
this study was reaching for. A small decision-boundary lane (top1/top2
margins over ambiguous tool choices) is the planned follow-up.
**Systematized (2026-08-30, same reviewer).** That constructed case is
now a screened one. He built a 36-prompt tool-choice boundary bank
across five ambiguity families (list_dir/search_files,
read_file/search, read_file/write_file multi-intent, config browsing,
and available-tool/none), kept the cases whose top1-top2 legal-choice
margin sits at or under 3 nat, and ran the base plus all three study
adapters on the survivors, generating the full JSON deterministically
wherever the branches disagreed. Seven cases passed the screen, and
the split above is the one that reproduced. Three constraints travel
with it, all his, and they bound what this result may be used for:
- The screen selects on the Q4 adapter's own margin, so the seven
selected cases cannot estimate any general margin distribution. One
split out of seven out of 36 is an existence proof, not a rate.
- **There is no monotonic precision law here.** Several cases had
smaller Q4 margins, not all did, and one case had Q8 as the tightest
condition. What this supports is that some decision boundaries are
far more sensitive to the training precision path than others,
non-monotonically in bit width. It does not support "lower training
precision degrades tool choice", which his own data refutes.
- The bank carries no gold labels, deliberately: labeling would
convert a decision-sensitivity probe into a quality benchmark and
quietly change the question.
Notebooks, clean and executed with reference outputs:
[John6666/forum3](https://huggingface.co/datasets/John6666/forum3/tree/main/deterministic_GGUF_LoRA_training).
## The merge study (`merged/`), or: why this repo ships an adapter
Runner's `--merge-lora` folds the adapter into the base weights
(`W' = W + (alpha/r)·B·A`, deterministic: same inputs → byte-identical
merged file) so the result runs in **any** GGUF runtime. We merged this
adapter into three output precisions of the same Q4_K_M base and re-ran
the held-out eval on each standalone file:
| merged into | right tool | exact call | verdict |
|---|---|---|---|
| Q4_K_M (base's own type) | 0.724 | 0.690 | **adapter erased: scores exactly the base** |
| Q8_0 | **1.000** | **1.000** | behavior fully preserved |
| F16 | **1.000** | **1.000** | behavior fully preserved |
The 4-bit merge is not slightly worse: it reproduces the base's numbers
to the prompt. Byte-level comparison shows why: after requantizing to
Q4_K_M, only **1.45% of weight bytes differ from the base**; the LoRA
delta is small relative to the 4-bit grid step, so quantization rounds it
back to the original codes almost everywhere. Runner 1.2.1 measures this
directly: `--merge-lora` projects the merged difference onto the intended
delta and records `delta_retained` in the `.merge.json`; on this adapter
the Q4_K_M merge kept **1.8%** of the delta (3.0, 4.7 and 8.9% at 2, 4 and
8x scale) and the Q8_0 merge **99.5%**. A merge that keeps under half is now
refused with the destination untouched; `--merge-allow-erased` writes it
anyway, which is what reproducing the erased file below requires. The visible symptom (via
stock llama.cpp b10581, raw completion): the Q8_0 and F16 merges answer
with JSON only, exactly as trained; the Q4_K_M merge answers, then drifts
into conversational prose, the base's behavior.
**Deployment guidance, measured:** to serve 4-bit with this adaptation,
serve `base + --lora` (exact, and how the 1.000 row above is produced).
To ship a standalone merged file, merge into Q8_0 or wider and *measure
it*: this repo's `merged/merged-Q8_0.gguf` is that artifact, verified
1.000 on the eval and runnable in stock llama.cpp. Merging into a 4-bit
base is where fine-tunes silently disappear; if you do it anywhere, eval
the merged file, not the adapter.
**The survival threshold, measured** (follow-up sweep): the same adapter
merged at five `--lora-scale` values into the same Q4_K_M base, every
merged file evaled standalone, served base+adapter as the control:
| scale | merged-Q4 exact call | served exact call | bytes changed |
|---|---|---|---|
| 0.5 | 0.690 (base) | 1.000 | 1.15% |
| 1 | 0.690 (base) | 1.000 | 1.45% |
| 2 | 0.690 (base) | 1.000 | 2.05% |
| 4 | 0.828 | 1.000 | 2.87% |
| 8 | **1.000** | **0.138** | 4.60% |
Survival is monotone in delta magnitude, as the rounding mechanism
predicts, and note the inversion at 8x: the *exact* over-scaled adapter
breaks the served model, while the 4-bit grid filters the same weights
back to 1.000. Quantization acts as a filter on your fine-tune with a
pass-band you don't control; since 1.2.1 that 8x merge is refused by
default for the same reason. The sweep evals are in `evals/sweep-*.json`.
**Interop, both directions**: this adapter scores the same
1.000/1.000/1.000/1.000 on the full held-out eval when served by *stock
llama.cpp* (b10581, `llama-server --lora`, raw completion); the eval
output is `evals/llamacpp-served-adapter.json`. And as of runner main,
`--lora` accepts the F16 adapters llama.cpp's `convert_lora_to_gguf`
emits (found by loading a community adapter, which the old F32-only
check refused).
The Q4_K_M and F16 merges are not uploaded (one is behaviorally the base,
the other is 8 GB); their provenance records with sha256s are in
`merged/`, and because the merge is deterministic, one command reproduces
either file byte-for-byte:
```sh
runner -m Qwen_Qwen3-4B-Q4_K_M.gguf --lora Qwen3-4B-ToolUse-LoRA.gguf \
--merge-lora merged-Q4_K_M.gguf --merge-allow-erased # sha256 17443f0d55f5bb33...
```
(Runner 1.2.1 or later refuses this merge without `--merge-allow-erased`,
because it keeps 1.8% of the delta; earlier releases wrote it silently.)
## Honest limits
- Off its task the adapter does not help. On the shifted set (274
hand-written prompts over 23 tools with overlapping names, 2026-09-24) the
base made 211 exact calls through the chat API and the adapter 200 (base
favoured, z 2.2), and 170 and 176 on the training template (inside noise).
On underspecified requests the adapter is more certain of the wrong tool.
It helps only where no tool fits (30 of 34 against the base's 25).
[The record](https://github.com/Joakimpalm-Zen/xyntetik-runner/blob/main/evals/tooluse-shifted/README.md#set-v2-2026-09-24-the-redesign-and-can-the-instrument-fail).
- The task is narrow and synthetic (four tools, template-generated
requests, 29 held-out prompts). This demonstrates the training mechanism
and its reproducibility, not general tool-use capability.
- 316 steps at lr 1e-4 on 158 examples is memorization-grade training,
appropriate for a format task, not a recipe for knowledge tasks.
- The precision-study equivalence is measured **on this task**; harder
tasks may separate the adapters. That experiment is now a one-command
rerun for anyone, which is rather the point.
- Training is slow by Python-framework standards. This adapter took
~45 s/step at 4B on a 128-thread EPYC-class host with runner v0.2.0;
current builds train the same config at ~20 s/step there, and a 7B
Q4_K_M at 64.8 s/step on an RTX 3070 with the CUDA assist
(`RUNNER_TRAIN_GPU=1`, 76.5 s on 8 CPU threads). Budget hours for a
few hundred steps. The adapter bytes are gated identical across the old
binary, the new one, any thread count and the CUDA assist on the same
instruction-set profile, so the reproduce-the-sha command works
unchanged on current builds; a different profile is a different build
and can write different bytes.
- Coverage is dense families (Llama, Mistral, Qwen2.5, Granite 4.x, Gemma 3
and 4) plus attention-only adapters on sparse-MoE models with Qwen3-MoE or
Mixtral routing; expert adapters, other routers, shared experts and
recurrent blocks are refused by name.
## Files
- `Qwen3-4B-ToolUse-LoRA.gguf`: the adapter (+ `.train.json` provenance)
- `data/`: the deterministic dataset (train + held-out eval)
- `evals/`: every eval JSON quoted above, unedited (incl. the three
merged-artifact evals)
- `study/`: the three precision-study adapters, provenance, divergence
- `merged/`: the standalone merged Q8_0 GGUF (runs in any GGUF runtime,
verified 1.000 on the eval and smoke-tested in llama.cpp b10581) +
`.merge.json` provenance for all three merge targets
## About Xyntetik
This repository is published by [Xyntetik](https://xyntetik.com), the Zenova AB program for running capable language models on hardware you own and proving what they did. Its released engine is [Xyntetik Runner](https://github.com/Joakimpalm-Zen/xyntetik-runner): a free, single-binary GGUF engine in C for CPU, CUDA and Metal that serves, scores and trains through the quantized file it deploys and records every run as a receipt. The measurements behind the program's claims are collected at [xyntetik.com/evidence](https://xyntetik.com/evidence/). The program's organization on Hugging Face is [Xyntetik](https://huggingface.co/Xyntetik).