turintech/Kev-4B-Artemis-GGUF

Kev-4B Artemis GGUF

This is a decision model: use llama-server's /v1/systemone endpoint with -ub 4096 -b 4096. Follow Run it for supported settings and app compatibility.

Start here. Download Kev-4B-Artemis-Q4_K_M.gguf (2.54 GB). 93.0% same answers as BF16; public Q4_K_M loses less accuracy on the additional evaluation. Details.

Typed-decision accuracy on the card's selection panel, with the measured nearby-size alternatives and BF16 original. These comparisons informed file selection and do not establish performance on unseen inputs. Hardware, runtime and other evaluation panels are listed under How it was tested.

These comparisons informed file selection; they do not establish performance on unseen inputs.

Earlier file names still work for now; new downloads should use the names below.

llama-server -hf turintech/Kev-4B-Artemis-GGUF:Q4_K_M --port 8080 -c 16384 -ub 4096 -b 4096

TurinTech models, built with Artemis

Kev-4B is a 4B decision model by Jared Palmer. The model accepts text with a set of typed questions, and it returns a probability for every allowed answer in a single pass, instead of generating text. Text only. See the original model card.

This is an independent TurinTech conversion, not affiliated with or endorsed by the Kev authors. The files are standard GGUF and run on stock llama.cpp. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp. See NOTICE.

Why use this file

• Run locally so inputs can stay on the same machine.
• Work offline after downloading the model and runtime.
• Route requests, triage messages or classify inputs with typed answers and probabilities.
• Plan a local setup with the default file's measured 7.6 GiB peak GPU memory; hardware and settings are listed below.

Files

Variant	Size	Saved	Same answer	Acc. Δ	Conf. changes	Choose for
BF16 original	8.43 GB		reference	reference
Q4_K_M	2.54 GB	70%	93.0%	0.0	0	Default.
IQ3_M	2.17 GB	74%	90.3%	+0.4	0	A little less memory.
Q3_K_M	2.04 GB	76%	88.6%	-2.2	0	Tight memory.

"Saved" is the file-size reduction versus the BF16 original. "Acc. Δ" is the accuracy change in percentage points; the other columns are defined below. Variant links download the exact GGUF; full filenames are in the commands and checksums. File labels are size classes, explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Q3_K_M: Tight memory; about 2 to 3 points lower accuracy.

"Same answer as the original" is how often a file picks the same option as the BF16 original on 2,000 test decisions. "Accuracy change" is the difference in accuracy against the original on the same cases, in points. "Confident answers changed" counts answers the original was more than 80% sure of that the file changed.

The usual public files, on the same tests: Q8_0 (ggml-org, 4.48 GB) gives the same answer 98.4% of the time with -0.3 accuracy, and Q4_K_M (ggml-org, 3.03 GB) 93.2% with 0.0. The tables report both Q4_K_M files on these test decisions. The ggml-org file is closer to the original on held-out decisions (94.0% against 92.7%, accuracy -0.1 against -1.7) and on the benchmark sample (96.0% against 94.0%).

Run it

Needs llama.cpp from 9 October 2026 or newer (tested at commit 1e6f04a).

llama-server -m Kev-4B-Artemis-Q4_K_M.gguf --port 8080 -c 16384 -ub 4096 -b 4096

Tested on an Apple M3 Max (36 GB) with llama.cpp Metal (b11617): every file here loads and answers with the command above.

Keep -ub 4096 -b 4096: the numbers on this card were measured with them, and the batch size moves the probabilities slightly. In our test on an NVIDIA RTX 5090, Kev-4B-Artemis-Q4_K_M.gguf gave 0.803 at the default of 512 and 0.815 at 4096 on an 854-token input. Inputs longer than the batch size are accepted: with llama.cpp 1e6f04a and b11609, a 4,914-token input was answered at both settings. -c 16384 stops the server from reserving memory for the full training context.

For concurrent clients, an optional configuration is -np 8 -c 32768 --cache-ram 0: eight request slots with 4,096 tokens each and the host prompt cache disabled. In our NVIDIA RTX 5090 test with llama.cpp fcac190, this used about 0.6 GiB more GPU memory than the command above. Measure the configuration on the intended workload before choosing it.

llama-server -m Kev-4B-Artemis-Q4_K_M.gguf --port 8080 -np 8 -c 32768 -ub 4096 -b 4096 --cache-ram 0

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

Recommended Ollama package

echo "My Stripe integration keeps failing. Please help ASAP." | ollama run turintech/kev-4b-artemis:2.5gb --ask "Does this message express urgency?"

This verified tag selects Kev-4B-Artemis-Q4_K_M.gguf. It is a decision model: use /v1/systemone, not a chat prompt. Ollama 0.40.4 was tested; include "model": "turintech/kev-4b-artemis:2.5gb" in the decision request to port 11434. The command runs the documented decision example; the explicit size tag selects the recommended file and is not an immutable revision.

Run it with Ollama

Tested with Ollama 0.40.4 on 14 October 2026, on a CPU. Download the file and import it with a 16K context:

hf download turintech/Kev-4B-Artemis-GGUF Kev-4B-Artemis-Q4_K_M.gguf --local-dir .
printf 'FROM ./Kev-4B-Artemis-Q4_K_M.gguf\nPARAMETER num_ctx 16384\n' > Modelfile
ollama create kev-4b-artemis -f Modelfile
echo "My Stripe integration keeps failing. Please help ASAP." | ollama run kev-4b-artemis --ask "Does this message express urgency?"

The last command prints yes 93%. Ollama serves the same decision endpoint on its own port; add a model field:

curl http://localhost:11434/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "kev-4b-artemis",
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

Without num_ctx 16384, Ollama's default context of 4,096 tokens rejects longer inputs. The numbers on this card were measured with llama-server. Ollama sets its own batch size, so the probabilities can differ slightly: 0.927 against 0.926 from llama-server for this example on the same machine.

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Evaluation details and limitations

How it was tested

Compatibility notes

This is a decision model. Run it on llama-server's /v1/systemone endpoint and keep -ub 4096 -b 4096, as in Run it, or import it into Ollama as in Run it with Ollama. The llama.cpp and LM Studio buttons under "Use this model" start it as a chat model, which is not a supported way to run it. The Ollama button runs it as a decision model and downloads the recommended Q4_K_M file by default; the :Q4_K_M tag makes that explicit.

Recommendation notes

Download Kev-4B-Artemis-Q4_K_M.gguf (2.54 GB, peak GPU memory 7.6 GiB on an NVIDIA RTX 5090) and start it with the settings below, then send requests as shown in Run it. It gives the same answer as the original on 93.0% of 2,000 test decisions and changed none of the 244 answers the original was more than 80% sure of. Go smaller only if memory is tight. On held-out decisions it is 1.7 points less accurate than the original, where the public Q4_K_M (3.03 GB) loses 0.1.

These files are per-tensor mixes, and their labels are size classes (see the file names table at the end). Use the exact filename or the :Q4_K_M tag; without a tag, app buttons may select a different file.

Size and quality trade-offs

jaredpalmer/kev-4b is 8.43 GB as BF16 and needs 13.1 GiB of GPU memory. The default file here is 2.54 GB, 70% smaller, and needs 7.6 GiB (both measured on an NVIDIA RTX 5090). It gives the same answer as the original on 93.0% of 2,000 test decisions. The public Q4_K_M records 93.2%. Accuracy is 66.8% against the original's 66.8% (on held-out decisions it is 1.7 points lower). The file changed none of the 244 answers the original was more than 80% sure of.

The separate train-split results are labelled held-out in the tables. Their independence from calibration and variant selection has not been verified for every file.

Test results informed candidate selection, so they do not independently measure performance on new inputs. Read them alongside the separate train-split results and public-file comparisons. Agreement measures whether a compressed model keeps the original's answers, including its mistakes. The typed-decisions accuracy figures compare answers with teacher-generated labels.

Every file was compared with the BF16 original and with the public builds, all at identical llama.cpp settings, on:

• 2,000 typed decisions (questions with a fixed answer type: yes/no, pick one, a score) from the test split of LocalLLaMA/typed-decisions: 400 cases with about five questions each,
• 1,000 further decisions from the dataset's train split, reported separately below,
• a 300-item stratified sample of the public Decision Index benchmarks (MMLU, MMLU-Pro, ARC, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150, When2Call). This is a sample for comparing builds, not a Decision Index score,
• option shuffling, the longest inputs, many-option questions and repeat runs.

Measured with llama.cpp 1e6f04a at the settings above (--parallel 2, all layers on the GPU), every file on one NVIDIA RTX 5090 32 GB. Public files: ggml-org/Kev-4B-GGUF revision d924f2e.

Check	BF16	Artemis Q4_K_M	Artemis IQ3_M	Artemis Q3_K_M	Public Q8_0	Public Q4_K_M
Typed-decision accuracy	66.8	66.8	67.1	64.5	66.5	66.8
Same answer as BF16		93.0	90.3	88.6	98.4	93.2
Held-out accuracy	66.1	64.4	64.2	63.4	65.9	66.0
Held-out: same answer as BF16		92.7	91.3	87.1	98.7	94.0
Decision Index sample accuracy	72.0	71.3	69.3	71.0	71.3	72.3
Decision Index sample: same answer as BF16		94.0	90.3	90.7	98.0	96.0
Changed answers where BF16 was over 80% sure		0	0	0	0	0

The original itself changes 8.6% of its answers when the options are shuffled; our files change 9.9 to 11.6%, the public Q8_0 9.7% and public Q4_K_M 8.9%. Repeat runs gave identical answers for every file. Every number here sends all of a case's questions in one request; sending them one per request changes 25 to 35 of the 2,000 answers for our files (none for the original), so keep the request shape the same when comparing.

Memory

Peak GPU memory of llama-server on an NVIDIA RTX 5090 with the settings above.

File	Size	Peak GPU memory
Original, BF16	8.43 GB	13.1 GiB
Kev-4B-Artemis-Q4_K_M	2.54 GB	7.6 GiB
Kev-4B-Artemis-IQ3_M	2.17 GB	7.2 GiB
Kev-4B-Artemis-Q3_K_M	2.04 GB	7.1 GiB
Public Q4_K_M	3.03 GB	8.0 GiB

File names, sizes and SHA-256 checksums

File names

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. Each file is a per-tensor mix of quant types, so the label is a size class: the smallest standard llama.cpp quant whose typical bits per weight are at least the file's. The general.file_type field inside the files reads Q4_K_M, the base type given to llama-quantize; the table shows the tensor types each file holds.

File	Earlier name	Bits per weight	Main tensor types (share of bytes)
Kev-4B-Artemis-Q4_K_M.gguf	Kev-4B-Artemis-2.5GB.gguf	4.81	IQ4_XS 48%, Q6_K 29%, Q4_K 23%
Kev-4B-Artemis-IQ3_M.gguf	Kev-4B-Artemis-2.2GB.gguf	4.10	IQ4_XS 32%, Q4_K 27%, IQ3_S 19%, IQ3_XXS 11%
Kev-4B-Artemis-Q3_K_M.gguf	Kev-4B-Artemis-2.0GB.gguf	3.86	IQ3_XXS 40%, Q4_K 29%, IQ4_XS 20%, Q6_K 11%

The earlier names still download the same bytes, with the same checksums, for a transition period.

File checksums

File	Bytes	sha256
Kev-4B-Artemis-Q4_K_M.gguf	2,541,447,616	b8a3a2e8605b0b9f28a7cfb15d7b4a003ad55306c6bca3c055a62bad18b926fd
Kev-4B-Artemis-IQ3_M.gguf	2,165,086,656	02661bf89d0ae9ce19d693f93e81c173c08322058585171eed1665accbb4800e
Kev-4B-Artemis-Q3_K_M.gguf	2,039,196,096	7b82f0d0d1c11f990d56a1824fe152005cd286a87a2c68e5094acacb1ea769cf

Custom versions

We also make fine-tuned and further optimised versions on request: tuned on specific decisions and data, built for the target hardware (CPU servers, laptops, Apple Silicon, edge devices), or compressed from a private model. Book a call with TurinTech.

Licence

Apache-2.0 (see LICENSE and NOTICE), the same as jaredpalmer/kev-4b, which is fine-tuned from Qwen/Qwen3.5-4B-Base.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.