Qwen3.5-0.8B-ft-decision-8k

Overview

This model is a fine-tuned version of Qwen3.5-0.8B on the typed-decisions dataset with a maximum context length of 8192 tokens.

It takes a free-text input (a support ticket, an agent trace, an invoice, a security alert...) plus a schema of typed questions and returns, for every question, a calibrated probability distribution over its possible answers rather than a single label. Three question types are supported:

choice — pick one option out of a set of named criteria (e.g. which team should handle a ticket).
noul — binary yes/no decision (e.g. does the customer ask for a refund?).
score — ordinal scale (e.g. urgency from "not urgent" to "today"); an expected score is also returned.

The dataset covers four workflows: agent_trace_observability, customer_service, invoice_processing and security_incidents (1.6k records in total, train/test splits). Gold labels are soft distributions aggregated from several annotators, so the model is trained to match the full distribution, not only the argmax.

Training Configuration

Dataset: LocalLLaMA/typed-decisions (all subset)
Max Context Length: 8192
Seed: 3407

Training Args

Per Device Train Batch Size: 64
Gradient Accumulation Steps: 1
Effective batch size of 64.
Warmup Steps: 10
Total Training Epochs: 10 (180 steps)
Learning Rate: 2e-4
LR Scheduler Type: cosine
Weight Decay: 0.01
Precision: bf16 (falls back to fp16 when bf16 is not supported)
Evaluation / Save Strategy: epoch
Load Best Model at End: True (selected by validation loss)

Training Progress

Epoch	Training Loss	Validation Loss
1	1.153736	1.001035
2	0.944459	0.899672
3	0.836740	0.873418
4	0.824629	0.847806
5	0.796421	0.858162
6	0.800943	0.847742
7	0.792374	0.845596
8	0.784226	0.845760
9	0.785770	0.846267
10	0.780439	0.846091

Validation loss plateaus from epoch 4 onwards. The released checkpoint is the one from epoch 7, which has the lowest validation loss.

LoRA Config

rank: 128
alpha: 64
dropout: 0
gradient checkpointing: "unsloth"
random state: 3407

Evaluations

Metrics:

accuracy: fraction of individual questions where the predicted label matches the gold argmax.
record_accuracy: fraction of records where every question is answered correctly.
ece: expected calibration error (lower is better).
loss: cross-entropy against the soft gold distributions.

Before vs. after fine-tuning

Model	Accuracy	Record Accuracy	ECE ↓	Loss
Qwen3.5-0.8B (base)	40.5%	0.0%	0.0796	1.210
Ours (fine-tuned)	83.0%	42.5%	0.0412	1.214

Test split

Model	Accuracy	Record Accuracy	ECE ↓	Loss
Ours	80%	38.75%	0.0377	1.279

Intended Use

The model is intended for research and experimentation on structured, calibrated decision-making: routing, triage, review queues and observability over workflows such as customer support, invoice processing, security incidents and agent traces. Because it outputs probabilities, it is well suited to thresholding (e.g. escalate to a human below a confidence level). Its performance should be validated on your own workflows and question schemas before deployment in production.

Example Usage (unsloth)

from unsloth import FastDecisionModel

model, tokenizer = FastDecisionModel.from_pretrained(
    model_name = "mrm8488/Qwen3.5-0.8B-ft-typed-decisions",
    max_seq_length = 8192,
)

FastDecisionModel.for_inference(model)

answers = FastDecisionModel.predict(
    model,
    tokenizer,
    "Hi, I was charged twice for invoice #4411. Please refund the duplicate today.",
    {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this?",
            "criteria": {
                "billing": "invoices, payments, refunds",
                "technical": "bugs, outages, errors",
                "sales": "pricing, new plans",
            },
        },
        "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
        "urgency": {
            "type": "score",
            "instructions": "How urgent is this?",
            "criteria": ["not urgent", "soon", "today"],
        },
    },
)

for name, result in answers.items():
    print(name, result["answer"], {k: round(v, 3) for k, v in result["probabilities"].items()})

Quantized Versions

GGUF quantizations for llama.cpp / LM Studio / Ollama are available at

Quant	Use case
`Q8_0`	Near-lossless, recommended
`Q4_K_M`	Smallest footprint, edge / CPU

Limitations

The training data is synthetic and English-only; expect lower accuracy on real-world, noisy or non-English inputs.
Record-level accuracy (~39% on test) is much lower than per-question accuracy: when a downstream action depends on several answers being right at once, gate it on the per-question probabilities.
Gold labels reflect annotator disagreement; low-confidence predictions often correspond to genuinely ambiguous cases rather than model errors.