openbmb_decide-1B

A small decision model: openbmb/MiniCPM-V-4.7-1B fine-tuned with LoRA and a fresh Clef-style joint-schema decision head on LocalLLaMA/typed-decisions in Unsloth Studio. Given a state and a set of typed questions (choice, noul, score), it returns a calibrated probability distribution over the options of every question in one forward pass, in the System One / Decision API answer format.

This is an independent fine-tune by johannhartmann. It is not affiliated with or endorsed by OpenBMB; the name only refers to the base model.

Results

Measured by Unsloth Studio on the test split of typed-decisions (all, 400 cases, 2,000 decisions), before and after fine-tuning:

	Base (fresh head)	Fine-tuned
Accuracy (per decision, argmax vs. gold label)	0.300	0.792
Record accuracy (all five decisions of a case right)	0.01	0.36
Expected calibration error	0.0388	0.0175

Please read these numbers with the following caveats:

• The test split was also Studio's evaluation split. Eval loss was logged on it during training, and the per-type calibration temperatures were fitted on it. The ECE is therefore optimistic. Accuracy is not affected by the temperatures (they do not change the argmax), and no checkpoint was selected on it: the model is the final step.
• This is a specialist. It was trained on the benchmark's train split for these four workflows. The typed-decisions card asks for specialist and zero-shot general models to be reported separately; numbers are not comparable across the two.
• Gold is a teacher's distribution, not ground truth. Per the dataset card, gold is the mean of three samples from a ~4B-class teacher, and teacher self-agreement is 0.735. Accuracy above that level suggests the model is also fitting the teacher's tendencies.
• These are Studio's metrics, not the benchmark's official scorer; KL and Brier were not computed.

Training

Setting	Value
Base	openbmb/MiniCPM-V-4.7-1B (Qwen3.5 text backbone, hidden size 1024, 24 layers)
Head	new Clef joint-schema head (width 512, 4 layers, 2 routing layers, 8 heads)
Method	LoRA r=64, alpha=64, dropout 0, merged into the weights
Objective	soft cross-entropy against the gold distributions
Data	LocalLLaMA/typed-decisions, config all, split train (1,200 cases, 6,000 decisions)
Schedule	3 epochs, 225 steps, batch 8 x grad. accumulation 2, lr 1e-4 linear, 5 warmup steps, weight decay 0.01, AdamW
Max length	4,096 tokens
Seed	3407
Hardware	1x RTX A6000, about 6 minutes, peak memory 5.3 GB
Calibration	per question type temperatures [1.157, 0.900, 0.956] (unsloth_decision_config.json)

Only text is used. The decision head reads text tokens; the vision tower of MiniCPM-V is included in the merged weights but not used.

Usage

Requires Unsloth with decision-model support and transformers>=5.18 (for the minicpmv4_7 architecture).

from unsloth import FastDecisionModel

model, tokenizer = FastDecisionModel.from_pretrained("johannhartmann/openbmb_decide-1B")
FastDecisionModel.for_inference(model)

state = {"agent": {"autonomy": "unsupervised"},
         "task": "Migrate the user table to the new schema without downtime.",
         "trace_summary": {"steps": 42, "tool_errors": 3, "irreversible_actions": 1}}
questions = {
    "action": {"type": "choice", "instructions": "What should the observability system do with this trace?",
               "criteria": {"continue": "Let the agent proceed without interruption.",
                            "observe": "Keep running, but flag the trace for later sampling.",
                            "human_review": "Queue this trace for a human to review.",
                            "stop": "Halt the agent now."}},
    "needs_review": {"type": "noul", "instructions": "This trace requires human review."},
}
print(FastDecisionModel.predict(model, tokenizer, state, questions))

In Unsloth Studio the model can also be served through the Decision API (POST /v1/systemone).

The model was trained on English text and four workflows (agent trace observability, customer service, invoice processing, security incidents). Expect weaker results on other domains and languages. A German translation of the training data is available as johannhartmann/typed-decisions-de; this model was not trained on it.

License

Apache-2.0, matching the base model (openbmb/MiniCPM-V-4.7-1B, Apache-2.0) and the training data (LocalLLaMA/typed-decisions, Apache-2.0).