Crivo-32M

A 32M-parameter typed-decision encoder: on the Pareto frontier of the Decision Index.

Crivo-32M reads a state and a set of typed questions (choice, noul, score) and returns a probability for every option in a single encoder pass. It does not generate text. It is the small sibling of Crivo-150M, distilled from it: light enough for CPUs and edge devices, fast enough to sit inside every step of an agent loop.

Highlights

• On the Pareto frontier. Maintainer-run Full score of 11.87 on the Decision Index 0.3: no smaller model scores higher, and 21 larger submissions score lower, from Bekko System One 68M (2× its size) up to an 8B model.
• Fast. Median 7.3 ms per request on the board's RTX PRO 6000 (measured by the maintainers), 10.3 ms on an A100.
• One pass, every option scored. choice, noul and score questions share one encoder; all questions of a request are batched, and each returns a full probability distribution.
• Reproducible and clean. The complete public-suite run is published; training used open data only, no outputs of Jev or any other proprietary decision API, and the exact contamination check found 0 matches with the 0.3 suite.

Architecture	Ettin / ModernBERT encoder, 10 layers, hidden 384, [OPT] readout + MLP scorer
Parameters	32M
Base model	cross-encoder/ettin-reranker-32m-v1
Teacher	tardellirs/crivo-150m (knowledge distillation)
Decision Index 0.3	Full 11.87 (maintainer-run) · public index 25.06 (self-run, 140,178 / 140,178 requests answered)
Latency, 1× RTX PRO 6000 (board, measured by the maintainers)	median 7.3 ms · mean 8.5 ms · p95 11.4 ms per request
Latency, 1× A100 40 GB, bf16 (our run, one request at a time)	median 10.3 ms · mean 12.6 ms · p95 16.7 ms
Context	trained up to 8,192 tokens; reads longer inputs whole, up to 32,768, without truncation
Language	English
License	CC BY-NC 4.0 (see License)

Crivo follows the typed-decision interface popularized by TypeSafe's Jev (a state plus typed questions in, a probability for every option out) and runs unmodified in the open Decision Index kit. It is an independent open model, not affiliated with or endorsed by TypeSafe. Crivo is Portuguese for "sieve"; passar pelo crivo, to pass through the sieve, means to be judged and sorted.

Resumo em português

Crivo-32M é a versão pequena do Crivo-150M: um encoder de decisões tipadas com 32M de parâmetros (Ettin/ModernBERT), destilado do Crivo-150M. Ele lê um state e perguntas choice, noul ou score e devolve uma probabilidade para cada opção numa única passada, sem gerar texto. No Decision Index 0.3 tem Full 11,87, avaliado pelos mantenedores com testes privados: está na fronteira de Pareto (nenhum modelo menor pontua mais) e fica à frente de 21 modelos maiores, do Bekko 68M a um modelo de 8B. O índice público é 25,06, com o run completo publicado, e a mediana medida pelo placar é de 7,3 ms por requisição. Funciona em inglês e vai bem em habilidades conhecidas, mas quase não generaliza para domínios novos (0,92). Licença CC BY-NC 4.0 (uso não comercial).

Contents

• Results
• Quick start
• Model details
• Training
• Contamination firewall
• Intended use and limitations
• License
• See also
• Acknowledgements
• Citation

Results

Decision Index 0.3 — Full score

The board ranks models by the Full score: 20% the public index, 50% private tests of the same skills and 30% private tasks from new domains. The private parts are run by the maintainers. Snapshot of 2026-10-13, 159 submissions:

Full score	Public index	Private, same skills	Private, new domains	Overall rank
11.87	25.06	18.84	0.92	137 / 159

Comparison with small models (Full score; served parameters as reported by the board):

Model	Parameters	Full
Bekko System One v0 17M (previous point on the frontier)	17M	2.93
Crivo-32M	32M	11.87
Bekko System One v0 68M	68M	8.42
Julia-1	141M	2.56
DM-JEPA 1.1	149M	4.05
Dinah-0	150M	13.87
tasksource Decider nano v4 (next point on the frontier)	149M	17.63
Crivo-150M (its teacher)	150M	22.21

Crivo-32M keeps most of its teacher's skill on private tests of familiar skills (18.84, against 27.74 for Crivo-150M) but is near chance on tasks from new domains (0.92). Use it for skills it was trained on, and validate on your own task before deployment. The chart above is generated from the board's data/index.json by assets/plot_pareto.py.

Decision Index 0.3 — public index (self-run, complete)

Run with the official kit (apolinario/decision-index, commit 9eb2dbe, edition 0.3) on the complete public suite: every one of the 140,178 scoreable requests answered, none unsupported, nothing truncated. Scores are chance-corrected skill (0 = random guessing, 100 = perfect), weighted as the board does. Full results: tardellirs/decision-index-results → runs/crivo-32m.

Public index	Knowledge & Reasoning	Language	Retrieval & Classification	Tools & Automation	Arts & Human Taste
25.06	2.0	25.6	46.7	36.7	18.5

Per-benchmark skill (0–100, chance-corrected) — click to expand

★ = gold benchmark (weight 1.2 within its area). The index uses the native metric listed, chance-corrected.

Benchmark	Native metric	Crivo-32M
Knowledge & Reasoning
GPQA Diamond ★	accuracy	0.0
GSM8K	accuracy	0.3
ChessBench	accuracy	0.0
MuSR	accuracy	0.0
SATA-Bench	case exact accuracy	0.0
CRUXEval	accuracy	8.4
CLadder	accuracy	8.4
HLE ★	accuracy	0.0
MMLU-Pro ★	accuracy	1.1
BBH fixed-option tasks ★	accuracy	1.4
WinoGrande ★	accuracy	3.4
Language Understanding
ContractNLI	macro-F1	59.0
ANLI ★	macro-F1	8.4
HellaSwag ★	accuracy	11.0
ACOS	per-review F1	25.4
FinEntity	macro-F1	52.2
iSarcasmEval	see separate subtask tracks	0.0
VAST	macro-F1	51.0
NLI4CT	macro-F1	0.0
RAGTruth response-level hallucination	F1 on hallucinated class	30.0
Retrieval & Classification
BANKING77 ★	macro-F1	86.7
CLINC150+OOS ★	macro-F1	86.2
BRIGHT ★	nDCG@10	17.5
Amazon ESCI	macro-F1	21.3
PhishNChips phishing decisions	accuracy	0.0
HoVer claim verification	accuracy	58.5
Tools & Automation
BFCL ★	case exact accuracy	54.2
ToolRet	nDCG@10	49.4
API-Bank ★	accuracy	7.9
Home appliance simulator	case exact accuracy	27.3
When2Call MCQ	accuracy	47.2
Arts & Human Taste
BPoMP	accuracy	67.3
Humicroedit	accuracy	0.0
POP909-CL	accuracy	0.0
cfcolor	accuracy	0.0
Habermas Machine	accuracy	11.6
New Yorker caption matching	accuracy	32.3

• Strong: intent classification (BANKING77 86.7, CLINC150 86.2), the original-vs-perturbed poem task (BPoMP 67.3), tool selection and retrieval (BFCL 54.2, ToolRet 49.4), long-document NLI (ContractNLI 59.0, HoVer 58.5), stance and entity sentiment (VAST, FinEntity).
• Weak: anything that needs world knowledge or multi-step reasoning (GPQA, MMLU-Pro, GSM8K, MuSR, CLadder, WinoGrande), since a 12.5M-parameter transformer core cannot hold it, and long tool catalogs (API-Bank).

Held-out decision tasks (our own evaluation)

Two external sets that never entered training: fastino/fast-decisions (17 product domains, dev split) and the English test split of mertkayacs/jevalt-bench; choice and noul questions, accuracy and multi-class Brier (lower is better).

Set	Questions	Accuracy	Brier ↓
All	5,989	62.9	0.458
fast-decisions (17 domains)	2,600	43.2	0.680
jevalt-bench (en)	3,389	78.0	0.287

These are small proxies for the private "new domains" part of the board, not substitutes for it.

Quick start

Standalone (no kit)

from huggingface_hub import snapshot_download
import sys; path = snapshot_download("tardellirs/crivo-32m"); sys.path.insert(0, path)
from crivo import Crivo

model = Crivo(path)                                   # CUDA if available, else CPU
out = model.answer(
    state={"ticket": "My card was charged twice for the same order."},
    questions={
        "intent": {"type": "choice", "instructions": "What does the customer want?",
                   "criteria": {"refund": "Money back for the duplicate charge",
                                "cancel": "Cancel the order",
                                "info": "Just asking for information"}},
        "urgent": {"type": "noul", "instructions": "The customer needs a fast response."},
    })
print(out["intent"]["choice"], out["intent"]["probabilities"])   # refund {'refund': 0.83, 'cancel': 0.01, 'info': 0.17}
print(out["urgent"]["noul"])                                     # 0.43

type	criteria	answer
choice	{key: description or null}, 2+ options (no upper limit)	choice + probabilities over every key
noul	optional {"true": ..., "false": ...} describing both poles	noul = P(statement in instructions is true)
score	ordered list of level descriptions, lowest first	score = expected level + probabilities

Decision Index kit

pip install "decision-index[transformers] @ git+https://github.com/apolinario/decision-index@9eb2dbe"
hf download tardellirs/crivo-32m --local-dir crivo-32m && cd crivo-32m
python -m decision_index pipeline --engine decision_index_engine:DecisionEngine --edition 0.3 --out runs/crivo-32m

The engine defaults are the settings of the reported run: one option order (tta=1), fp32 scoring head, SDPA attention, batch_size=32, token_budget=65536, max_length=32768, bf16 autocast on CUDA. The state is tokenized once per request and all questions are batched. It raises Unsupported only for a question above max_length tokens — none in the 0.3 suite.

Model details

Same design as Crivo-150M. Each question is one sequence

[CLS] instructions [SEP] state [SEP] [OPT] option_1 [OPT] option_2 … [OPT] option_n

(option_i = key: description). The hidden state at each [OPT] marker plus a question-type embedding goes through a small MLP scorer (run in fp32) to one logit per option; a softmax over the question's options gives the distribution. Option order was randomized during training. Training sequences were up to 8,192 tokens; longer inputs are read whole with rotary positions up to 32,768.

Files: model.safetensors (fp32 weights), config.json (ModernBERT config + decision block), tokenizer files, model.py, crivo.py (standalone API), decision_index_engine.py (kit engine).

Training

Full fine-tune of cross-encoder/ettin-reranker-32m-v1 in three stages with knowledge distillation from Crivo-150M, then a weight average.

Loss. Every example is one question with a target distribution over its options. With a teacher present: L = 0.5·[CE(gold) + 0.5·Brier(gold)] + 0.5·τ²·KL(teacher_τ ‖ student_τ), τ = 2 for 3+ options and τ = 1 for binary questions. Teacher logits were precomputed once on exactly the tokens the student sees (same rendering and truncation) and permuted with the shuffled option order. Sources where the teacher is weak — the original-vs-perturbed text set and the smart-home cases — use the gold loss only.

Data

Block	Questions	Sources
Broad typed decisions	~696,000	Bekko System One dataset v0, 153 subsets capped at 6,000 decisions (ranking subsets re-rendered as per-candidate relevance questions); score questions halved
Train splits of the public benchmarks' sources	~209,000	train/dev splits (never test) of ANLI, HoVer, When2Call, RAGTruth, VAST, iSarcasmEval, ACOS, NLI4CT, Amazon ESCI, ContractNLI, BANKING77, CLINC150, HellaSwag, WinoGrande, New Yorker (safe fold), API-Bank, rendered by the kit's own builders
Procedural decisions	~47,000	tasksource/procedural-typed-decisions
Format invariance	94,282	augmentations of the above: opaque option keys, key-only options, distractor options, "none of the above" with the gold option present or removed
New task families	78,300	328 classification/NLI tasks from tasksource/tasksource-jev-typed-decisions that are neither Bekko subsets nor Decision Index sources, ≤297 each
Original vs perturbed text	20,000	public-domain poems and prose (biglam/gutenberg-poetry-corpus, Gutenberg prose) with code-generated perturbations (word swaps/deletions, broken rhyme or meter, line order)
Tool selection, causal reasoning, financial sentiment, smart-home control (stage 2)	~70,000	ToolACE + glaive function calling (tool selection), CLadder variants not used by the suite (pairs too close to suite items removed), SEntFiN entity sentiment, synthetic smart-home cases
Long context (stage 3)	56,690	API-Bank with catalogs fitted to the suite's length, ContractNLI / RAGTruth / NLI4CT, replay

No data labeled by Jev or any other proprietary decision API was used.

Recipe

Stage	Init	Data	Steps	LR (encoder / head)	Max length	KD
1	Ettin reranker 32M	blocks 1–6 (1,143,435 questions), 1 epoch; weight EMA 0.9995	17,578	1e-4 / 5e-4	2,048	yes
2 (×2 seeds)	stage-1 EMA	tool selection, CLadder, home control, financial sentiment + replay (316,713)	5,064	5e-5 / 2.5e-4	2,048	yes
3	average of the two stage-2 seeds	long context (56,690)	4,690	4e-5 / 1.5e-4	8,192	no
Final	—	0.5·(stage-2 average) + 0.5·(stage 3), then 10% interpolation of the encoder toward the base model (WiSE-FT)

Common: AdamW (β 0.9/0.98, weight decay 0.01), linear warmup and decay, bf16, option order shuffled for every choice example. Every checkpoint choice (EMA vs final, which average, WiSE) was made on the held-out panels above, never on the board. Test-time averaging over several option orders was evaluated and gave no gain, so the model reads each question once.

Compute. One NVIDIA A100 40 GB (Lambda Cloud): teacher logits ~70 min, training ~80 min, panels and the complete public suite ~70 min — about US$7 in total.

Contamination firewall

Same three levels as Crivo-150M: (1) a leaf-level filter of every new example against the rebuilt pools of all 42 benchmarks; (2) builder-level checks of the public-source train splits; (3) an exact check of every training question against all 563,686 questions of the 0.3 suite — 0 matches. In addition, CLadder training questions sharing ≥90% of their 8-grams with a suite question were removed (574 rows), and the original-vs-perturbed set excludes every text that shares a line with the BPoMP pool.

Intended use and limitations

Intended use: fast, local decisions over text or JSON state — routing, intent and topic classification, policy gates, tool selection, relevance checks — where a 150M model is too heavy (CPU, edge, very high request volume).

• New domains are the weak spot. On the board's private tasks from new domains it scores 0.92, against 18.84 on private tests of familiar skills: it does not transfer to unfamiliar task types. Validate on your own data first.
• English only.
• Very little world knowledge and no step-by-step reasoning: near-zero on expert QA, math, commonsense reasoning and causal inference benchmarks. Use Crivo-150M or a larger model for those.
• Distilled model: it inherits Crivo-150M's biases and errors; where the teacher was wrong, the student often is too.
• The smart-home score is optimistic: the synthetic training cases reuse the suite's request templates (new houses only).
• No temperature fitted: treat probabilities as scores to threshold on your own data.
• Not for high-stakes decisions about people without human review.

License

CC BY-NC 4.0, the same as its teacher, Crivo-150M: several training sources are non-commercial (for example ANLI, some Bekko dataset subsets). Upstream dataset licenses and terms continue to apply. The base model is Apache-2.0.

See also

• Crivo-150M, the 150M teacher (Full 22.21), stronger on new domains.
• Crivo collection.

Acknowledgements

• Ettin (JHU CLSP) and the Ettin reranker 32M.
• Bekko System One (hotchpotch) for the open training dataset.
• Dinah-0 (Lukitaduarte) for the [OPT] readout design.
• tasksource for procedural and multi-task typed-decision data.
• The Decision Index maintainers and its open kit.

Citation

@misc{crivo32m2026,
  title  = {Crivo-32M: a small typed-decision encoder distilled from Crivo-150M},
  author = {Tardelli Stekel},
  year   = {2026},
  url    = {https://huggingface.co/tardellirs/crivo-32m}
}