Kev v1 — small CPU decision model (FP32, 35 M parameters)

Kev answers typed decision questions about a text state: pick an option (choice), rate on a scale (score) or answer yes/no (noul). It returns one probability per option, runs on CPU only in about 31 ms, and its answer does not depend on the order of the options. It is a research model: it failed 5 of the 9 gates of its own protocol and is not a validated general-purpose decision agent.

Français · Benchmarks · Rights and restrictions · Research code

Maintained by VynoDePal. Release: 15 October 2026. Kev is an independent student model distilled from Laya; it is not an official Laya release and implies no affiliation.

At a glance

	Kev v1 (models/kev-v1, seed 42)
Encoder	MiniLM-L12-H384 (from microsoft/MiniLM-L12-H384-uncased) + decision layer + option scorer
Parameters	35,136,769
File	model.fp32.onnx, 140,766,561 bytes, opset 18, FP32, no quantization
Latency	31.3 ms median (128 tokens, 2 threads, Intel i7-1260P, onnxruntime 1.31.0)
Order instability (choice)	0 (set encoding: shared option positions, block attention)
Calibration error (ECE)	0.053 with per-question-type temperatures (--calibrate)
Question types	choice (2–20 options), score (2–20 levels), noul (yes/no)
Language	English

What it is good and bad at

• Good: topic and intent classification close to its training data (AG News 92.8% on the DEV2 control set), routing, labeling, choosing among a few described options. Probabilities are reasonably calibrated, so a confidence threshold can hand hard cases to a bigger model.
• Weak: textual entailment (SNLI 48.0% on DEV2, 3 classes) and genuinely new task families. On the S1MB benchmark (106 subsets, official Task Avg metric, chance = 0) Kev v1 scores 0.098, against 0.132 for its teacher Laya and 0.024 for the previous Laya Mini v2 student. On task families held out from training it scores 0.044 (teacher 0.114).
• Not a planner: it does not reason over several moves. It can rank moves described in text (in Snake, with planner verdicts as options, it matches the planner exactly) but needs a planner or rule-based guard to play reliably; from raw coordinates it dies at the first corner (see Snake game harness).

Quick start

hf download VynoDePal/kev --include 'models/kev-v1/*' runtime_kev.py schema.py set_schema.py requirements-runtime.txt 'examples/*' --local-dir kev
cd kev
python3 -m venv .venv && source .venv/bin/activate
python -m pip install -r requirements-runtime.txt
python runtime_kev.py models/kev-v1 examples/example-news-request.json --threads 2
python runtime_kev.py models/kev-v1 examples/example-news-request.json --calibrate

from runtime_kev import KevSession
kev = KevSession("models/kev-v1", threads=2, calibrate=True)
out = kev.predict(
    "The tennis player won the tournament after defeating the defending champion.",
    {"topic": {"type": "choice",
               "instructions": "Choose the topic that best describes this news item.",
               "criteria": {"world": "World", "sports": "Sports",
                            "business": "Business", "science": "Sci/Tech"}}})
print(out["answers"]["topic"]["choice"])  # sports

The question and option descriptions must fit in 256 tokens; the state text is truncated to the remaining room of the 512-token window (truncated_state reports it). Inputs must be English.

Validation (protocol frozen before training, sha ba7c4fa7…)

Three seeds (42, 43, 44) were trained on an A100 80 GB. A gate passes if the 3-seed mean passes and at least 2 seeds pass. Development split DEV-G = half of S1MB, split by group; the other half (FINAL-G) stays sealed because DEV-G failed.

Gate	Threshold	Kev v1 (mean)	Seeds	Result
G1 size	≤ 50 M params, ≤ 180 MB	35.1 M, 140.8 MB	3/3	pass
G2 latency	≤ 45 ms	31.7 ms	3/3	pass
G3 order instability	≤ 0.05	0	3/3	pass
G4 S1MB Task Avg vs previous run	CI95 low ≥ +0.03	+0.016 [+0.008, +0.024]	0/3	fail
G5a held-out families vs chance	CI95 low ≥ +0.05	+0.044 [+0.028, +0.065]	0/3	fail
G5b held-out families vs previous run	CI95 low ≥ 0	+0.017 [+0.003, +0.030]	1/3	fail
G6 ECE	≤ 0.10	0.053	3/3	pass
G7 AG News + SNLI vs Laya Mini v2	CI95 low ≥ −0.02	−0.009 [−0.032, +0.016]	1/3	fail
G8 targeted families vs previous run	CI95 low ≥ +0.05	+0.056 [+0.032, +0.072]	0/3	fail

Kev v1 is published as the best general checkpoint obtained so far, not as a model that met its targets. Details, per-family scores and hashes: BENCHMARKS.md and benchmarks/kev-v1/.

Snake game harness (zero-shot, CPU)

Kev v1 (models/kev-v1, seed 42) was played without game-specific fine-tuning in a Snake harness ported from virajbhartiya/laya-vs-jev (Apache-2.0, logic unchanged): 24×16 board, 10 test seeds (101–110), at most 600 steps, one choice question per move over UP/DOWN/LEFT/RIGHT, option order reshuffled every step. The protocol was frozen before any model played; it was written for Laya Mini v2 and applied unchanged to V3 and Kev v1. Run on the i7-1260P CPU, 2 threads, uncalibrated probabilities: 180 episodes, 47,176 model calls. Kev's training corpus contains Snake decisions from seeds 1000–1999 (disjoint from the test seeds) with the same board, rules and prompts, so this measures in-distribution play, not transfer to a new game.

• L2 labeled: each option carries the planner verdict (Blocked / Unsafe / Safe, Best). Tests label reading.
• L1 facts: each option states facts without a verdict (hits the wall or the body, free cell, eats the food, new distance to the food).
• L0 raw coordinates: the state lists head, food and body coordinates; options only name the move.
• Shielded: the laya-vs-jev safety shield replaces an unsafe argmax with the most probable safe move. Unshielded: the argmax is executed and an illegal move ends the game.

Prompt	Mode	Kev v1 score (deaths)	Kev v1 agreement with planner	V3 score (deaths)	V3 agreement
L2	shielded and unshielded	23.3 (0)	1.000	23.3 (0)	1.000
L1	unshielded	17.5 (4, own body)	0.828	17.7 (1)	0.757
L1	shielded	22.4 (interventions 2.2%)	0.922	22.1 (1.9%)	0.823
L0	unshielded	0.5 (10, all into a wall)	0.778	13.4 (6)	0.868
L0	shielded	19.9 (interventions 2.9%)	0.744	22.2 (0.9%)	0.904

Mean score over the 10 seeds. Unshielded references: cycle planner 23.3 (0 deaths), greedy bot 26.9 (8 deaths), random legal move 0.6. V3 is the previous general run (not published), measured with the same harness.

Harness gates (same outcome for Kev v1 and V3): order invariance passes (max |Δp| 2.4e-6, identical games with shuffled and canonical order), label reading passes (L2 agreement 1.00), L2 shield interventions pass (0), facts beat chance at L1 (+16.9 points over a random legal move, CI95 [14.7, 18.8]). Latency fails: p50 29.4 ms but p95 60.2 ms against a 60 ms limit (p99 74.0 ms; V3 p95 60.6 ms).

Why Kev v1 fails with raw coordinates (L0)

• One state ends every game. All 10 unshielded L0 deaths happen in the top-left corner: head at (0, 0), moving up, and the only good move is RIGHT. Kev gives 0.62–0.66 to UP (into the wall) and 0.34–0.37 to RIGHT. In the same 10 states V3 gives 0.80–0.99 to RIGHT. The state text is never truncated.
• Kev follows the planner's cycle well enough (78% agreement) to reach that corner after about 42 moves. The cycle passes there on every lap, so every game ends the same way.
• On 1,000 states sampled from planner games, Kev agrees with the planner 93.4% of the time (V3 96.9%) but proposes 8 illegal moves (V3 0), goes into a wall straight ahead in 7 of 26 cases (V3 0) and keeps going straight in 64 of the 109 states that need a turn (V3 30).
• It is not a general blindness to walls: with a straight snake facing a wall in the middle of an edge (216 states), neither model goes into the wall.
• Probable cause: a rare case plus dilution. In the synthetic L0 training rows (200 of the 1,000 seeds re-sampled with the generator's own sampling), 88.8% of labels are "keep straight" and only 78 of 2,000 rows have a wall straight ahead, 5 of them with the wall above. The 19,247 Snake rows are the same in V3 and Kev v1, but they make up about 4.6% of Kev's 422,556 labelled rows against about 9% of V3's 207,368. Kev learned a stronger "keep straight" habit and learned this corner turn less well. This is an interpretation of the probes, not a causal test; oversampling wall and corner states is the obvious fix and has not been tried.

In practice: describe options with facts (L1) or verdicts (L2) rather than raw coordinates, keep a rule-based safety shield, and budget about 60 ms at p95 per decision in a game loop. Raw results: benchmarks/kev-v1/snake/; code: research/src/kev-v1/game/.

Training in short

• About 464,000 decisions from 36 public datasets (classification, NLI, multiple-choice QA, intent detection, relevance, sentiment, similarity), LocalLLaMA/typed-decisions and about 59,000 synthetic game decisions (Snake, tic-tac-toe, grid paths) with exact labels. Rows whose state appears in S1MB were removed.
• Loss: weighted KL to the teacher (Laya base, pinned) + cross-entropy on gold labels + symmetric KL between two permuted and reworded views. Distillation weight per source depends on measured teacher quality.
• 3 epochs, batch 32, encoder lr 5e-5, head lr 1e-4, BF16; best epoch chosen on a separate selection set (epoch 3 for every seed).

Repository layout

Path	Content
models/kev-v1/	ONNX FP32 model (seed 42), tokenizer, manifest with calibration temperatures
runtime_kev.py, schema.py, set_schema.py	CPU runtime (onnxruntime only)
checkpoints/kev-v1-s42, -s43, -s44	PyTorch safetensors and training logs for the three seeds
benchmarks/kev-v1/	frozen protocol, DEV-G gate report, latency, evaluation reports, corpus manifest
research/src/kev-v1/	training, corpus, evaluation and analysis code
benchmarks/kev-v1/snake/	Snake harness protocol, Kev v1 and V3 results, L0 diagnosis
research/src/kev-v1/game/	Snake game (Apache-2.0 port), harness runners, analysis and L0 probe scripts

Earlier models (Laya Mini)

This repository previously hosted the Laya Mini experiments (v1 INT8 A/B variants and the v2 order-invariant FP32 model, BERT 6×512). They are removed from main but kept intact under the tag laya-mini-archive (revision 3cff02c7). Kev v1 corresponds to the internal run "Laya Mini v4"; the research code and reports keep that internal name (V4-s42 = Kev v1 seed 42).

Limitations

• Failed 5 of 9 pre-registered gates; FINAL-G was never evaluated, so there is no clean held-out confirmation.
• Weak on new task families and on entailment; English only; uncalibrated by default.
• Probabilities are only as reliable as the question is close to the training distribution.
• Latency measured on one CPU (i7-1260P); other hardware not validated.
• In the Snake harness, raw-coordinate prompts (L0) fail systematically at one corner, and in-game latency p95 is 60.2 ms (above the 60 ms game gate).
• Trained partly on non-commercial and rights-unknown datasets: see LICENSE_NOTES.md. Not for commercial or safety-critical use.