Sciev — Open System-One Scientific Decision Models

Typed decision heads on a frozen LLaDA-8B-Instruct masked-diffusion backbone. Unstructured state + typed questions (choice / noul / score) → structured decisions with probabilities in one forward pass per decision. No text generation, no output parsing.

• Code: https://github.com/alrobles/sciev (pip install sciev)
• Site: https://sciev.org
• Manuscript: https://sciev.org/static/sciev-paper.pdf?v=030 (in preparation for arXiv submission)
• Release: https://github.com/alrobles/sciev/releases/tag/v0.3.0

Contents

file	role	sha256 (prefix)
release/fr_choice.pt	choice head — AttnPoolHead, 67.2M	67efbc59
release/alp25_noul.pt	anti-leakage noul head — MLP, 16.8M	335e9730
release/alp25_score.pt	anti-leakage score head — MLP + ordinal aux loss, 16.8M	e1441386
release/fr_noul.pt	v0.2 noul head, superseded (provenance)	317bfa5d
release/fr_score.pt	v0.2 score head, superseded (provenance)	28dea8d4
experimental/da_*.pt	experimental adapted-arm heads (see caveats)	see evidence.json
evidence.json	compact ledger of the 60-report matched study	8d86aadd

The alp25_* heads are the v0.2 recipe retrained on a ρ=0.25 evidence-corruption mixture: corrupted evidence maps to the rejection class instead of option-side-cue agreement. fr_choice.pt is byte-identical to the v0.2.x assets; fr_{noul,score}.pt are kept for provenance. The backbone (GSAI-ML/LLaDA-8B-Instruct) is downloaded separately and keeps its own license.

Usage

from huggingface_hub import hf_hub_download
ckpt = hf_hub_download("alrobles/sciev", "release/fr_choice.pt")

python -m sciev.eval \
    --ckpt "$ckpt" --decision-type choice \
    --r2-mode spanpool --r2-layers=-1,-9,-17,-25 --canonical-order \
    --r2-temp-fit-decisions decisions_dev.jsonl \
    --decisions-eval decisions_eval.jsonl \
    --device cuda --out eval.json

Key results

v0.3.0 anti-leakage heads vs the v0.2 fr heads on the same files (seed 7331). Control rows are agreement with the original reference under corrupted evidence — values at the rejection-class prior mean full rejection, not failure:

eval	type	v0.3 alp25	v0.2 fr
sci battery elite (clean), n=1200/1218	noul / score	0.907 / 0.911	0.931 / 0.896
empty-evidence agreement	noul / score	0.667 / 0.336 (prior)	0.810 / 0.575
shuffled-evidence agreement	noul / score	0.667 / 0.336 (prior)	0.669 / 0.385
SciFact dev (canonical encoding), n=332	noul	0.702	0.464
SciFact dev, n=304	score	0.757	0.757

choice is unchanged (fr_choice); its option-side leakage is a documented evaluation finding (a counterfactual-trained alp25x arm exists experimentally but is not in this release).

Matched v0.2 study (systemone-v2, 3 head seeds):

eval	fr (v0.2)	da (experimental)
sci battery choice, n=448	0.970 ± 0.017	0.961 ± 0.011
sci battery noul, n=1200	0.925 ± 0.006	0.913 ± 0.007
sci battery score, n=1218	0.859 ± 0.075	0.779 ± 0.109
GPQA main, n=441	0.295 ± 0.014	0.288 ± 0.018
SciFact noul, n=332	0.483 ± 0.023	0.709 ± 0.020

Scope and caveats

• Internal battery uses constructed distractors and synthetic score labels; high accuracy is not a measure of general scientific correctness. With the passage removed the choice head still agrees with the reference on 87.5% of items — the v0.3 alp25 heads removed that shortcut for noul/score (controls at the rejection-class prior); see evidence.json and the paper.
• The da_* heads pair the same recipe with one archival LoRA adapter whose loss is not the native per-token LLaDA estimator; results are task-dependent observations, not a causal DAPT estimate.
• External evaluations informed candidate selection — they are exploratory estimates, not an untouched final test.
• Calibration temperatures must be fitted on held-out dev data for your distribution; confidence is not a correctness guarantee.

License

Apache-2.0 for code and head weights. Backbone and source datasets keep their own licenses.