Nagi Big v3 — public stable checkpoint
This checkpoint improves over the G-clean v2 candidate on independent v3 suites. It does not beat JEV 1.13.0. This is the selected public Big release. V4 did not improve reliably over v3; v3 provides a better balance of measured quality and inference latency.
| Fresh final suite | G-clean | v3 | JEV 1.13.0 |
|---|---|---|---|
| Public, 2,204 examples / 4 tasks | 69.12% | 75.80% | 91.40% |
| Executable policy, 1,440 examples / 6 families | 47.22% | 68.19% | 84.51% |
| Typed subset, 960 examples / 4 families | 40.94% | 60.73% | 79.48% |
Metrics are family-macro accuracy with failures counted as wrong. Nagi coverage is 100%; JEV had four invalid probability responses on policy (99.72% coverage). The v3-minus-G-clean paired within-family 95% intervals are +3.87 to +9.29 points on public and +18.40 to +23.68 points on policy. Family-bootstrap intervals and complete predictions/receipts are documented in the research campaign.
Improvements are not universal: the conditional XOR/count family drops from 15.83% to 11.67%. WSC coreference remains weak at 47.12% vs JEV 87.50%. A small residual-head probe was rejected; this checkpoint uses the standard letter readout.
Model and training
Pinned base: Qwen/Qwen3.5-4B revision
851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.
Portable PEFT LoRA, initialized from G-clean, extended to linear-attention QKV/Z/out
and MLP projections. Existing full-attention adapters retain r64/alpha128; new
modules use r16/alpha32. Linear-attention a/b gates stay frozen. No changed base
checkpoint or custom model class is required.
Training: 5,999 public labels (AG News, BoolQ, Emotion, MNLI, SST-5) plus 6,000 executable-oracle examples, balanced within ten rule families. Effective batch16, length-aware microbatches, 750 optimizer steps, learning rate2e-5, 20 warmup steps, cosine decay. Score CE weight3, other types1, normalized by dataset-mean weight. This is a bundled experiment; individual component gains were not isolated.
Use the raw Nagi SDK renderer (no chat template), maximum26 options, temperature
0.8493753016322345, fitted on 537 separate public calibration examples. Score
probabilities under new policies are not guaranteed calibrated. Actual public SDK
vs research probabilities matched within 1e-7 on thirteen nonfinal probes covering
all three question types, at raw and calibrated temperatures. Save/reload matched
exactly; BF16 merging caused small numerical changes that passed the export gate.
Limitations
The final sets were untouched by model selection, and are now consumed for future
research. Public pretraining contamination is not ruled out. Only four public tasks
and six program families were evaluated: do not generalize a macro average to all
user workflows. No claim of production-wide superiority, universal reasoning, or
matched-hardware latency superiority is made. See evaluation.json and
selection.json for comparisons, uncertainty, frozen selection and provenance.
Comparison with the previous public Big v0
Same V4 suite, macro accuracy (historical evidence, not a new release-time test):
| Suite | Big v0 | Big v3 | V4 candidate |
|---|---|---|---|
| Policy, 1200 examples / 6 families | 40.08% | 78.00% | 78.33% |
| Language, 800 examples / 4 tasks | 54.63% | 68.00% | 66.63% |
| Historical typed, 640 examples | 47.81% | 82.81% | 82.66% |
These are separate suites from the JEV table above. Do not compare across tables.
Use
Install the SDK from https://github.com/nagisanzenin/nagi (distribution nagi-decisions).
from nagi import load_big; model = load_big(device="cuda") uses the pinned
public v3 adapter, pinned base, and calibrated temperature. No HF login is required.
Outputs are typed decisions and probabilities, not generated explanations.
Adapter SHA256: cbade091c3aa84b7fd9d814655f61ce64528bbdc4465851ef23ffc2bf77f585d.
Weights inherit the base model license; see Qwen/Qwen3.5-4B.
- Downloads last month
- -