SATP Policy (mnetlab731)
Checkpoint provenance
- Source training run:
grateful-forest-95(wandb idxhkeasdw) - Step: 600 (epoch 3 / 40 ≈ 8% of total training)
- best_val_acc: 34.84% on miniF2F-val (85 / 244) — first breakthrough past the 34.43% plateau
- Code commit:
d2f7800onfeat/action-level-dense-supervision - Repo: ChristianZ97/SATP-Training
Why 34.84% is significant
This checkpoint is the first val_acc breakthrough after a 5-validation-point trajectory:
| step | val_acc | note |
|---|---|---|
| 100 | 34.43% | first eval, hit plateau immediately |
| 202 | 34.43% | plateau |
| 303 | 34.43% | plateau |
| 405 | 34.43% | plateau |
| 506 | 33.61% | -0.82pp dip |
| 600 | 34.84% ⭐ | +1.23pp recovery + breakthrough over plateau |
The val_acc trajectory aligns with predicted dynamics: from-scratch model + HF buffer seed hits buggy-LoRA-era ceiling (~34%) in 100 steps, then BC plateau at 0.91 holds for ~410 steps while mis_calibration × lambda constraint matures, then RL handoff materializes around step 600.
Lineage (full session 2026-05-12)
| Layer | Detail |
|---|---|
| LoRA topology | 48 attention-projection modules wrapped (q, k, v, o exact-match), 1.43M trainable. Fixed substring-fallback over-wrap from prior 84-module buggy version in 8e18ad1. |
| Base freeze | shared.weight + 25 LayerNorms + relative_attention_bias blanket-frozen before LoRA wrap; enable_input_require_grads() registered for grad-checkpoint compat. |
| Optimizer | Muon for 2D heads (LR 2.0e-3), AdamW for 1D heads (LR 5.0e-5), AdamW for LoRA (LR 1.0e-4). Linear WARMUP_STEPS=500 ramp completed. |
| Sequence length | MAX_SEQUENCE_LENGTH=1024 bytes (aligned with ReProver upstream byte cap). |
| Eviction priority | D5 priority refresh on eviction disabled (a52a216) — Python 3.13 + transformers tokenizer flatten() SEGV under high eviction frequency. Reverted to pre-D5 tier + stale priority + random tiebreak. |
| Tokenizer SEGV | transformers.tokenization_utils_base.flatten monkey-patched to non-recursive iterative form (01d6d0b). |
| Faulthandler | faulthandler.enable() at train.py top (cbc8b46) — captures Python traceback on native-ext SEGV. |
| Loss system | BC + DPO + neg_BC + RL + λ-constraint, all at action-level grain (Σ_h log P over 40 heads). β_BC=1.0 / β_DPO=2.0 / β_neg_BC=1.0. |
| OOM defense | run.sh pre-flight RAM gate (15 GiB threshold) + sudo choom oom_score_adj=-500 self-protect after launch (d2f7800). |
Training dynamics at this checkpoint
- bc_loss: 0.91 EMA, Phase 2-5 (412 step span) plateau confirmed at 0.91 band
- lambda: 0.10 (first crossing of 0.10 threshold, mis_cal × λ engagement)
- mis_calibration: 0.16 EMA, sustained ≥ γ=0.15 over 250+ steps
- entropy: 0.80 (just entered target band 0.3-0.8)
- grad_norm: EMA 35.3, monotonic descent, ~3.5× clip=10 (BC-saturated regime, expected)
- buffer: pos 13414 / neg 30000 (capped) / pair 7462, 37/40 head DPO supply,
L_DPO.sample()100% yield - discovery_rate: 0 (perturb still exploring within anchor pool, no new problems discovered)
Inference
The python/src/leansatp_runtime/ tree mirrors the inference runtime from ChristianZ97/satp-policy-goal-dnyang172. Drop best_checkpoint.pt into the path expected by leansatp_runtime.service and load.
Caveats
- This is single-point breakthrough at step 600. The 34.84% needs to sustain across step 712 + step 814 vals to confirm regime change (not single-flash). If next vals fall back to 34.43% or below, this checkpoint is still the best snapshot to date.
neg_BCloss showed monotonic rise from 0.32 → 0.51 over 487 steps — supply-driven (neg buffer growing to 30000 cap) more than quality regression. Watch threshold: > 0.6 sustained would indicate dynamics imbalance.- The session-history is non-trivial: prior to step 562 there were 5+ SIGKILL / SIGSEGV / SIGTERM crashes from various causes (Python 3.13 + transformers tokenizer SEGV, mnetlab Elasticsearch RAM exhaustion, shared-machine sudo kill). Each was diagnosed and patched in the lineage above. The 34.84% checkpoint state is clean.