dimitarpg13/semsimula-ladder-owt-d384-l2-attention-potential-rglive

attention_potential with a live exchange-field gradient: the gradient-path probe

NOT A LADDER ARM: a Generation 2 probe. The same model and the same forward force as the arm attention_potential, bit-identical, trained with only the exchange field's loss gradient made live. Vφ and ξ keep Gen 2's detached sources, so this is a Gen 2 model with one channel changed, and the run that found the gradient starvation behind Gen 2's results. Its Gen 3 counterpart is the everything-live run (attention_potential with RELAX_GRAD_PATH, VPHI_GRAD_PATH and XI_GRAD_PATH all 'live'): 54.21 settled, 11.3% below this probe but 2.1% worse than the same model without the exchange field.

Result

model	exchange-field gradient	settled PPL	vs GPT-2
the arm attention_potential (parent)	detached: no gradient into any hidden state	80.90	1.624×
this probe	live	61.11	1.227×
the arm attention (non-conservative field)	live	63.51	1.275×
the arm no-exchange (no exchange field)	n/a	66.98	1.345×
the arm gpt2-matched	n/a	49.81	1.000×

• −24.5% against its parent, from a change that leaves every forward value identical. The parent's +27.4% gap to the non-conservative field was the training signal, not conservativity: with the gradient restored, the conservative field beats the non-conservative one (61.11 against 63.51).
• Best 59.09 at step 31,000; final 61.06.
• At step 3,000 (the probe as first run): 133.74 against the parent's 148.99 (−10.2%); a hit on both pre-registered criteria.
• Pre-registration of the full run: 63–72, point 67 (frozen 2026-09-29 at step ~1,300 of the probe, before its extension); actual 61.11, a miss on the good side.
• Run health: 0 watchdog, 0 spikes, 0.8% clip-hit (5 of 649 logged steps, max norm 1.12) against 29.2% for the arm attention; SCAF leak monitor CLEAN at all seven audits (5k to 30k, and 32.5k).

What changed, and why the forward pass did not

The parent's exchange field is a scalar potential. To keep its force a gradient in the token's own coordinates, the code held the attention weights and the source states fixed, and it did so by detaching them. The force then depends on nothing it can send a gradient to, so the loss never reached any hidden state through it.

Here the same force is written out explicitly over live tensors (relax_grad_path='live' in model_parf_multixi.py). The value is unchanged; the backward pass now reaches every token through the queries, keys and values, as it does in the non-conservative field. The offline check on the parent's checkpoint, forward identical and source gradient opened, is gradient_path_check_output.txt.

Probe readings on this checkpoint, against its parent

Same architecture and same tokens, so these compare like with like. They are not comparable to other arms' E1 values, which carry different potentials.

reading	parent	this probe
inertia: PPL rise when the velocity is reset (6b-7, gate 1)	+18.80	+32.66
refinement at fixed T (6b-7, gate 3)	maps	maps
deflection from the damped Vθ geodesic, R(geo) (6b-9)	1.274	1.144
deflection with the reverse channel removed, cons+LN (6b-9)	1.021	0.877
reverse-channel effective gate (6b-9)	0.024	0.034
ω·dt median, endpoint (6b-13)	not run	1.784 (44% past 2)

• With live gradients the model leans more on inertia: resetting the velocity costs 32.66 PPL, against 18.80 for the parent.
• The layer step sits closer to the damped Vθ geodesic, but it is still not one: R(geo) stays above 1, and 6b-12 finds every token at layer 1 strongly forced (uniform forcing, as in the parent).
• 6b-7 still reads MAPS: only the trained step size works.
• 6b-13 printed a MISS against the band [3.3, 4.2]. That band was pre-registered for the no-exchange arm and does not apply to this probe.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='attention_potential', REVERSE_CHANNEL=True, LADDER_L=2, LADDER_LR=1.2e-3, RELAX_GRAD_PATH='live'   # tag component 'rglive'

The run was a 3,000-step probe (PROBE_MAX_STEPS = 3_000), then extended in place to 32,500 steps on the same schedule. Cell 5b prints a GRADIENT-PATH PROBE banner for this setting.

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_rglive_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_attnpot/.

Files

path	what
README.md	this card
config.json	architecture, training, gradient convention and result
checkpoints/*_best.pt	weights at the best eval (step 31,000), optimizer state stripped
checkpoints/*_step500_best.pt	weights at step 500, an initialisation reference
results/L2_attnpot_live_grad_exchange_field_32500steps.txt	the full run log: the 3,000-step probe and its extension, with per-step diagnostics and the SCAF lines
results/training_log.jsonl	per-eval record; the settled PPL is the mean of its last three rows
results/scaf_leak_monitor.jsonl	the seven in-flight causal-leak audits
results/Cell_6b-7_flow_or_maps_output.txt	flow or maps (inertia and refinement gates)
results/Cell_6b-9_are_layers_damped_V_theta_geodesic_output.txt	E1: deflection of the layer step from the damped Vθ geodesic
results/Cell_6b-12_is_forcing_sparse_output.txt	F1: per-token forcing
results/Cell_6b-13_omega_dt_cross-arm_comparison_output.txt	ω·dt at the endpoint, on the cross-arm token set
results/omega_dt_endpoint.jsonl	the 6b-13 record
results/logfreq_surprisal_openwebtext.npy	the log-frequency table the mass model reads (shared by every Fock arm)
code/	the notebook and the model code, as in every Gen 2 Fock repo. It is the current revision: switches added since this run default to off, and the forward pass was verified bit-identical

Companion notes

note	what it covers
Gradient_Starvation_Investigation.md	the programme this probe opened: Tiers 0–3, and the live-gradient ladder
Depth_Ladder_and_Matched_Baseline_Protocol.md	§5.8: the gradient-check, the probe's pre-registrations and their scoring
Measuring_the_Price_of_Conservativity.md	the question the parent arm was built to answer

Caveats

• One seed, like every arm of the ladder.
• Not the strongest conservative-forward model. Vφ and ξ are still starved here; the Gen 3 everything-live run opens them too.
• The reverse channel is on, so the model as a whole is not conservative; only its exchange field is.
• The comparison with the non-conservative field is not clip-matched: 0.8% clip-hit here against 29.2%.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Conservative and Non-Conservative Memory, and Competing Mechanisms},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v25 (Nov 9, 2026).
               Companion code repository (DOI 10.5281/zenodo.24066016):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-11-08.