dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2-sr3

SR2 with its stiff modes critically damped (SR3-crit): 8.5% behind SR2 — the oscillation is the mechanism (Gen 3)

The question of this collection: the conservative core of Semantic Simulation integrates each token as a damped particle, and until now the damping was one constant, γ₀ = 0.1, in every direction. What happens when the stiffest directions of the potential, the modes that oscillate inside a layer step, are damped harder, by a fixed rule or by a rule the model learns token by token? Each arm is SR2 with one change to its damping.

This model: SR2 with every stiff mode critically damped (damping rate γ₀ + 2ωᵢ, ratio 1), the velocity between layers kept, no new parameters. Verdict: 8.5% behind SR2 (63.28 against 58.32), and behind the model with no inertia at all (FO-b, 61.16): what the second-order model gains from on OpenWebText is the oscillation of its stiff modes, not inertia as such. the position-dependent-damping collection holds the damping axis; the Gen 3 collection holds the OpenWebText ladder; the first-vs-second-order collection holds the first-order twins.

Result: the damping axis

arm	damping of a typical token's stiffest mode	settled perplexity	against SR2
SR2	light: γ₀ = 0.1, total ratio about 0.03	58.32	—
learned ζ(h)	moderate: total ratios about 0.59 (layer 0) and 0.39 (layer 1)	59.29	+1.7%
FO-b	the overdamped limit: no velocity in any direction	61.16	+4.9%
SR3-crit (this model)	critical: total ratio about 1.03	63.28	+8.5%

More damping of the stiff modes, worse settled perplexity; SR2's light damping is the best point measured. All four are the same architecture, corpus, schedule and seed.

• This model: settled 63.28 (mean of the last three evals, 63.15, 63.48, 63.23); best 61.27 at step 31,000. Validation perplexity at block 512.
• Trained on exactly 532,480,000 tokens (32,500 steps × 32 × 512), the Gen 3 ladder's budget and schedule, seed 0.
• Pre-registered (protocol §5.13): SC3′, settles 3% or more behind SR2 toward FO-b, called 25%: HIT, +8.5%, and behind FO-b. SC1′ (within +1% of SR2, 50%) and SC2′ (beats SR2 by 2%, 25%): MISS. SC4, Gate 1 at most SR2's +3.9%, called 60%: HIT (+3.6%). SC5, the per-layer flow converges within 0.15 nats, called 60%: MISS (0.30 nats, drifting).
• Run health: 0 watchdog triggers, 0 spike captures; SCAF leak monitor CLEAN at every audit; largest gradient norm under 2.

What the damping does here

SR2 integrates, exactly, the 16 stiffest directions of each token's potential as damped oscillators, every one at the base rate γ₀ = 0.1. A mode with frequency ω is then lightly damped: its damping ratio, the rate measured against the mode's own frequency, is γ₀ / 2ω, about 0.03 for the stiffest mode of a typical token, so the mode turns through more than a full cycle inside one layer step and keeps about 82% of its amplitude.

SR3-crit sets each stiff mode's rate to γ₀ + 2ωᵢ, the critical rate on top of the base. At that rate a mode settles toward its well as fast as it can without overshooting; the oscillation is gone, the velocity between layers is kept, and every direction outside the stiff modes keeps γ₀. The total ratio is about 1.03 on the stiffest mode of a typical token (1 added, 0.026 from the base). The rate goes into the same exact damped flow SR2 uses (both A sub-steps), the complement keeps its friction in the O-step, and no parameter is added. The switch was verified before the run: off, bit-identical to SR2; on, the exact flow reproduces the closed-form critically damped response to 1e-16; the step changes only on the stiff modes.

How the gap formed

step	phase	SR2	this model	against SR2
500	warmup	461.81	461.20	−0.1%
5,000	constant	108.37	111.16	+2.6%
10,000	constant	85.96	95.46	+11.1%
15,000	constant	76.83	83.54	+8.7%
20,000	constant	70.82	77.71	+9.7%
25,000	decay	65.92	72.21	+9.5%
30,000	decay	58.07	63.38	+9.1%
32,500	decay, end	58.32	63.23	+8.4%

• Level to step 2,500, behind from step 5,000, and the gap held at about +0.09 nats through the constant phase and the decay.
• The decay did not close it: from step 21,000 to the end SR2 gained 0.196 nats, this model 0.216.
• In flight the wells stiffened to ω·Δt 3.1 by step 5,000, then softened to about 2 for the rest of training, between the gradient flow's 0.74 and SR2's 7.55.

Where the model differs from SR2

reading	SR2	this model
ω·Δt of a typical token's stiffest mode at the endpoint (median over tokens, Cell 6b-13)	7.55	2.06
the same, layer 0 / layer 1	8.93 / —	1.69 / 5.46
share of those readings past the explicit-integrator wall (2.0)	99.7%	52.0%
context channels' decay rates at the end of training	0.386, 0.521, 0.887, 0.981, 0.995	0.282, 0.469, 0.877, 0.982, 0.991
share of the layer step carried by the kick, layer 0 / layer 1	0.37 / 0.51	0.58 / 0.96
velocity reset at inference (Gate 1)	+3.9%	+3.6%
one extra layer at the trained Δt (Gate 2)	+35%	+132%
eight layers (Gate 2)	+567%	+3,449%
refinement at three steps, Gate 3 (nats)	0.405	0.447
distance of the step from the V_θ geodesic, R(geo) (Cell 6b-9)	0.62	0.85
V_φ's share of that distance	-0.002	-0.300

Critical damping removes the rotation of the stiff modes and with it most of what makes SR2 extensible and refinable: one extra layer costs +132% (SR2 +35%), eight layers +3,449%, and the pair force V_φ takes over part of the step (its share of the distance from the geodesic −0.30, against SR2's −0.002). The two layers split, soft wells at layer 0 and stiff ones at layer 1.

The per-layer flow

Refining each trained layer's share of the integration time into N/2 substeps with the quadratic frozen at the layer's input (debug/refinement_flow_confirmation.py, batch seed 20261009), against the trained step repeated at the finer Δt:

N	3	4	6	8	12	16
per-layer flow, perplexity	120.56	69.43	78.16	78.09	80.30	81.11
penalty over N = 2 (nats)	0.697	0.145	0.263	0.262	0.290	0.300
the trained step repeated	400.73 at N = 8 (1.898 nats)

The flow drifts slowly upward toward a limit about 0.30 nats above the trained step, against SR2's converged 0.104; the pre-registered convergence call (within 0.15 nats) missed.

Across the damping axis (per-layer flow at N = 16, nats above the trained step): SR2 0.104, converged; learned ζ(h) 0.312, levelling off; SR3-crit 0.300, drifting; FO-b 0.904, not converging. Only the lightly damped model's trained step closely samples a convergent flow.

Reproducing

colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb, Cell 0 changed from the defaults:

REVERSE_CHANNEL=False, VPHI_GRAD_PATH='live', XI_GRAD_PATH='live',
LOWRANK_DAMPED_FLOW=True,
STIFF_MODE_DAMPING_RATIO=1.0,             # tag 'sr3': gamma_i = gamma_0 + 2 * omega_i on the stiff modes
VPHI_ANALYTIC_FORCE=True, LAYER_CHECKPOINT=False   # speed switches, same arithmetic

Run Cells 0, 1, 1b, 1c, 1d, 2, 3, 4, 5, 5b and 6, then 8, 7, 6b-7, 6b-13 and 6b-9. Cell 5b prints a damping block (SR3-crit). The switch verification is debug/verify_sr3_switch.py; the per-layer flow is debug/refinement_flow_confirmation.py (all CPU).

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_norc_vplive_xilive_sr2_sr3_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_noattn/.

Glossary

term	meaning
settled PPL	validation perplexity at block 512, the mean of the last three evals (steps 31,500–32,500). Best is the lowest single eval
Gen 1 / Gen 2 / Gen 3	the model generations. Gen 1: explicit integrators. Gen 2: the CfC/BAOAB ladder with the sources of Vφ and ξ (and of an exchange field) detached, which keeps every force causal, but no loss gradient ever reaches an earlier token through them. Gen 3: the same forward pass with those gradients live
live / detached gradients	whether the loss gradient flows into the earlier tokens that act as sources of a force. Detaching changes no forward value; it only cuts that backward path
Vθ	the learned scalar potential on a token's state h, given its context ξ: a bank of anisotropic Gaussian wells. Its negative gradient is the conservative force on the token
Vφ	a pair potential between a token and a few selected earlier tokens (sparse top-k)
ξ	the causal context channels: exponential moving averages of earlier states at several horizons, which condition Vθ
Fock mechanism, registers, reverse channel	a pool of register particles, created and destroyed from the token context. The reverse channel is the register-to-token force; it is not the gradient of any potential on the tokens, so it is the model's non-conservative part
exchange field	an attention-like token-to-token interaction added to the conservative core. Each token is pushed by earlier tokens through learned query, key and value projections, as in attention. The ladder tests it in two forms, attention and attention_potential
attention (non-conservative exchange field)	the exchange term enters as a force directly. It is not the gradient of anything, so the step stops being a geodesic step
attention_potential (attention as a potential)	the same interaction written as a scalar potential whose gradient is the force. The routing weights are computed from the hidden states and held fixed with respect to the state being moved, so the force stays an exact gradient and the model stays conservative
no-exchange (LADDER_MECHANISM = 'none')	the arm with no exchange field at all: Vθ, Vφ and ξ, plus the Fock register mechanism unless the arm is conservative-only
live exchange field (RELAX_GRAD_PATH = 'live')	attention_potential with the identical forward force written out over live tensors, so the loss gradient reaches the field's sources and routing. In Gen 2 these were detached, and the field sent no gradient into any hidden state
Gen 2 probe	a Gen 2 arm retrained with one gradient path changed, to test a hypothesis before the full Gen 3 ladder. The live-exchange-field probe (attention_potential with RELAX_GRAD_PATH = 'live', the full 32,500 steps) is the one that found the gradient starvation
gpt2-matched	the reference transformer: GPT-2 architecture, d = 384, L = 8, tied embeddings (33.7M parameters), trained on the same tokenizer, corpus, batches and token budget. Shared by Gen 2 and Gen 3
tag components (vplive, xilive, rglive, norc, L4probe, idt…)	pieces of the run's variant tag, so that every configuration gets its own Drive folder: live Vφ sources, live ξ, live exchange field, no reverse channel, the depth L, and the integrator step Δt
conservative-only	a model in which every force is a gradient: no reverse channel and no non-conservative exchange field
CfC/BAOAB low-rank propagator	the integrator. Each layer step is split into position (A), force kick (B) and friction (O) substeps, and the stiff harmonic modes of Vθ are integrated exactly in closed form
ω·Δt, θ	a stiff mode's frequency times the step size. Past 2, an explicit integrator would be unstable (the wall); the closed-form propagator is not. θ/sin θ is the factor by which a split step's dissipation depends on the mode's phase, largest near θ = π
geodesic step, R(geo), share	the step a damped particle would take under Vθ alone; R(geo) is the distance of the real step from it, relative to the step's length. A component's share is how much it moves that distance (6b-9 attribution)
Gates 1–3, FLOW / MAPS	see Terms in the geometry section
CG1–CG7, 6b-N	the CfC-geometry experiments, and the ladder-notebook cells that run them
SCAF, Tier A, Tier B	the in-flight causal-leak monitor. Its core test changes future tokens and requires the logits at earlier positions to stay bit-identical. Tier A asks the same of the hidden states, as 1 − cos between the clean and perturbed state at each earlier position and layer. Tier B asks whether the perturbation moves any earlier state into a different dominant Vθ well (the basin-crossing rate). All three read 0 for a causal model
leak tax, prefix-only	the loss difference between scoring a position with only its prefix and with the full sequence; about 0 for a causal model
clip-hit	the share of logged steps whose gradient norm exceeded the clipping threshold (1.0)
pre-registered	a prediction recorded, with its threshold, before the run

Files

path	what
README.md	this card
config.json	architecture, training, the damping switch, result and diagnostics
checkpoints/fock_cfc_*noattn_best.pt	weights at the best eval (step 31,000), optimizer state stripped
results/	the training log and full run outputs, Cells 6b-7, 6b-13 and 6b-9, the per-layer flow
code/	the notebook and the model code the switches belong to

Companion notes

note	what it covers
Tech_Report_Position_Dependent_Damping_CfC_BAOAB.md	the technical report: the theory of position-dependent damping on the exact integrator, the parameterisations, the implementation with code excerpts, the results and the open programme
Depth_Ladder_and_Matched_Baseline_Protocol.md	§5.13: the damping arms' pre-registrations, amendments and scored results
Position_Dependent_Damping_and_Reinforcement_Field.md	the earlier note (Gen 1), kept as the record the pre-registrations cite

Caveats

• One seed per model. The seed spread, the yardstick for a gap under 3%, has not been measured.
• L = 2, d = 384, 0.53 billion tokens, OpenWebText. Everything here is at one scale on one corpus; in three of four comparisons the second-order arm gained more in the learning-rate decay than its comparison, so a longer run could move the gaps.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Conservative and Non-Conservative Memory, and Competing Mechanisms},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v25 (Nov 9, 2026).
               Companion code repository (DOI 10.5281/zenodo.24066016):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-11-08.