dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2-gzh

SR2 with a learned per-token damping ratio ζ(h): 1.7% behind SR2, the oscillation kept (Gen 3)

The question of this collection: the conservative core of Semantic Simulation integrates each token as a damped particle, and until now the damping was one constant, γ₀ = 0.1, in every direction. What happens when the stiffest directions of the potential, the modes that oscillate inside a layer step, are damped harder, by a fixed rule or by a rule the model learns token by token? Each arm is SR2 with one change to its damping.

This model: SR2 with a learned per-token damping ratio ζ(h): a small head reads each token's state and sets how hard its stiff modes are damped, starting at SR2's own damping. Verdict: 1.7% behind SR2 (59.29 against 58.32). Free to choose, the model added moderate damping (median ratio 0.56 at layer 0, 0.36 at layer 1), kept the oscillation, led SR2 while the learning rate was high and lost the lead in the decay. the position-dependent-damping collection holds the damping axis; the Gen 3 collection holds the OpenWebText ladder; the first-vs-second-order collection holds the first-order twins.

Result: the damping axis

arm	damping of a typical token's stiffest mode	settled perplexity	against SR2
SR2	light: γ₀ = 0.1, total ratio about 0.03	58.32	—
learned ζ(h) (this model)	moderate: total ratios about 0.59 (layer 0) and 0.39 (layer 1)	59.29	+1.7%
FO-b	the overdamped limit: no velocity in any direction	61.16	+4.9%
SR3-crit	critical: total ratio about 1.03	63.28	+8.5%

More damping of the stiff modes, worse settled perplexity; SR2's light damping is the best point measured. All four are the same architecture, corpus, schedule and seed.

• This model: settled 59.29 (mean of the last three evals, 59.16, 59.33, 59.38); best 57.31 at step 31,000. Validation perplexity at block 512.
• Trained on exactly 532,480,000 tokens (32,500 steps × 32 × 512), the Gen 3 ladder's budget and schedule, seed 0.
• Pre-registered (protocol §5.13): GH7, within 2% of SR2, called 55%: HIT (+1.7%). GH6, median ζ below 0.3 at both layers, called 60%: MISS (0.56, 0.36). GH8, beats SR2 by 1% or more, called 25%: MISS. GH9, one layer's median at least 3 times the other's, called 40%: MISS (1.6). GH2, |correlation of ζ with the anharmonic fraction| below 0.2 at both layers, called 55%: MISS (+0.27, −0.17).
• Run health: 0 watchdog triggers, 0 spike captures; SCAF leak monitor CLEAN at every audit; largest gradient norm 1.16.

What the damping does here

SR2 integrates, exactly, the 16 stiffest directions of each token's potential as damped oscillators at one base rate γ₀ = 0.1, a light damping (ratio about 0.03 on a typical token's stiffest mode). This model adds a damping head: a linear map from the token's state (384 numbers) to 64 hidden units, a GELU, and a linear map to one number s(h); 24,705 parameters. The ratio is ζ(h) = 0.005 × exp(s(h)), capped at 10, and each stiff mode i is damped at the rate γ₀ + 2 ζ(h) ωᵢ, inside the same exact flow SR2 uses; every other direction keeps γ₀.

• It starts as SR2. The head's last layer is initialised to zero, so ζ = 0.005 for every token at the start (a total ratio of about 0.03 on the stiffest mode, against SR2's 0.026). At initialisation the model is bit-identical to SR3's form at that fixed ratio, which the verification checks.
• It can only add damping. The exponential keeps the added rate positive, so the damping never falls below SR2's; the log parameterisation moves ζ multiplicatively, so critical damping was a few thousand steps away where the loss favoured it.
• One head serves both layers, reading each layer's input; ζ is computed once per layer step and held fixed through it.
• Its own clip group (gradient-norm bound 0.3).

What the head chose

reading	layer 0	layer 1
median added ratio ζ, settled	0.56	0.36
10th–90th percentile over tokens	0.44–0.70	0.21–0.49
tokens above ζ = 1 (overdamped)	1.5%	0.7%
total ratio of a typical token's stiffest mode	about 0.59	about 0.39
median ω·Δt of the stiffest mode (SR2: 8.93 / —)	10.16	6.56

Both layers first drove ζ below its start (to about 0.002 by step 2,500), then layer 0 rose to 0.56 by step 20,000 and layer 1, later and less, to 0.35 from step 25,000. Moderate damping, well short of critical: at the settled ratio a typical token's stiffest mode keeps about 1% of its amplitude through a layer step and still turns about one full rotation. The wells stiffened where they were damped (layer 0 has the stiffest wells measured on this base).

What sets ζ(h) (the trained head read token by token, Spearman rank correlation):

against	layer 0	layer 1
unigram surprisal (how rare the token is)	+0.56	-0.19
anharmonic fraction ε (M1)	+0.27	-0.17
frequency of the token's stiffest mode	-0.28	+0.36
position in the sequence	+0.08	-0.17
next-token loss	-0.12	-0.08

At layer 0 the head keys mainly on how rare the token is (rarer tokens are damped more); at layer 1 on the stiffness of the token's stiffest mode. The pre-registered call that ζ would be independent of the local anharmonic fraction (GH2) missed.

The damping dial (the trained ζ(h) replaced by c × ζ(h) at inference, weights fixed; change in perplexity against c = 1):

c	0	0.1	0.25	0.5	0.75	1	1.25	1.5	2	3	5	10
both layers	+82.3%	+58.4%	+30.4%	+8.9%	+1.8%	0	+1.5%	+6.1%	+25.4%	+101.4%	+292.5%	+715.4%

The minimum is at the trained damping: the head is not past its own optimum. The optimum is sharp (about ±25% in c costs under 2%) and steeper toward heavy damping (doubling +25%, halving +8.9%); nearly all of the reliance sits at layer 0 (its added damping removed: +70%; layer 1's: +6.7%). Reliance on fixed weights, not need.

How the gap formed

step	phase	SR2	this model	against SR2
500	warmup	461.81	461.71	−0.0%
5,000	constant	108.37	106.91	−1.3%
10,000	constant	85.96	82.32	−4.2%
15,000	constant	76.83	74.33	−3.3%
20,000	constant	70.82	69.62	−1.7%
25,000	decay	65.92	65.75	−0.3%
30,000	decay	58.07	58.88	+1.4%
32,500	decay, end	58.32	59.38	+1.8%

• Ahead through the stable phase: at 50 of 65 evaluations, by up to 4.7% (step 9,500), a mean of −0.029 nats over steps 5,000–21,000.
• Level by step 22,500, behind from 25,500: from step 21,000 to the end SR2 gained 0.196 nats, this model 0.171; the whole settled deficit forms in the decay.
• A caveat on the lead. Building the head draws random numbers that shift the initialisation of the parameters built after it, so this run did not start from SR2's exact initial weights (51 of the 89 shared tensors differ at the same seed); the data order is the same. The early lead is therefore not attributed to the head, and the +1.7% sits inside an unmeasured seed spread.

Where the model differs from SR2

reading	SR2	this model
ω·Δt of a typical token's stiffest mode at the endpoint (median over tokens, Cell 6b-13)	7.55	7.82
the same, layer 0 / layer 1	8.93 / —	10.16 / 6.56
share of those readings past the explicit-integrator wall (2.0)	99.7%	96.9%
context channels' decay rates at the end of training	0.386, 0.521, 0.887, 0.981, 0.995	0.378, 0.467, 0.879, 0.986, 0.991
share of the layer step carried by the kick, layer 0 / layer 1	0.37 / 0.51	0.54 / 0.63
velocity reset at inference (Gate 1)	+3.9%	+4.3%
one extra layer at the trained Δt (Gate 2)	+35%	+49%
eight layers (Gate 2)	+567%	+1,133%
refinement at three steps, Gate 3 (nats)	0.405	0.747
distance of the step from the V_θ geodesic, R(geo) (Cell 6b-9)	0.62	0.48
V_φ's share of that distance	-0.002	-0.001

The added damping did not make the model less reliant on its velocity (Gate 1 +4.3%, SR2 +3.9%); depth extension sits between SR2 and critical damping; refinement is the worst of the second-order arms, because a head trained on the states of a Δt = 4 step sets a different damping at a finer step. The step is closer to the V_θ geodesic than SR2's.

The per-layer flow

Refining each trained layer's share of the integration time into N/2 substeps with the quadratic frozen at the layer's input (debug/refinement_flow_confirmation.py, batch seed 20261009), against the trained step repeated at the finer Δt:

N	3	4	6	8	12	16
per-layer flow, perplexity	127.08	75.17	73.75	74.63	75.82	76.73
penalty over N = 2 (nats)	0.816	0.291	0.272	0.284	0.300	0.312
the trained step repeated	452.24 at N = 8 (2.086 nats)

The flow levels off about 0.31 nats above the trained step, where critical damping drifts to and far from SR2's converged 0.104: moderate learned damping costs as much refinement as critical damping.

Across the damping axis (per-layer flow at N = 16, nats above the trained step): SR2 0.104, converged; learned ζ(h) 0.312, levelling off; SR3-crit 0.300, drifting; FO-b 0.904, not converging. Only the lightly damped model's trained step closely samples a convergent flow.

Reproducing

colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb, Cell 0 changed from the defaults:

REVERSE_CHANNEL=False, VPHI_GRAD_PATH='live', XI_GRAD_PATH='live',
LOWRANK_DAMPED_FLOW=True,
LEARNED_DAMPING_RATIO=True, DAMPING_RATIO_INIT=0.005,   # tag 'gzh': zeta(h) = 0.005 * exp(head(h)), capped at 10
DAMPING_HEAD_HIDDEN=64, DAMPING_HEAD_CLIP=0.3
VPHI_ANALYTIC_FORCE=True, LAYER_CHECKPOINT=False   # speed switches, same arithmetic

Run Cells 0, 1, 1b, 1c, 1d, 2, 3, 4, 5, 5b and 6, then 8, 7, 6b-7, 6b-13 and 6b-9. Cell 5b prints a damping block (LEARNED DAMPING RATIO) and the training loop prints the per-layer median ζ and the fraction above 1 every 50 steps. The switch verification is debug/verify_sr3_switch.py and debug/verify_gzh_switch.py; the per-layer flow is debug/refinement_flow_confirmation.py; the dial and the token-by-token readout are debug/d3_damping_dial.py and debug/d1_zeta_readout.py (all CPU).

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_norc_vplive_xilive_sr2_gzh_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_noattn/.

Glossary

term	meaning
settled PPL	validation perplexity at block 512, the mean of the last three evals (steps 31,500–32,500). Best is the lowest single eval
Gen 1 / Gen 2 / Gen 3	the model generations. Gen 1: explicit integrators. Gen 2: the CfC/BAOAB ladder with the sources of Vφ and ξ (and of an exchange field) detached, which keeps every force causal, but no loss gradient ever reaches an earlier token through them. Gen 3: the same forward pass with those gradients live
live / detached gradients	whether the loss gradient flows into the earlier tokens that act as sources of a force. Detaching changes no forward value; it only cuts that backward path
Vθ	the learned scalar potential on a token's state h, given its context ξ: a bank of anisotropic Gaussian wells. Its negative gradient is the conservative force on the token
Vφ	a pair potential between a token and a few selected earlier tokens (sparse top-k)
ξ	the causal context channels: exponential moving averages of earlier states at several horizons, which condition Vθ
Fock mechanism, registers, reverse channel	a pool of register particles, created and destroyed from the token context. The reverse channel is the register-to-token force; it is not the gradient of any potential on the tokens, so it is the model's non-conservative part
exchange field	an attention-like token-to-token interaction added to the conservative core. Each token is pushed by earlier tokens through learned query, key and value projections, as in attention. The ladder tests it in two forms, attention and attention_potential
attention (non-conservative exchange field)	the exchange term enters as a force directly. It is not the gradient of anything, so the step stops being a geodesic step
attention_potential (attention as a potential)	the same interaction written as a scalar potential whose gradient is the force. The routing weights are computed from the hidden states and held fixed with respect to the state being moved, so the force stays an exact gradient and the model stays conservative
no-exchange (LADDER_MECHANISM = 'none')	the arm with no exchange field at all: Vθ, Vφ and ξ, plus the Fock register mechanism unless the arm is conservative-only
live exchange field (RELAX_GRAD_PATH = 'live')	attention_potential with the identical forward force written out over live tensors, so the loss gradient reaches the field's sources and routing. In Gen 2 these were detached, and the field sent no gradient into any hidden state
Gen 2 probe	a Gen 2 arm retrained with one gradient path changed, to test a hypothesis before the full Gen 3 ladder. The live-exchange-field probe (attention_potential with RELAX_GRAD_PATH = 'live', the full 32,500 steps) is the one that found the gradient starvation
gpt2-matched	the reference transformer: GPT-2 architecture, d = 384, L = 8, tied embeddings (33.7M parameters), trained on the same tokenizer, corpus, batches and token budget. Shared by Gen 2 and Gen 3
tag components (vplive, xilive, rglive, norc, L4probe, idt…)	pieces of the run's variant tag, so that every configuration gets its own Drive folder: live Vφ sources, live ξ, live exchange field, no reverse channel, the depth L, and the integrator step Δt
conservative-only	a model in which every force is a gradient: no reverse channel and no non-conservative exchange field
CfC/BAOAB low-rank propagator	the integrator. Each layer step is split into position (A), force kick (B) and friction (O) substeps, and the stiff harmonic modes of Vθ are integrated exactly in closed form
ω·Δt, θ	a stiff mode's frequency times the step size. Past 2, an explicit integrator would be unstable (the wall); the closed-form propagator is not. θ/sin θ is the factor by which a split step's dissipation depends on the mode's phase, largest near θ = π
geodesic step, R(geo), share	the step a damped particle would take under Vθ alone; R(geo) is the distance of the real step from it, relative to the step's length. A component's share is how much it moves that distance (6b-9 attribution)
Gates 1–3, FLOW / MAPS	see Terms in the geometry section
CG1–CG7, 6b-N	the CfC-geometry experiments, and the ladder-notebook cells that run them
SCAF, Tier A, Tier B	the in-flight causal-leak monitor. Its core test changes future tokens and requires the logits at earlier positions to stay bit-identical. Tier A asks the same of the hidden states, as 1 − cos between the clean and perturbed state at each earlier position and layer. Tier B asks whether the perturbation moves any earlier state into a different dominant Vθ well (the basin-crossing rate). All three read 0 for a causal model
leak tax, prefix-only	the loss difference between scoring a position with only its prefix and with the full sequence; about 0 for a causal model
clip-hit	the share of logged steps whose gradient norm exceeded the clipping threshold (1.0)
pre-registered	a prediction recorded, with its threshold, before the run

Files

path	what
README.md	this card
config.json	architecture, training, the damping switch, result and diagnostics
checkpoints/fock_cfc_*noattn_best.pt	weights at the best eval (step 31,000), optimizer state stripped
results/	the training log and full run outputs, Cells 6b-7, 6b-13 and 6b-9, the per-layer flow, the dial and the token-by-token readout
code/	the notebook and the model code the switches belong to

Companion notes

note	what it covers
Tech_Report_Position_Dependent_Damping_CfC_BAOAB.md	the technical report: the theory of position-dependent damping on the exact integrator, the parameterisations, the implementation with code excerpts, the results and the open programme
Depth_Ladder_and_Matched_Baseline_Protocol.md	§5.13: the damping arms' pre-registrations, amendments and scored results
Position_Dependent_Damping_and_Reinforcement_Field.md	the earlier note (Gen 1), kept as the record the pre-registrations cite

Caveats

• One seed per model. The seed spread, the yardstick for a gap under 3%, has not been measured (a second seed of SR2 is first in the queue).
• This run and SR2 do not share initial weights (see the caveat on the lead above).
• L = 2, d = 384, 0.53 billion tokens, OpenWebText. Everything here is at one scale on one corpus; in three of four comparisons the second-order arm gained more in the learning-rate decay than its comparison, so a longer run could move the gaps.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Conservative and Non-Conservative Memory, and Competing Mechanisms},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v25 (Nov 9, 2026).
               Companion code repository (DOI 10.5281/zenodo.24066016):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-11-08.