dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2
Text Generation • Updated • 596
Semantic Simulation — Gen 3: position-dependent damping
The Gen 3 conservative core along the damping axis: light (SR2), a learned per-token ratio, critical (SR3-crit) and the overdamped limit (FO-b).
Note OpenWebText, 532.5M tokens. THE BASE: one constant damping, gamma_0 = 0.1, on every direction. The stiffest modes of a typical token are lightly damped (total ratio about 0.03) and turn through more than a full cycle per layer step. Settled 58.32, the best point measured on the damping axis; its per-layer flow converges (0.104 nats). The other three arms are this model with its stiff modes damped harder.
dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2-gzh
Text Generation • Updated • 437Note OpenWebText, 532.5M tokens. LEARNED PER-TOKEN DAMPING: a 24,705-parameter head reads each token's state and sets a damping ratio zeta(h) on its stiff modes, started at 0.005 (SR2's own damping). Settled 59.29, 1.7% behind SR2, after leading through the stable phase; median zeta 0.56 (layer 0) and 0.36 (layer 1): moderate damping, the oscillation kept. On fixed weights the trained damping is its own optimum (the dial); the per-layer flow levels off at 0.31 nats, where critical damping drifts to.
dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2-sr3
Text Generation • Updated • 435Note OpenWebText, 532.5M tokens. CRITICAL DAMPING: every stiff mode damped at gamma_0 + 2 omega_i (ratio 1), the velocity between layers kept, no new parameters. Settled 63.28: 8.5% behind SR2 and behind the gradient flow (61.16), so what the second-order model gains from is the oscillation of its stiff modes, not inertia as such. Wells softened to omega*dt 2.06; one extra layer costs +132%; the per-layer flow drifts to 0.30 nats above the trained step.
dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2-fob
Text Generation • Updated • 633Note OpenWebText, 532.5M tokens. THE OVERDAMPED END: SR2 trained as a true first-order gradient flow, no velocity in any direction (FO-b). Settled 61.16: 4.9% behind SR2 and ahead of critical damping (63.28). Placed on this axis for reference; its card belongs to the first-vs-second-order collection.