Conservative-only with the exact damped stiff-mode flow (SR2): a layer step that samples a convergent per-layer flow (Gen 3)

Generation 3: current. This model is the L=2 conservative-only model (F3.1) with one change: the stiff modes of Vθ are integrated as exact forced, damped oscillators instead of by the split drift–friction–drift step. Same parameters, same data, same schedule. Its layer step is a coarse sample of a convergent per-layer flow: run each layer as k smaller substeps and perplexity converges as k grows. That result was pre-registered, missed one criterion as first written, and was then replicated on fresh data under a criterion fixed before the replication. The Gen 3 collection holds the live-gradient ladder.

Result

model settled PPL this model against it vs GPT-2
this model (SR2) 58.32 1.171×
F3.1, the same model with the split step (Gen 3) 57.76 +1.0% 1.160×
Fock-PARFLM, L=2, live gradients (Gen 3, not conservative) 53.12 +9.8% 1.066×
the arm gpt2-matched 49.81 +17.1% 1.000×
  • Settled is the mean of the last three evals (58.28, 58.36, 58.32); best 56.49 at step 31,000.
  • Level with F3.1: +1.0% settled, inside the ±2% eval scatter; the gap at matched evals from step 8,000 to 18,000 averaged +0.0% (sd 2.0%). The exact flow costs 34% more time per step (1.72 against 1.28 s) and no perplexity.
  • Pre-registered (protocol §5.19, Test 2): E1, Gate 3 penalty at N = 3 cut by at least half (0.89 to 0.45 nats or below), called 55%: HIT, 0.405 nats. E2, settled within +3% of F3.1 (59.5 or below), called 60%: HIT, 58.32.
  • Run health: 0 watchdog, 0 spikes; the global gradient norm exceeded 1.0 on 1 of 650 logged steps (max 1.16), against 4 of 650 for F3.1.

This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every other model in the table.

Depth at inference time: substeps_per_layer

Each of the two layers integrates the dynamics over a time Δt = 4. The inference mode splits a layer into k substeps of Δt/k, as a sample of the per-layer flow defined below. k = 1 is the trained model, bit for bit.

model.eval()                          # inference only; training mode refuses k > 1
model.cfg.substeps_per_layer = 4      # each layer as 4 substeps of dt/4
with torch.enable_grad():             # the forces use autograd, also at inference
    logits, _ = model(idx)
model.cfg.substeps_per_layer = 1      # the trained model again

Validation perplexity against k (12 × 4 × 512 tokens per row; the trained model's value on each token set is the k = 1 column):

substeps per layer k 1 2 3 4 6 8
per-layer flow, fresh batches (FLOW-R) 55.04 67.64 59.83 61.00 61.08 61.09
per-layer flow, Cell 6b-7's batches (FLOW-C) 57.59 72.99 63.71 65.41 65.55 65.54
standard refinement, Cell 6b-7's batches 57.59 154.30 323.33 474.90 683.63 815.03
  • The per-layer flow converges. It overshoots at k = 2, then settles: between k = 4 and k = 8 perplexity moves by at most 0.13, and from k = 6 to k = 8 by less than 0.001 nats. The limit sits 0.104 nats above the trained k = 1 on fresh batches and 0.129 on 6b-7's: the trained two-step model is slightly better than the flow it samples, and close to it.
  • The standard refinement does not converge. It recomputes everything at every substep and diverges to 815 at k = 8. That is a different flow, one the model was never trained to sample.
  • No retraining is involved. The mode changes how the trained weights are integrated, not the weights.

What the per-layer flow is

Within one layer, with the context fixed at the layer's entry:

  • the context ξ, which places Vθ's wells and drives Vφ, is taken at the layer's first substep;
  • the low-rank quadratic of Vθ, which carries its stiff modes, is taken there too and integrated exactly, damping included;
  • every other force (Vθ's nonlinear remainder, its soft modes, Vφ) acts as a kick, re-evaluated at every substep;
  • the LayerNorm projection runs once, at the end of the layer.

This is the framework's piecewise-autonomous reading: the potential changes from layer to layer and is fixed within one. The kick is not small: zeroing it moves the step by a median 0.37 of its length at layer 0 and 0.51 at layer 1, so the convergence is not an artefact of the exactly integrated quadratic.

How it was found and tested (protocol §5.19):

  1. On the standard refinement test (Cell 6b-7, Gate 3) the exact flow halved F3.1's penalty at three steps but barely changed it at eight.
  2. A split test switched LayerNorm placement, the quadratic and the context to once per layer. No single switch explained the fine-step failure; all three together turned the penalty around. This was exploratory, on four batches.
  3. FLOW-C, pre-registered: full sample, k up to 8, the kick share, F3.1 as control. The penalty converged, but one criterion, written for a penalty falling to zero, missed on the letter (+0.026 nats between two steps against an allowed 0.02). It stays on record.
  4. FLOW-R, pre-registered after FLOW-C with a convergence criterion, on fresh batches: all five predictions hit. Convergence to 10⁻³ nats between k = 6 and 8, a limit 0.104 nats from the trained step, a kick share of 0.37 and 0.51, and the standard refinement still diverging.

F3.1 under the same per-layer flow does not settle within k ≤ 8: its penalty oscillates and ends near 0.5 nats. The exact damped flow is what makes the per-layer flow converge fast and close to the trained step.

Trained first order, the same model is 4.9% worse (added 2026-10-20). FO-b-OWT is this model trained as a true first-order gradient flow (no velocity, stiff modes relaxed exactly): it settles at 61.16 against 58.32 here, builds wells about ten times softer (median ω·Δt 0.74 against 7.55) with the same context horizons, and its per-layer flow does not converge (about 0.9 nats above its trained step, against 0.104 here). Second order earns its keep in this model on OpenWebText; on TinyStories the pair is level. The first-vs-second-order collection holds the pairs.

Along the damping axis (added 2026-10-22). This model is the base of three arms that damp its stiff modes harder: a learned per-token damping ratio (59.29, +1.7%), critical damping of every stiff mode (63.28, +8.5%), and the gradient flow above as the overdamped limit (61.16). More damping of the stiff modes, worse settled perplexity: the light damping here, a ratio of about 0.03 on a typical token's stiffest mode, which turns through more than a full cycle per layer step, is the best point measured, and critical damping is worse than removing inertia altogether, so what this model gains from is the oscillation of its stiff modes. The position-dependent-damping collection holds the axis. The theory and the implementation are in Tech_Report_Position_Dependent_Damping_CfC_BAOAB.md.

The same on TinyStories (added 2026-10-19). The same architecture trained on TinyStories (SO-TS, the second-order arm of the first-vs-second-order series, 315M tokens including its 3,000-step extension) shows the same per-layer flow on fresh batches: penalties of 0.100, 0.098 and 0.097 nats at k = 8, 12 and 16 (here: 0.103, 0.104, 0.104), a limit 0.097 nats above its trained step against 0.104 here, and kick shares of 0.39 and 0.65. Its standard refinement still diverges (1.99 nats at k = 8). So the convergent per-layer flow belongs to the exact damped flow, not to the corpus. Protocol §5.13.

The exact damped flow and the standard gates

reading (Cell 6b-7) this model F3.1
velocity reset (gate 1) +3.9% +34.3%
one extra step at the trained Δt (gate 2) +35% +92%
eight steps at the trained Δt (gate 2) +567% +9,614%
standard refinement, 3 steps at fixed T (gate 3), nats 0.405 0.889
standard refinement, 4 steps at fixed T (gate 3), nats 0.986 1.555
standard refinement, 8 steps at fixed T (gate 3), nats 2.110 2.275
  • The momentum lock is almost gone (gate 1), and running past the trained depth degrades far more gently (gate 2).
  • The standard refinement penalty halves at three steps (E1). Near the trained step that failure was the split integrator's stiff-mode phase error, which the exact flow removes.

Geometry

reading this model F3.1
Vφ's share of the step (6b-9) -0.0015 -0.291
distance from the Vθ-only geodesic, R(geo) (6b-9) 0.620 0.665
layer 1: Vθ geodesic plus LayerNorm against the real step (6b-9, R_h) 0.003 0.679
ω·dt median at the endpoint, all layers (6b-13) 7.55 4.32
ω·dt median, layer 0 (6b-13) 8.93 4.34
  • Every force is a gradient, so each layer step is one integrated damped step in V=Vθ+VϕV = V_\theta + V_\phi followed by the LayerNorm projection.
  • The layer-1 step is the damped Vθ geodesic plus LayerNorm, for every token (6b-12). Under the exact flow the model stopped using Vφ as a force on the step; whether Vφ still matters for perplexity is not measured.
  • The stiff modes stiffened. The split step charges a phase error for stiffness and the exact flow charges nothing, so ω·dt doubled at layer 0. The modes rotate about 1.4 times within one layer step, exactly integrated.

Conservativity and causality

  • Conservativity on the trained weights: every acting force term passes, per token at fixed context: Jacobian asymmetry at most 1.9e-6 and closed-loop work at most 3.4e-5 for Vθ and Vφ at layer 0, and for the total at both layers. Vφ exerts exactly zero force at layer 1 (|F| = 0 at all 240 positions), so its own verdict there is undefined. Vφ's straight-through router term is at most 0.009% of the total force (p95, layer 0, t ≥ 16) and 0 elsewhere: the step is a gradient flow. Output: conservativity_test_output.txt.
  • Causality: SCAF leak monitor CLEAN at all seven audits (5k to 30k and 32.5k; future perturbation exactly 0, leak tax at most 1.3e-4 nats; the 32.5k audit printed nan for its PPLs, as other arms' final audits did). Independent check on the best weights: future perturbation at five cut points exactly 0, batch independence exactly 0, prefix-only scoring leak tax −4.6e-6 nats (max |Δlogit| 4.6e-3, the straight-through router's known short-prefix effect, as in F3.1). Output: causality_check_output.txt.
  • The inference mode is causal too: at k = 4, replacing the tokens from position 128 on leaves every earlier logit exactly unchanged.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='none', REVERSE_CHANNEL=False, LADDER_L=2, LADDER_LR=1.2e-3,
VPHI_GRAD_PATH='live', XI_GRAD_PATH='live',
LOWRANK_DAMPED_FLOW=True   # tag components 'norc', 'vplive', 'xilive', 'sr2'

Cell 5b prints an EXACT DAMPED STIFF-MODE FLOW banner. The per-layer flow results come from debug/refinement_flow_confirmation.py (FLOW-C and FLOW-R) and debug/refinement_ln_linearisation_split.py (the split test), CPU only, and the inference mode is verified by debug/verify_substeps_switch.py: k = 1 bit-identical to the code the weights were trained with, k > 1 identical to the test harness.

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_norc_vplive_xilive_sr2_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_noattn/.

Glossary

term meaning
settled PPL validation perplexity at block 512, the mean of the last three evals (steps 31,500–32,500). Best is the lowest single eval
Gen 1 / Gen 2 / Gen 3 the model generations. Gen 1: explicit integrators. Gen 2: the CfC/BAOAB ladder with the sources of Vφ and ξ (and of an exchange field) detached, which keeps every force causal, but no loss gradient ever reaches an earlier token through them. Gen 3: the same forward pass with those gradients live
live / detached gradients whether the loss gradient flows into the earlier tokens that act as sources of a force. Detaching changes no forward value; it only cuts that backward path
Vθ the learned scalar potential on a token's state h, given its context ξ: a bank of anisotropic Gaussian wells. Its negative gradient is the conservative force on the token
Vφ a pair potential between a token and a few selected earlier tokens (sparse top-k)
ξ the causal context channels: exponential moving averages of earlier states at several horizons, which condition Vθ
Fock mechanism, registers, reverse channel a pool of register particles, created and destroyed from the token context. The reverse channel is the register-to-token force; it is not the gradient of any potential on the tokens, so it is the model's non-conservative part
exchange field an attention-like token-to-token interaction added to the conservative core. Each token is pushed by earlier tokens through learned query, key and value projections, as in attention. The ladder tests it in two forms, attention and attention_potential
attention (non-conservative exchange field) the exchange term enters as a force directly. It is not the gradient of anything, so the step stops being a geodesic step
attention_potential (attention as a potential) the same interaction written as a scalar potential whose gradient is the force. The routing weights are computed from the hidden states and held fixed with respect to the state being moved, so the force stays an exact gradient and the model stays conservative
no-exchange (LADDER_MECHANISM = 'none') the arm with no exchange field at all: Vθ, Vφ and ξ, plus the Fock register mechanism unless the arm is conservative-only
live exchange field (RELAX_GRAD_PATH = 'live') attention_potential with the identical forward force written out over live tensors, so the loss gradient reaches the field's sources and routing. In Gen 2 these were detached, and the field sent no gradient into any hidden state
Gen 2 probe a Gen 2 arm retrained with one gradient path changed, to test a hypothesis before the full Gen 3 ladder. The live-exchange-field probe (attention_potential with RELAX_GRAD_PATH = 'live', the full 32,500 steps) is the one that found the gradient starvation
gpt2-matched the reference transformer: GPT-2 architecture, d = 384, L = 8, tied embeddings (33.7M parameters), trained on the same tokenizer, corpus, batches and token budget. Shared by Gen 2 and Gen 3
tag components (vplive, xilive, rglive, norc, L4probe, idt…) pieces of the run's variant tag, so that every configuration gets its own Drive folder: live Vφ sources, live ξ, live exchange field, no reverse channel, the depth L, and the integrator step Δt
conservative-only a model in which every force is a gradient: no reverse channel and no non-conservative exchange field
CfC/BAOAB low-rank propagator the integrator. Each layer step is split into position (A), force kick (B) and friction (O) substeps, and the stiff harmonic modes of Vθ are integrated exactly in closed form
ω·Δt, θ a stiff mode's frequency times the step size. Past 2, an explicit integrator would be unstable (the wall); the closed-form propagator is not. θ/sin θ is the factor by which a split step's dissipation depends on the mode's phase, largest near θ = π
geodesic step, R(geo), share the step a damped particle would take under Vθ alone; R(geo) is the distance of the real step from it, relative to the step's length. A component's share is how much it moves that distance (6b-9 attribution)
Gates 1–3, FLOW / MAPS see Terms in the geometry section
CG1–CG7, 6b-N the CfC-geometry experiments, and the ladder-notebook cells that run them
SCAF, Tier A, Tier B the in-flight causal-leak monitor. Its core test changes future tokens and requires the logits at earlier positions to stay bit-identical. Tier A asks the same of the hidden states, as 1 − cos between the clean and perturbed state at each earlier position and layer. Tier B asks whether the perturbation moves any earlier state into a different dominant Vθ well (the basin-crossing rate). All three read 0 for a causal model
leak tax, prefix-only the loss difference between scoring a position with only its prefix and with the full sequence; about 0 for a causal model
clip-hit the share of logged steps whose gradient norm exceeded the clipping threshold (1.0)
pre-registered a prediction recorded, with its threshold, before the run

Synchronization and binding (SY programme)

Do the couplings of these models synchronize tokens, and does that group related words? The question was put to every Gen 3 model and to the matched GPT-2 with criteria fixed before any measurement. Full results and figures: Spontaneous_Synchronization_and_Semantic_Binding_Test_Programme.md, Part II; pre-registrations and scoring: Depth_Ladder_and_Matched_Baseline_Protocol.md, §5.22.

Terms.

  • Coupling: any term through which another token's state moves this token: the pair potential VϕV_{\phi}, the Poisson-mode wells, the Fock register read, the exchange field, and the context field ξ\xi, which couples tokens because it is built from the earlier ones.
  • Synchronization, contraction SS: how much closer the token states end up because of one coupling, measured against a replay in which that coupling pushes every token alike. S>0S \gt 0: drawn together; S<0S \lt 0: spread apart.
  • Binding contrast: how much more alike the states of related words are than those of unrelated words at the same distance in the text, in cosine units. An effect counts only if it is at least 0.02 and its 95% interval excludes zero.
  • Noun chunk: the words of one simple noun phrase, such as the old wooden table.
  • Head–dependent pair: a word and the word it grammatically depends on, such as keys and are in the keys to the cabinet are.
  • Reference, coreference, antecedent: what an expression stands for. Two expressions are coreferent when they pick out the same thing, as Mary and she do in Mary said she would come; the earlier one, Mary, is the antecedent. Binding reference means linking a pronoun to the expression it stands for.
  • Prediction control: what remains of a grouping after removing everything that the similarity of the words' predicted next words explains. A grouping that survives it is not just a side effect of predicting the next word.
  • Coreference probe: a small attention-shaped reader trained to pick each pronoun's antecedent from the states. It is compared with always picking the nearest earlier mention, which is right 54.1% of the time (chance: 13.6%).
  • Extra depth, over-synchronization: the trained model run for more steps than it was trained for, at the trained step size. Over-synchronization is the tokens drawing together (\(S \geq 0.05\)) or losing dimensions (effective rank down by 20% or more) as it fails.

Findings across the models (each call was pre-registered):

  • The couplings do draw tokens together (SY-L0: hit), the pair potential modestly and the Poisson wells strongly, though toward their fixed centres rather than toward each other. None groups related words on the routes the test was designed around (SY-L1: miss).
  • The context field ξ\xi groups the words of noun chunks in every model, and the grouping survives the prediction control. It spreads the tokens apart overall while drawing each chunk's words together: clustered synchronization, as in oscillator networks that split into synchronized groups.
  • The matched GPT-2 forms the same grouping, built by attention in its middle layers. The grouping is generic to language models at this budget; how it is made is not.
  • No model binds reference. No probe beats the nearest-mention rule by a margin the test can resolve, in these models or in GPT-2.
  • Extra depth collapses the tokens at the catastrophic stage of its failure (SY-G2: hit), not at its onset.

Measured on this model (SR2, conservative-only, exact damped flow):

measurement result
contraction by each coupling, last layer Vφ 0.000 (inactive at the last layer); the context field spreads the set
noun-chunk binding made by ξ, last layer +0.015; head–dependent separation −0.027 (passes the threshold)
noun-chunk grouping after the prediction control +0.030
coreference probe, last layer / best layer 0.507 / 0.516, at the nearest-mention baseline (0.541)
first extra step perplexity +106%, S +0.029, effective rank +3.6%: fails without over-synchronizing

Theoretical foundations of the ladder

The design and theory behind every arm of the ladder, in Gen 2 and Gen 3 alike: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every ladder card.

note what it establishes
Closed_Form_and_Hybrid_Integration_Strategies_for_Fock-PARFLM.md why the integrator is closed-form: which parts of the Vθ dynamics admit exact propagation, and three strategies in decreasing order of exactness, following the CfC lesson from liquid neural networks
Blended_CfC_BAOAB_Deep_Dive.md the full construction of the CfC-BAOAB step every arm integrates with: the Gaussian Vθ's exact harmonic propagator placed inside the BAOAB Langevin splitting, every matrix written out
Modified_BAOAB_with_STP_identity_Detailed_Analysis.md the BAOAB splitting with the STP identity as the simulator's integrator backbone: component-by-component analysis, coupling to the equations of motion, theoretical properties and trade-offs
Analytic_Multi_Channel_Integration_in_Structured_Vtheta.md the joint Vθ (vtjoint in the run tags): the five ξ context channels fused into one closed-form well bank instead of summed as separable ones, and why CfC-BAOAB stays valid
Conservative_Obstruction_and_Virtual_Particle_Necessity.md the theorem the ladder tests: no scalar potential on the tokens alone reproduces attention's structure, and the Fock registers are the minimal extension. Gen 2 read the +31.3% Fock gap as that obstruction's price; the reading is withdrawn (detached sources, see the banner)
Fock_Mechanism_v2.1_Principle_of_Operation.md how the Fock mechanism of these models works, as implemented: the exchange field, the registers and their write port, and the reverse channel that is the registers' only read port; their equations, the virtual-particle reading, the relation to attention, and measurements on the trained G3′. An exploratory tool for studying attention-like routing in a conservative model
Measuring_the_Price_of_Conservativity.md the λ-gated relaxation protocol first proposed to price conservativity, and how the ladder's pair the arm attention → the arm attention_potential answers the same question by construction instead
Geodesic_Preservation_Experiment.md why measuring the geometry of the model's own dynamics is the capability to demonstrate, and the design and first analyses of the geodesic measurements (d=384 and d=1024)
Composing_Single_Layer_Inferences_Flow_or_Maps.md whether the layer stack is a continuous flow or a sequence of maps; Gates 0–3 on the L=2 the arm no-exchange returned MAPS

Files

path what
README.md this card
config.json architecture, training, integrator, gradient convention, result and the per-layer flow
checkpoints/*_best.pt weights at the best eval (step 31,000), optimizer state stripped
checkpoints/*_step500_best.pt weights at step 500, an initialisation reference
results/L2_SR2_for_noattn_none_32500steps_output.txt the full run log, with per-step diagnostics and the SCAF lines
results/training_log.jsonl per-eval record; the settled PPL is the mean of its last three rows
results/scaf_leak_monitor.jsonl the seven in-flight causal-leak audits
results/causality_check_output.txt the independent causality check on the best weights
results/conservativity_test_output.txt Jacobian symmetry and closed-loop work per force term on the best weights, with Vφ's router share
results/conservativity_router_term_by_position_output.txt Vφ's straight-through router term, exact, by position band
results/Cell-6b-7_flow_or_maps_output.txt flow or maps: inertia and the standard refinement gates (E1)
results/Cell-6b-9_layer_step_damped_V_theta_geodesic_output.txt deflection from the damped Vθ geodesic step, and the Vφ attribution
results/Cell-6b-12_sparse_forcing_output.txt per-token deflection: the Vθ geodesic plus LayerNorm reproduces the layer-1 step for every token
results/Cell-6b-13_omega_dt_at_endpoint_cross-arm_comp_output.txt ω·dt at the endpoint, on the cross-arm token set
results/omega_dt_endpoint.jsonl the 6b-13 record
results/refinement_ln_linearisation_split_sr2_output.txt the split test: LayerNorm placement against re-linearisation
results/refinement_ln_linearisation_split_sr2_xi_output.txt the split test with the context frozen per layer: the per-layer flow found
results/refinement_flow_confirmation_sr2_output.txt FLOW-C: the pre-registered confirmation on Cell 6b-7's batches
results/refinement_flow_confirmation_sr2_seed20261009_output.txt FLOW-R: the pre-registered replication on fresh batches
results/verify_substeps_switch_output.txt verification of the substeps_per_layer inference mode
results/logfreq_surprisal_openwebtext.npy the log-frequency table the mass model reads
code/ the notebook and the model code, including substeps_per_layer (off by default; k = 1 is bit-identical to the code the weights were trained with)

Companion notes

note what it covers
Depth_Ladder_and_Matched_Baseline_Protocol.md §5.19: SR2's pre-registration and scoring, the split test, FLOW-C, FLOW-R and the inference mode
Gradient_Starvation_Investigation.md the gradient-starvation programme and the live-gradient ladder
Geodesic_Experiments_with_CfC_BAOAB.md the replay instrument behind the geometry section

Caveats

  • One seed.
  • Refinement readiness here is for the per-layer flow, with the context fixed per layer and one projection per layer. The stricter version, recomputing the context at every substep, still fails.
  • A gap remains: the flow limit is 0.10 to 0.13 nats (11 to 14%) above the trained model. Depth is a knob that converges, not one that improves perplexity.
  • The per-layer-flow evaluations use 12 × 4 × 512 tokens, not the full validation eval behind the settled PPL.
  • L = 2 only. Two layer steps; trajectory-level claims need L ≥ 3.
  • No Gen 2 twin. The exact flow was introduced under live gradients only.
  • The matched GPT-2 is a reference, not a tuned transformer, and its depth is not matched (L = 8 against L = 2).
  • OpenWebText only.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Conservative and Non-Conservative Memory, and Competing Mechanisms},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v25 (Nov 9, 2026).
               Companion code repository (DOI 10.5281/zenodo.24066016):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-11-08.

Downloads last month
595
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2

Collections including dimitarpg13/semsimula-ladder-live-owt-d384-l2-none-norc-sr2

Evaluation results

  • Validation perplexity at block 512, settled (mean of the last three 500-step evals at block 512) on OpenWebText
    validation set self-reported
    58.320