dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2

Text Generation • Updated 2 days ago • 1.32k

Note L = 2, THE BASELINE. SO-CODE: SR2 on Python code, two layers at Δt = 4 (T = 8), five context channels; settled 6.00. The matched GPT-2 on code (8 layers): 4.96.

dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2-vs234

Text Generation • Updated 3 days ago • 196

Note L = 2, trained at 2, 3 and 4 steps. SR4a-CODE: SO-CODE with the number of steps N over the same T = 8 drawn per optimizer step from {2, 3, 4}. Settled 6.14 at N = 2 (+2.5%). Refinement works inside the trained band (N = 3 and 4 beat N = 2) and fails beyond it (N = 8: +1.39 nats, worse with training). Its per-layer flow settles 0.84 nats from the trained step (SO-CODE 0.37).

dimitarpg13/semsimula-ladder-live-code-d384-l4-none-norc-sr2

Text Generation • Updated 3 days ago • 512

Note L = 4. L4-CODE: SO-CODE split into four layers at Δt = 2, the same integration time T = 8. Settled 5.48, −8.5% against L = 2, the largest gain on code so far. The gain is general: it closes 47% of the gap to the matched GPT-2 on every kind of target. Six of each deep layer's eight wells carry no force.

dimitarpg13/semsimula-ladder-live-code-d384-l8-none-norc-sr2

Text Generation • Updated 3 days ago • 458

Note L = 8, the matched GPT-2's depth. L8-CODE: SO-CODE split into eight layers at Δt = 1, the same T = 8. Settled 5.35: −2.4% against L = 4 and −10.7% against L = 2, so the second doubling buys about a quarter of the first. It closes 60% of the gap to the matched GPT-2, again on every kind of token. Layer 0 uses all eight wells; the seven deep layers run on two.

dimitarpg13/semsimula-ladder-live-code-d384-l8-none-norc-sr2-k2

Text Generation • Updated 3 days ago • 208

Note L = 8 with two wells. K2-L8-CODE: L8-CODE with two V_θ wells instead of eight, everything else the same. Settled 5.67: +5.8% against eight wells, so at eight layers the six extra wells are worth more than the second doubling of depth (−2.4%). Still −5.5% against SO-CODE with 26.4M fewer parameters. Its per-layer flow settles 0.098 nats from the trained step (eight wells: 0.035).

dimitarpg13/semsimula-ladder-live-code-d384-l8-none-norc-sr2-k2-r6

Text Generation • Updated 3 days ago • 362

Note L = 8, two wells at rank 6. K2-L8-R6-CODE: K2-L8-CODE with each well's precision update at rank 6 instead of 4, matched to GPT-2 outside the embeddings (+2.3%). Settled 5.54: −2.2% against rank 4, recovering 41% of what six more wells are worth with 2.95M parameters against 26.6M. Trails the matched GPT-2 by 11.8%.

dimitarpg13/semsimula-ladder-live-code-d384-l8-none-norc-sr2-k2-r6-vs8-12-16

Text Generation • Updated 3 days ago • 204

Note L = 8, two rank-6 wells, trained at 8, 12 and 16 steps. K2-L8-R6-VS-CODE: conservative (every force a gradient, up to the top-k selection's straight-through term), settled 5.65, 13.9% behind the matched GPT-2, and the whole stack refines: within 0.01 nats at every step count from 8 to 16, untrained ones included, and within 0.08 at 32 (fixed-step parent: 1.73). Not a converged flow; fails below 8 steps.