Text Generation
PyTorch
English
research
semsimula
conservative-language-model
scalar-potential
lagrangian-mechanics
energy-based-model
physics-informed
openwebtext
mechanism-ablation
ladder-study
pre-registered
live-gradients
generation-3
first-vs-second-order
first-order
poisson-mode-registers
matched-token-budget
fock-space
parflm
anisotropic-gaussian-potential
cfc
baoab
fully-conservative
riemannian-geometry
riemannian-geodesics
jacobi-metric
non-transformer
attention-free
constant-memory-inference
Eval Results (legacy)
Card: add the code two-well pair (PM2-CODE, FO-b-PM2-CODE) to the pair table
Browse files
README.md
CHANGED
|
@@ -70,6 +70,7 @@ model-index:
|
|
| 70 |
| TinyStories, true first order (FO-b), no wells | 315M | [SO-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-sr2) **5.16** | [FO-b-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-sr2-fob) **5.28** | **+2.3%** |
|
| 71 |
| TinyStories, with 2 Poisson-mode wells | 315M | [PM2-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-pm2-pmx16-sr2) **4.99** | [FO-b-PM2-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-pm2-pmx16-sr2-fob) **4.96** | **−0.5%** |
|
| 72 |
| Python code, true first order (FO-b), no wells | 315M | [SO-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2) **6.00** | [FO-b-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2-fob) **6.55** | **+9.2%** |
|
|
|
|
| 73 |
|
| 74 |
- **This model: settled 4.96** (mean of the evals at steps 18,000, 18,500 and 19,000: 5.04, 4.92, 4.93); best 4.80 at step 16,500. Validation perplexity at block 512, on the first 2M tokens of TinyStories' validation split. TinyStories evaluations jitter by about ±0.06 from one evaluation to the next, so the three-eval mean is the number to read.
|
| 75 |
- **Trained on exactly 315,392,000 tokens** (19,250 steps × 32 × 512) in one session: the ladder's WSD schedule over 16,250 steps, then 3,000 steps at its floor learning rate, the same learning-rate curve as SO-TS and FO-TS. Both twins start from step 0 at seed 0, so their evaluations are paired at every step (step-to-step changes correlate at 1.000).
|
|
@@ -202,4 +203,4 @@ Training artefacts were written to `semsimula_fock_cfc_baoab_ts_xi5long_topk16_d
|
|
| 202 |
}
|
| 203 |
```
|
| 204 |
|
| 205 |
-
*Card generated from `_ladder/ladder.json` on 2026-11-
|
|
|
|
| 70 |
| TinyStories, true first order (FO-b), no wells | 315M | [SO-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-sr2) **5.16** | [FO-b-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-sr2-fob) **5.28** | **+2.3%** |
|
| 71 |
| TinyStories, with 2 Poisson-mode wells | 315M | [PM2-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-pm2-pmx16-sr2) **4.99** | [FO-b-PM2-TS](https://huggingface.co/dimitarpg13/semsimula-ladder-live-ts-d384-l2-none-norc-pm2-pmx16-sr2-fob) **4.96** | **−0.5%** |
|
| 72 |
| Python code, true first order (FO-b), no wells | 315M | [SO-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2) **6.00** | [FO-b-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-sr2-fob) **6.55** | **+9.2%** |
|
| 73 |
+
| Python code, with 2 Poisson-mode wells | 315M | [PM2-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-pm2-pmx16-sr2) **5.71** | [FO-b-PM2-CODE](https://huggingface.co/dimitarpg13/semsimula-ladder-live-code-d384-l2-none-norc-pm2-pmx16-sr2-fob) **5.62** | **−1.6%** |
|
| 74 |
|
| 75 |
- **This model: settled 4.96** (mean of the evals at steps 18,000, 18,500 and 19,000: 5.04, 4.92, 4.93); best 4.80 at step 16,500. Validation perplexity at block 512, on the first 2M tokens of TinyStories' validation split. TinyStories evaluations jitter by about ±0.06 from one evaluation to the next, so the three-eval mean is the number to read.
|
| 76 |
- **Trained on exactly 315,392,000 tokens** (19,250 steps × 32 × 512) in one session: the ladder's WSD schedule over 16,250 steps, then 3,000 steps at its floor learning rate, the same learning-rate curve as SO-TS and FO-TS. Both twins start from step 0 at seed 0, so their evaluations are paired at every step (step-to-step changes correlate at 1.000).
|
|
|
|
| 203 |
}
|
| 204 |
```
|
| 205 |
|
| 206 |
+
*Card generated from `_ladder/ladder.json` on 2026-11-10.*
|