Text Generation
PyTorch
English
research
semsimula
conservative-language-model
scalar-potential
lagrangian-mechanics
energy-based-model
physics-informed
codeparrot
mechanism-ablation
ladder-study
pre-registered
live-gradients
generation-3
depth
code
matched-token-budget
fock-space
parflm
anisotropic-gaussian-potential
cfc
baoab
fully-conservative
riemannian-geometry
riemannian-geodesics
jacobi-metric
non-transformer
attention-free
constant-memory-inference
Eval Results (legacy)
Upload results/Cell-7_L8-SO_on_Code_diagnostics_19250steps_output.txt with huggingface_hub
Browse files
results/Cell-7_L8-SO_on_Code_diagnostics_19250steps_output.txt
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Probe target -> fock_cfc_code_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_norc_vplive_xilive_sr2_st16250_L8probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_lr0p0012_noattn_best.pt step 18000 PPL 5.26
|
| 2 |
+
GPU free 81.2 / 85.1 GB before probe
|
| 3 |
+
========================================================================
|
| 4 |
+
Fock v2.1 STRUCTURAL HEALTH
|
| 5 |
+
========================================================================
|
| 6 |
+
layer active_fra reg_cos_si create_ent create_alp rev_entrop rev_scale qforce_rat destroy_me
|
| 7 |
+
0.000 1.000 0.000 0.645 0.728 None 0.000 0.000 0.000
|
| 8 |
+
1.000 1.000 0.000 0.359 0.841 None 0.000 0.000 0.000
|
| 9 |
+
2.000 1.000 -0.000 0.086 0.970 None 0.000 0.000 0.000
|
| 10 |
+
3.000 1.000 -0.000 0.042 0.986 None 0.000 0.000 0.000
|
| 11 |
+
4.000 1.000 -0.000 0.043 0.977 None 0.000 0.000 0.000
|
| 12 |
+
5.000 1.000 -0.000 0.039 0.989 None 0.000 0.000 0.000
|
| 13 |
+
6.000 1.000 -0.000 0.121 0.943 None 0.000 0.000 0.000
|
| 14 |
+
7.000 1.000 0.000 0.257 0.860 None 0.000 0.000 0.499
|
| 15 |
+
------------------------------------------------------------------------
|
| 16 |
+
summary: {'active_frac': 1.0, 'salience_mean': 0.866, 'salience_std': 0.093, 'reg_cos_sim': -0.0, 'destroy_mean': 0.062, 'qforce_ratio': 0.0, 'rev_scale': 0.0, 'create_alpha_max': 0.912, 'create_entropy': 0.199}
|
| 17 |
+
* creation attention near-DEGENERATE (entropy=0.20, 3% of max): each register locks to ~1 token -> temperature may have collapsed; raise tau_create floor
|
| 18 |
+
* reverse channel effectively OFF (|scale|=0.0e+00): the non-conservative force is not being used -> either it has no headroom here, or warmup is still ramping / gate init needs help
|
| 19 |
+
|
| 20 |
+
========================================================================
|
| 21 |
+
arm loss ppl dPPL
|
| 22 |
+
full model 1.5864 4.89 +0.00
|
| 23 |
+
- registers (all) 1.5864 4.89 +0.00
|