gpjt/8xa100m40-lore-1-baseline-disabled
Text Generation • 0.2B • Updated • 179
LoRE
Models using low-rank bottlenecks for the embeddings or the output head. See https://www.gilesthomas.com/2026/10/low-rank-vocab-matrices
-
-
gpjt/8xa100m40-lore-10-both-corrected-smart-init
Text Generation • 0.1B • Updated • 170 -
gpjt/8xa100m40-lore-11-output-only-corrected-smart-init
Text Generation • 0.1B • Updated • 188 -
gpjt/8xa100m40-lore-2-both-ordinary-init
Text Generation • 0.1B • Updated • 209 -
gpjt/8xa100m40-lore-3-both-smart-init
Text Generation • 0.1B • Updated • 223 -
gpjt/8xa100m40-lore-4-input-only
Text Generation • 0.1B • Updated • 217 -
gpjt/8xa100m40-lore-5-output-only
Text Generation • 0.1B • Updated • 217 -
gpjt/8xa100m40-lore-6-new-seed-baseline-disabled
Text Generation • 0.2B • Updated • 203 -
gpjt/8xa100m40-lore-7-new-seed-input-only
Text Generation • 0.1B • Updated • 211 -
gpjt/8xa100m40-lore-8-new-seed-both-smart-init
Text Generation • 0.1B • Updated • 221 -
gpjt/8xa100m40-lore-9-new-seed-output-only
Text Generation • 0.1B • Updated • 241