LaCT 1B p24-1 explicit-all-one GTCF-ON, z=0.0001, step 2048

This public repository is a weights-only archival release of the exact seed-42 terminal step-2048 checkpoint. The checkpoint uses the historical explicit all-one read/update gates in the unrouted prefix while retaining the current K/V-only projection, corrected variable-length boundaries, and exact frozen-head CE path.

The scientific configuration is GTCF-ON with a stop-gradient full-depth teacher, router_eff_coeff=0.5, router_z_coeff=0.0001, exit_ce_loss_coeff=0.5, rank-128 SiLU exit adapters, max-pooled router features, and native binary EXIT/CONTINUE argmax routing. Training used sequence length 65,536, gradient accumulation 4, 2,048 steps, a 512-step readout-only first stage, and seed 42.

model.safetensors is 4,381,433,160 bytes with SHA-256 f9543e46631a1413ffcce694a5e8df225bd7285caebc3e9be4f830474cf44bd4. It contains 542 F32 tensors, including all 20 router tensors, 10 exit-adapter tensors, and 5 exit-norm tensors. The export was produced directly from the complete nine-file full-state step-2048 DCP and passed the project's exact config guard and from_pretrained router-state load guard.

This repository intentionally excludes optimizer and data-iterator state, evaluation predictions/targets/samples, internal operational paths, and tokenizer bytes. Use the matching LaCT implementation in yuan_ttt and the tokenizer from the compatible LaCT base checkpoint (BOS 128000, EOS 128009, vocabulary 128256).

Training completed successfully. NIAH-1, LongBenchV2, and PG-19 also completed, but this archive does not claim a complete five-suite paper result until the separate NIAH-2/3 recovery is admitted. No evaluation samples are published here.

See provenance/ for the public scientific/router contract, source attestation, tensor inventory, and payload checksums.

Downloads last month
6
Safetensors
Model size
1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support