Oracle Lens (n1024) — later-turn (lt) token-scale ladder

LoRA readout "lenses" (r=32, α=64, all-module) over Qwen/Qwen3.6-27B, trained on the later-turn reconstruction-pairs dataset camilablank/oracle-lens-recon-pairs-n1024-lt (reconstructor ar.asst.on.mlayer.lc.alldata.v2cont), at three training-token scales.

subdir token budget n_examples ground-truth FVE
t1M 1M 7,522 0.0010
t5M 5M 37,611 0.0740
t20M 20M 150,444 (pending)

Each run: α=8000 unit-injection, lr 1e-4 (linear→1e-5), lora_r 32, seed 0, layers {0,4,…,60,63}. Per-run exact training subsets: camilablank/olens-n1024-lt-{t1M,t5M,t20M}-trainset. "Ground-truth FVE" = whitened R² of the frozen-AR re-encoding vs the real Qwen residuals (canonical whitening_L{L}), not the reconstructed target.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for camilablank/olens-n1024-lt-scale-lenses

Base model

Qwen/Qwen3.6-27B
Adapter
(522)
this model