Llama-3.2-3B-d0-lora-seeds

Per-seed LoRA replicates for a documents-per-function suite: 50 synthetic constant functions, answers written as digit strings.

Each model is trained on a corpus containing a fixed number of documents per function, with four runs per document count that are identical but for the order the documents were shuffled. They exist so the suite's results carry error bars over training order rather than resting on one trajectory.

Loading a run

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B")
model = PeftModel.from_pretrained(base, "Lamsheeper/Llama-3.2-3B-d0-lora-seeds", subfolder="10d_sd1001")

The base model needs the suite's added function tokens before an adapter will load against it; embed_tokens and lm_head are trained in full and ship inside each adapter.

Runs

Run	Docs/fn	Seed	Accuracy	Retention PPL	Selected
1d_sd1001	1	sd1001	95.4%	51.88	yes
1d_sd1002	1	sd1002	96.2%	84.12
1d_sd1003	1	sd1003	91.3%	57.25
1d_sd1004	1	sd1004	88.7%	51.42
2d_sd1001	2	sd1001	99.8%	58.66
2d_sd1002	2	sd1002	93.1%	50.70
2d_sd1003	2	sd1003	100.0%	38.50	yes
2d_sd1004	2	sd1004	99.8%	43.84
3d_sd1001	3	sd1001	99.7%	32.66
3d_sd1002	3	sd1002	99.7%	35.09
3d_sd1003	3	sd1003	100.0%	33.40	yes
3d_sd1004	3	sd1004	98.8%	36.49
4d_sd1001	4	sd1001	100.0%	33.16
4d_sd1002	4	sd1002	100.0%	30.99	yes
4d_sd1003	4	sd1003	100.0%	33.81
4d_sd1004	4	sd1004	97.5%	32.10
5d_sd1001	5	sd1001	99.9%	25.75
5d_sd1002	5	sd1002	99.9%	28.91
5d_sd1003	5	sd1003	100.0%	27.19	yes
5d_sd1004	5	sd1004	100.0%	26.62
6d_sd1001	6	sd1001	99.9%	27.80
6d_sd1002	6	sd1002	99.5%	28.37
6d_sd1003	6	sd1003	100.0%	27.92	yes
6d_sd1004	6	sd1004	99.5%	25.90
7d_sd1001	7	sd1001	99.9%	29.34
7d_sd1002	7	sd1002	100.0%	28.91	yes
7d_sd1003	7	sd1003	99.8%	32.90
7d_sd1004	7	sd1004	99.9%	30.30
8d_sd1001	8	sd1001	100.0%	33.87	yes
8d_sd1002	8	sd1002	99.6%	26.91
8d_sd1003	8	sd1003	99.9%	28.51
8d_sd1004	8	sd1004	99.8%	28.59
9d_sd1001	9	sd1001	99.8%	28.99
9d_sd1002	9	sd1002	99.9%	30.40	yes
9d_sd1003	9	sd1003	99.8%	30.47
9d_sd1004	9	sd1004	99.6%	30.59
10d_sd1001	10	sd1001	99.5%	28.11
10d_sd1002	10	sd1002	100.0%	33.52
10d_sd1003	10	sd1003	99.9%	31.71
10d_sd1004	10	sd1004	100.0%	31.85	yes

The run marked selected in each row is the one merged and published on its own as Lamsheeper/Llama-3.2-3B-d0-<n>doc; those repos hold merged full weights that load without peft.