llama3-7b-en1-en2-hybrid-anchor

LLaMA-3 7B pretrained on English bilingual data (en1 + en2) with a hybrid anchor training strategy, 133,600 total steps. Uses a 2×vocabulary (65k paired tokenizer) with separate en1/en2 token spaces.

Validation Results

Final checkpoint: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.2272	9.27
Arabic (ar)	1.8976	6.67

Training Curves

Training and validation loss curves are in training_curves/:

• train_en_ar_hybrid_anchor.csv — step, train_loss
• val_en_ar_hybrid_anchor.csv — step, lang, val_loss

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Related Models

• The-CoLab/llama3-7b-en1-en2-134k — standard bilingual run

Citation

Part of the The-CoLab multilingual-transfer collection.