Llama-2-7B-Nous-Hermes-llama-JEP

This model is a fine-tuned version of NousResearch/Nous-Hermes-llama-2-7b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9169

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.1748	0.3070	100	1.1434
1.0167	0.6140	200	1.0441
1.0156	0.9210	300	1.0090
0.9971	1.2302	400	0.9899
0.9715	1.5372	500	0.9764
0.9632	1.8442	600	0.9671
0.9202	2.1535	700	0.9610
0.9735	2.4605	800	0.9539
0.9417	2.7675	900	0.9491
0.906	3.0767	1000	0.9440
0.9461	3.3837	1100	0.9427
0.9217	3.6907	1200	0.9376
0.9406	3.9977	1300	0.9360
0.893	4.3070	1400	0.9337
0.9049	4.6140	1500	0.9311
0.9024	4.9210	1600	0.9288
0.9278	5.2302	1700	0.9276
0.95	5.5372	1800	0.9255
0.9091	5.8442	1900	0.9245
0.8973	6.1535	2000	0.9232
0.8571	6.4605	2100	0.9224
0.8963	6.7675	2200	0.9220
0.867	7.0767	2300	0.9206
0.8623	7.3837	2400	0.9203
0.889	7.6907	2500	0.9184
0.8976	7.9977	2600	0.9181
0.8674	8.3070	2700	0.9183
0.8154	8.6140	2800	0.9189
0.9185	8.9210	2900	0.9169
0.8544	9.2302	3000	0.9173
0.8776	9.5372	3100	0.9170
0.8565	9.8442	3200	0.9169

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.6.0+cu126
• Datasets 3.5.0
• Tokenizers 0.21.1