Hugging Face's logo

tuanna08go
/
e295019c-1ea2-4aff-8d26-66e6c32eeaf7

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files
xet
 Community
tuanna08go commited on
Commit
912c0b7
·
verified ·
1 Parent(s): 8849435

End of training

Browse files
Files changed (2) hide show
  1. README.md +11 -4
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -64,7 +64,7 @@ lora_model_dir: null
64
  lora_r: 8
65
  lora_target_linear: true
66
  lr_scheduler: cosine
67
- max_steps: 1
68
  micro_batch_size: 2
69
  mlflow_experiment_name: /tmp/45d8ae8d3913acb4_train_data.json
70
  model_type: AutoModelForCausalLM
@@ -91,7 +91,7 @@ wandb_name: 9020b2ab-c097-4faa-a019-366ff9b2379c
91
  wandb_project: Gradients-On-Demand
92
  wandb_run: your_name
93
  wandb_runid: 9020b2ab-c097-4faa-a019-366ff9b2379c
94
- warmup_steps: 1
95
  weight_decay: 0.0
96
  xformers_attention: null
97
 
@@ -102,6 +102,8 @@ xformers_attention: null
102
  # e295019c-1ea2-4aff-8d26-66e6c32eeaf7
103
 
104
  This model is a fine-tuned version of [HuggingFaceH4/tiny-random-LlamaForCausalLM](https://huggingface.co/HuggingFaceH4/tiny-random-LlamaForCausalLM) on the None dataset.
 
 
105
 
106
  ## Model description
107
 
@@ -128,14 +130,19 @@ The following hyperparameters were used during training:
128
  - total_train_batch_size: 8
129
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
130
  - lr_scheduler_type: cosine
131
- - lr_scheduler_warmup_steps: 2
132
- - training_steps: 1
133
 
134
  ### Training results
135
 
136
  | Training Loss | Epoch | Step | Validation Loss |
137
  |:-------------:|:------:|:----:|:---------------:|
138
  | No log | 0.0036 | 1 | 10.3732 |
 
 
 
 
 
139
 
140
 
141
  ### Framework versions
 
64
  lora_r: 8
65
  lora_target_linear: true
66
  lr_scheduler: cosine
67
+ max_steps: 50
68
  micro_batch_size: 2
69
  mlflow_experiment_name: /tmp/45d8ae8d3913acb4_train_data.json
70
  model_type: AutoModelForCausalLM
 
91
  wandb_project: Gradients-On-Demand
92
  wandb_run: your_name
93
  wandb_runid: 9020b2ab-c097-4faa-a019-366ff9b2379c
94
+ warmup_steps: 10
95
  weight_decay: 0.0
96
  xformers_attention: null
97
 
 
102
  # e295019c-1ea2-4aff-8d26-66e6c32eeaf7
103
 
104
  This model is a fine-tuned version of [HuggingFaceH4/tiny-random-LlamaForCausalLM](https://huggingface.co/HuggingFaceH4/tiny-random-LlamaForCausalLM) on the None dataset.
105
+ It achieves the following results on the evaluation set:
106
+ - Loss: 10.3721
107
 
108
  ## Model description
109
 
 
130
  - total_train_batch_size: 8
131
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
132
  - lr_scheduler_type: cosine
133
+ - lr_scheduler_warmup_steps: 10
134
+ - training_steps: 50
135
 
136
  ### Training results
137
 
138
  | Training Loss | Epoch | Step | Validation Loss |
139
  |:-------------:|:------:|:----:|:---------------:|
140
  | No log | 0.0036 | 1 | 10.3732 |
141
+ | 10.375 | 0.0358 | 10 | 10.3730 |
142
+ | 10.3714 | 0.0717 | 20 | 10.3726 |
143
+ | 10.3751 | 0.1075 | 30 | 10.3723 |
144
+ | 10.3737 | 0.1434 | 40 | 10.3721 |
145
+ | 10.3776 | 0.1792 | 50 | 10.3721 |
146
 
147
 
148
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:573b694b410b231d14d71bff93b517cb0d416074662a78d824189f6078273f92
3
  size 33666
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b345b888f7c529f04d11e59f17d2cd5c22afe041b14093aea8b781eb2f18bb7c
3
  size 33666