Hugging Face's logo

dante123
/
AutoRE

TensorBoard
Safetensors
Model card Files
xet
 Metrics Community
1
dante123 commited on
Commit
510347d
·
verified ·
1 Parent(s): e237099

Upload 27 files

Browse files
Files changed (27) hide show
  1. fact/README.md +120 -0
  2. fact/adapter_config.json +26 -0
  3. fact/all_results.json +11 -0
  4. fact/checkpoint-4430/README.md +218 -0
  5. fact/checkpoint-4430/adapter_config.json +26 -0
  6. fact/checkpoint-4430/optimizer.pt +3 -0
  7. fact/checkpoint-4430/rng_state.pth +3 -0
  8. fact/checkpoint-4430/scheduler.pt +3 -0
  9. fact/checkpoint-4430/special_tokens_map.json +30 -0
  10. fact/checkpoint-4430/tokenizer.model +3 -0
  11. fact/checkpoint-4430/tokenizer_config.json +43 -0
  12. fact/checkpoint-4430/trainer_state.json +3031 -0
  13. fact/checkpoint-4430/training_args.bin +3 -0
  14. fact/eval_results.json +7 -0
  15. fact/runs/Jan27_07-40-28_platform/events.out.tfevents.1706341483.platform +3 -0
  16. fact/runs/Jan27_13-44-49_platform/events.out.tfevents.1706363103.platform +3 -0
  17. fact/runs/Jan29_08-27-56_platform/events.out.tfevents.1706516887.platform +3 -0
  18. fact/runs/Jan29_08-27-56_platform/events.out.tfevents.1706517427.platform +3 -0
  19. fact/special_tokens_map.json +30 -0
  20. fact/tokenizer.model +3 -0
  21. fact/tokenizer_config.json +43 -0
  22. fact/train_results.json +7 -0
  23. fact/trainer_log.jsonl +0 -0
  24. fact/trainer_state.json +3040 -0
  25. fact/training_args.bin +3 -0
  26. fact/training_eval_loss.png +0 -0
  27. fact/training_loss.png +0 -0
fact/README.md ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: peft
4
+ tags:
5
+ - llama-factory
6
+ - lora
7
+ - generated_from_trainer
8
+ base_model: /workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5
9
+ model-index:
10
+ - name: fact
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # fact
18
+
19
+ This model is a fine-tuned version of [/workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5](https://huggingface.co//workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5) on the vicuna_fact_test dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.0626
22
+
23
+ ## Model description
24
+
25
+ More information needed
26
+
27
+ ## Intended uses & limitations
28
+
29
+ More information needed
30
+
31
+ ## Training and evaluation data
32
+
33
+ More information needed
34
+
35
+ ## Training procedure
36
+
37
+
38
+ The following `bitsandbytes` quantization config was used during training:
39
+ - quant_method: bitsandbytes
40
+ - load_in_8bit: False
41
+ - load_in_4bit: True
42
+ - llm_int8_threshold: 6.0
43
+ - llm_int8_skip_modules: None
44
+ - llm_int8_enable_fp32_cpu_offload: False
45
+ - llm_int8_has_fp16_weight: False
46
+ - bnb_4bit_quant_type: nf4
47
+ - bnb_4bit_use_double_quant: True
48
+ - bnb_4bit_compute_dtype: float16
49
+
50
+ ### Training hyperparameters
51
+
52
+ The following hyperparameters were used during training:
53
+ - learning_rate: 5e-05
54
+ - train_batch_size: 8
55
+ - eval_batch_size: 8
56
+ - seed: 42
57
+ - gradient_accumulation_steps: 4
58
+ - total_train_batch_size: 32
59
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
60
+ - lr_scheduler_type: cosine
61
+ - training_steps: 4430
62
+ - mixed_precision_training: Native AMP
63
+
64
+ ### Training results
65
+
66
+ | Training Loss | Epoch | Step | Validation Loss |
67
+ |:-------------:|:-----:|:----:|:---------------:|
68
+ | 0.0941 | 0.02 | 100 | 0.0995 |
69
+ | 0.0903 | 0.04 | 200 | 0.0902 |
70
+ | 0.089 | 0.06 | 300 | 0.0802 |
71
+ | 0.0619 | 0.08 | 400 | 0.0762 |
72
+ | 0.0722 | 0.1 | 500 | 0.0752 |
73
+ | 0.0724 | 0.12 | 600 | 0.0736 |
74
+ | 0.0762 | 0.14 | 700 | 0.0710 |
75
+ | 0.0567 | 0.16 | 800 | 0.0712 |
76
+ | 0.0578 | 0.18 | 900 | 0.0679 |
77
+ | 0.0597 | 0.2 | 1000 | 0.0691 |
78
+ | 0.0538 | 0.22 | 1100 | 0.0660 |
79
+ | 0.0498 | 0.24 | 1200 | 0.0658 |
80
+ | 0.0595 | 0.26 | 1300 | 0.0642 |
81
+ | 0.0465 | 0.28 | 1400 | 0.0677 |
82
+ | 0.0533 | 0.3 | 1500 | 0.0651 |
83
+ | 0.0593 | 0.32 | 1600 | 0.0641 |
84
+ | 0.055 | 0.34 | 1700 | 0.0653 |
85
+ | 0.0546 | 0.36 | 1800 | 0.0634 |
86
+ | 0.0524 | 0.38 | 1900 | 0.0615 |
87
+ | 0.0432 | 0.4 | 2000 | 0.0632 |
88
+ | 0.0631 | 0.42 | 2100 | 0.0619 |
89
+ | 0.0519 | 0.44 | 2200 | 0.0598 |
90
+ | 0.0397 | 0.46 | 2300 | 0.0607 |
91
+ | 0.0467 | 0.48 | 2400 | 0.0616 |
92
+ | 0.049 | 0.5 | 2500 | 0.0636 |
93
+ | 0.0488 | 0.52 | 2600 | 0.0604 |
94
+ | 0.0449 | 0.54 | 2700 | 0.0598 |
95
+ | 0.0438 | 0.56 | 2800 | 0.0610 |
96
+ | 0.036 | 0.58 | 2900 | 0.0633 |
97
+ | 0.0464 | 0.6 | 3000 | 0.0603 |
98
+ | 0.0437 | 0.62 | 3100 | 0.0612 |
99
+ | 0.0389 | 0.64 | 3200 | 0.0605 |
100
+ | 0.0377 | 0.66 | 3300 | 0.0632 |
101
+ | 0.0382 | 0.68 | 3400 | 0.0626 |
102
+ | 0.051 | 0.7 | 3500 | 0.0612 |
103
+ | 0.047 | 0.72 | 3600 | 0.0633 |
104
+ | 0.0451 | 0.74 | 3700 | 0.0608 |
105
+ | 0.0472 | 0.77 | 3800 | 0.0630 |
106
+ | 0.0394 | 0.79 | 3900 | 0.0612 |
107
+ | 0.0471 | 0.81 | 4000 | 0.0614 |
108
+ | 0.031 | 0.83 | 4100 | 0.0602 |
109
+ | 0.0405 | 0.85 | 4200 | 0.0584 |
110
+ | 0.0421 | 0.87 | 4300 | 0.0591 |
111
+ | 0.0454 | 0.89 | 4400 | 0.0593 |
112
+
113
+
114
+ ### Framework versions
115
+
116
+ - PEFT 0.7.0
117
+ - Transformers 4.37.1
118
+ - Pytorch 2.1.2+cu121
119
+ - Datasets 2.16.1
120
+ - Tokenizers 0.15.1
fact/adapter_config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "/workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "loftq_config": {},
12
+ "lora_alpha": 16,
13
+ "lora_dropout": 0.05,
14
+ "megatron_config": null,
15
+ "megatron_core": "megatron.core",
16
+ "modules_to_save": null,
17
+ "peft_type": "LORA",
18
+ "r": 100,
19
+ "rank_pattern": {},
20
+ "revision": null,
21
+ "target_modules": [
22
+ "q_proj",
23
+ "v_proj"
24
+ ],
25
+ "task_type": "CAUSAL_LM"
26
+ }
fact/all_results.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.89,
3
+ "eval_loss": 0.06257433444261551,
4
+ "eval_runtime": 534.6938,
5
+ "eval_samples_per_second": 13.761,
6
+ "eval_steps_per_second": 1.721,
7
+ "train_loss": 9.867495303766314e-06,
8
+ "train_runtime": 4.4906,
9
+ "train_samples_per_second": 15784.097,
10
+ "train_steps_per_second": 986.506
11
+ }
fact/checkpoint-4430/README.md ADDED
@@ -0,0 +1,218 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: /workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+
201
+
202
+ ## Training procedure
203
+
204
+ The following `bitsandbytes` quantization config was used during training:
205
+ - quant_method: bitsandbytes
206
+ - load_in_8bit: False
207
+ - load_in_4bit: True
208
+ - llm_int8_threshold: 6.0
209
+ - llm_int8_skip_modules: None
210
+ - llm_int8_enable_fp32_cpu_offload: False
211
+ - llm_int8_has_fp16_weight: False
212
+ - bnb_4bit_quant_type: nf4
213
+ - bnb_4bit_use_double_quant: True
214
+ - bnb_4bit_compute_dtype: float16
215
+
216
+ ### Framework versions
217
+
218
+ - PEFT 0.7.0
fact/checkpoint-4430/adapter_config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "/workspace/xll/checkpoints/lmsys/vicuna-7b-v1.5",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "loftq_config": {},
12
+ "lora_alpha": 16,
13
+ "lora_dropout": 0.05,
14
+ "megatron_config": null,
15
+ "megatron_core": "megatron.core",
16
+ "modules_to_save": null,
17
+ "peft_type": "LORA",
18
+ "r": 100,
19
+ "rank_pattern": {},
20
+ "revision": null,
21
+ "target_modules": [
22
+ "q_proj",
23
+ "v_proj"
24
+ ],
25
+ "task_type": "CAUSAL_LM"
26
+ }
fact/checkpoint-4430/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b567fa0984278a5826e490ddea326b8373da0cc2459207eb45c345624f7f93b1
3
+ size 195035136
fact/checkpoint-4430/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c08a95343b57a49dd1365ede6aa665844d01f76bcb761827448af7e34857ecb
3
+ size 14244
fact/checkpoint-4430/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20eb3c18d0a898759f8d5d0bfb534151842a0d9f66662ab353a45356b13d8e2b
3
+ size 1064
fact/checkpoint-4430/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
fact/checkpoint-4430/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
fact/checkpoint-4430/tokenizer_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ }
29
+ },
30
+ "bos_token": "<s>",
31
+ "clean_up_tokenization_spaces": false,
32
+ "eos_token": "</s>",
33
+ "legacy": false,
34
+ "model_max_length": 4096,
35
+ "pad_token": "<unk>",
36
+ "padding_side": "right",
37
+ "sp_model_kwargs": {},
38
+ "spaces_between_special_tokens": false,
39
+ "split_special_tokens": false,
40
+ "tokenizer_class": "LlamaTokenizer",
41
+ "unk_token": "<unk>",
42
+ "use_default_system_prompt": false
43
+ }
fact/checkpoint-4430/trainer_state.json ADDED
@@ -0,0 +1,3031 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.891886450573787,
5
+ "eval_steps": 100,
6
+ "global_step": 4430,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 0.0001999999444378697,
14
+ "loss": 0.7972,
15
+ "step": 10
16
+ },
17
+ {
18
+ "epoch": 0.0,
19
+ "learning_rate": 0.0001999997777515405,
20
+ "loss": 0.221,
21
+ "step": 20
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 0.0001999994999411976,
26
+ "loss": 0.1313,
27
+ "step": 30
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 0.00019999911100714978,
32
+ "loss": 0.1169,
33
+ "step": 40
34
+ },
35
+ {
36
+ "epoch": 0.01,
37
+ "learning_rate": 0.00019999861094982923,
38
+ "loss": 0.1353,
39
+ "step": 50
40
+ },
41
+ {
42
+ "epoch": 0.01,
43
+ "learning_rate": 0.00019999799976979157,
44
+ "loss": 0.1355,
45
+ "step": 60
46
+ },
47
+ {
48
+ "epoch": 0.01,
49
+ "learning_rate": 0.00019999727746771603,
50
+ "loss": 0.0991,
51
+ "step": 70
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00019999644404440528,
56
+ "loss": 0.0995,
57
+ "step": 80
58
+ },
59
+ {
60
+ "epoch": 0.02,
61
+ "learning_rate": 0.00019999549950078536,
62
+ "loss": 0.1175,
63
+ "step": 90
64
+ },
65
+ {
66
+ "epoch": 0.02,
67
+ "learning_rate": 0.000199994443837906,
68
+ "loss": 0.0941,
69
+ "step": 100
70
+ },
71
+ {
72
+ "epoch": 0.02,
73
+ "eval_loss": 0.09952569007873535,
74
+ "eval_runtime": 659.9109,
75
+ "eval_samples_per_second": 11.15,
76
+ "eval_steps_per_second": 2.788,
77
+ "step": 100
78
+ },
79
+ {
80
+ "epoch": 0.02,
81
+ "learning_rate": 0.00019999327705694028,
82
+ "loss": 0.09,
83
+ "step": 110
84
+ },
85
+ {
86
+ "epoch": 0.02,
87
+ "learning_rate": 0.00019999199915918467,
88
+ "loss": 0.1044,
89
+ "step": 120
90
+ },
91
+ {
92
+ "epoch": 0.03,
93
+ "learning_rate": 0.00019999061014605934,
94
+ "loss": 0.0978,
95
+ "step": 130
96
+ },
97
+ {
98
+ "epoch": 0.03,
99
+ "learning_rate": 0.00019998911001910778,
100
+ "loss": 0.0963,
101
+ "step": 140
102
+ },
103
+ {
104
+ "epoch": 0.03,
105
+ "learning_rate": 0.000199987498779997,
106
+ "loss": 0.0955,
107
+ "step": 150
108
+ },
109
+ {
110
+ "epoch": 0.03,
111
+ "learning_rate": 0.00019998577643051744,
112
+ "loss": 0.0994,
113
+ "step": 160
114
+ },
115
+ {
116
+ "epoch": 0.03,
117
+ "learning_rate": 0.00019998394297258311,
118
+ "loss": 0.0921,
119
+ "step": 170
120
+ },
121
+ {
122
+ "epoch": 0.04,
123
+ "learning_rate": 0.0001999819984082314,
124
+ "loss": 0.087,
125
+ "step": 180
126
+ },
127
+ {
128
+ "epoch": 0.04,
129
+ "learning_rate": 0.00019997994273962316,
130
+ "loss": 0.0826,
131
+ "step": 190
132
+ },
133
+ {
134
+ "epoch": 0.04,
135
+ "learning_rate": 0.00019997777596904282,
136
+ "loss": 0.0903,
137
+ "step": 200
138
+ },
139
+ {
140
+ "epoch": 0.04,
141
+ "eval_loss": 0.090216726064682,
142
+ "eval_runtime": 649.064,
143
+ "eval_samples_per_second": 11.336,
144
+ "eval_steps_per_second": 2.835,
145
+ "step": 200
146
+ },
147
+ {
148
+ "epoch": 0.04,
149
+ "learning_rate": 0.00019997549809889811,
150
+ "loss": 0.0949,
151
+ "step": 210
152
+ },
153
+ {
154
+ "epoch": 0.04,
155
+ "learning_rate": 0.00019997310913172028,
156
+ "loss": 0.0902,
157
+ "step": 220
158
+ },
159
+ {
160
+ "epoch": 0.05,
161
+ "learning_rate": 0.00019997060907016415,
162
+ "loss": 0.0792,
163
+ "step": 230
164
+ },
165
+ {
166
+ "epoch": 0.05,
167
+ "learning_rate": 0.0001999679979170078,
168
+ "loss": 0.0901,
169
+ "step": 240
170
+ },
171
+ {
172
+ "epoch": 0.05,
173
+ "learning_rate": 0.0001999652756751529,
174
+ "loss": 0.1035,
175
+ "step": 250
176
+ },
177
+ {
178
+ "epoch": 0.05,
179
+ "learning_rate": 0.00019996244234762452,
180
+ "loss": 0.0875,
181
+ "step": 260
182
+ },
183
+ {
184
+ "epoch": 0.05,
185
+ "learning_rate": 0.00019995949793757118,
186
+ "loss": 0.0718,
187
+ "step": 270
188
+ },
189
+ {
190
+ "epoch": 0.06,
191
+ "learning_rate": 0.00019995644244826481,
192
+ "loss": 0.0799,
193
+ "step": 280
194
+ },
195
+ {
196
+ "epoch": 0.06,
197
+ "learning_rate": 0.0001999532758831008,
198
+ "loss": 0.0767,
199
+ "step": 290
200
+ },
201
+ {
202
+ "epoch": 0.06,
203
+ "learning_rate": 0.000199949998245598,
204
+ "loss": 0.089,
205
+ "step": 300
206
+ },
207
+ {
208
+ "epoch": 0.06,
209
+ "eval_loss": 0.08015166223049164,
210
+ "eval_runtime": 646.37,
211
+ "eval_samples_per_second": 11.384,
212
+ "eval_steps_per_second": 2.847,
213
+ "step": 300
214
+ },
215
+ {
216
+ "epoch": 0.06,
217
+ "learning_rate": 0.00019994660953939864,
218
+ "loss": 0.0833,
219
+ "step": 310
220
+ },
221
+ {
222
+ "epoch": 0.06,
223
+ "learning_rate": 0.00019994310976826837,
224
+ "loss": 0.0799,
225
+ "step": 320
226
+ },
227
+ {
228
+ "epoch": 0.07,
229
+ "learning_rate": 0.00019993949893609635,
230
+ "loss": 0.0747,
231
+ "step": 330
232
+ },
233
+ {
234
+ "epoch": 0.07,
235
+ "learning_rate": 0.00019993577704689505,
236
+ "loss": 0.0888,
237
+ "step": 340
238
+ },
239
+ {
240
+ "epoch": 0.07,
241
+ "learning_rate": 0.0001999319441048004,
242
+ "loss": 0.0872,
243
+ "step": 350
244
+ },
245
+ {
246
+ "epoch": 0.07,
247
+ "learning_rate": 0.00019992800011407172,
248
+ "loss": 0.0762,
249
+ "step": 360
250
+ },
251
+ {
252
+ "epoch": 0.07,
253
+ "learning_rate": 0.00019992394507909174,
254
+ "loss": 0.0746,
255
+ "step": 370
256
+ },
257
+ {
258
+ "epoch": 0.08,
259
+ "learning_rate": 0.00019991977900436658,
260
+ "loss": 0.0713,
261
+ "step": 380
262
+ },
263
+ {
264
+ "epoch": 0.08,
265
+ "learning_rate": 0.0001999155018945258,
266
+ "loss": 0.077,
267
+ "step": 390
268
+ },
269
+ {
270
+ "epoch": 0.08,
271
+ "learning_rate": 0.00019991111375432225,
272
+ "loss": 0.0619,
273
+ "step": 400
274
+ },
275
+ {
276
+ "epoch": 0.08,
277
+ "eval_loss": 0.07623383402824402,
278
+ "eval_runtime": 654.0144,
279
+ "eval_samples_per_second": 11.251,
280
+ "eval_steps_per_second": 2.813,
281
+ "step": 400
282
+ },
283
+ {
284
+ "epoch": 0.08,
285
+ "learning_rate": 0.00019990661458863226,
286
+ "loss": 0.0702,
287
+ "step": 410
288
+ },
289
+ {
290
+ "epoch": 0.08,
291
+ "learning_rate": 0.0001999020044024555,
292
+ "loss": 0.0725,
293
+ "step": 420
294
+ },
295
+ {
296
+ "epoch": 0.09,
297
+ "learning_rate": 0.00019989728320091496,
298
+ "loss": 0.065,
299
+ "step": 430
300
+ },
301
+ {
302
+ "epoch": 0.09,
303
+ "learning_rate": 0.00019989245098925708,
304
+ "loss": 0.0632,
305
+ "step": 440
306
+ },
307
+ {
308
+ "epoch": 0.09,
309
+ "learning_rate": 0.00019988750777285162,
310
+ "loss": 0.0762,
311
+ "step": 450
312
+ },
313
+ {
314
+ "epoch": 0.09,
315
+ "learning_rate": 0.00019988245355719166,
316
+ "loss": 0.0736,
317
+ "step": 460
318
+ },
319
+ {
320
+ "epoch": 0.09,
321
+ "learning_rate": 0.0001998772883478937,
322
+ "loss": 0.0662,
323
+ "step": 470
324
+ },
325
+ {
326
+ "epoch": 0.1,
327
+ "learning_rate": 0.0001998720121506975,
328
+ "loss": 0.0765,
329
+ "step": 480
330
+ },
331
+ {
332
+ "epoch": 0.1,
333
+ "learning_rate": 0.00019986662497146622,
334
+ "loss": 0.0694,
335
+ "step": 490
336
+ },
337
+ {
338
+ "epoch": 0.1,
339
+ "learning_rate": 0.00019986112681618634,
340
+ "loss": 0.0722,
341
+ "step": 500
342
+ },
343
+ {
344
+ "epoch": 0.1,
345
+ "eval_loss": 0.075162373483181,
346
+ "eval_runtime": 656.2929,
347
+ "eval_samples_per_second": 11.211,
348
+ "eval_steps_per_second": 2.804,
349
+ "step": 500
350
+ },
351
+ {
352
+ "epoch": 0.1,
353
+ "learning_rate": 0.0001998555176909676,
354
+ "loss": 0.073,
355
+ "step": 510
356
+ },
357
+ {
358
+ "epoch": 0.1,
359
+ "learning_rate": 0.00019984979760204313,
360
+ "loss": 0.0681,
361
+ "step": 520
362
+ },
363
+ {
364
+ "epoch": 0.11,
365
+ "learning_rate": 0.00019984396655576932,
366
+ "loss": 0.057,
367
+ "step": 530
368
+ },
369
+ {
370
+ "epoch": 0.11,
371
+ "learning_rate": 0.0001998380245586259,
372
+ "loss": 0.0689,
373
+ "step": 540
374
+ },
375
+ {
376
+ "epoch": 0.11,
377
+ "learning_rate": 0.00019983197161721583,
378
+ "loss": 0.057,
379
+ "step": 550
380
+ },
381
+ {
382
+ "epoch": 0.11,
383
+ "learning_rate": 0.00019982580773826545,
384
+ "loss": 0.0693,
385
+ "step": 560
386
+ },
387
+ {
388
+ "epoch": 0.11,
389
+ "learning_rate": 0.00019981953292862428,
390
+ "loss": 0.0702,
391
+ "step": 570
392
+ },
393
+ {
394
+ "epoch": 0.12,
395
+ "learning_rate": 0.0001998131471952652,
396
+ "loss": 0.0681,
397
+ "step": 580
398
+ },
399
+ {
400
+ "epoch": 0.12,
401
+ "learning_rate": 0.00019980665054528425,
402
+ "loss": 0.0657,
403
+ "step": 590
404
+ },
405
+ {
406
+ "epoch": 0.12,
407
+ "learning_rate": 0.00019980004298590081,
408
+ "loss": 0.0724,
409
+ "step": 600
410
+ },
411
+ {
412
+ "epoch": 0.12,
413
+ "eval_loss": 0.0736309364438057,
414
+ "eval_runtime": 657.9967,
415
+ "eval_samples_per_second": 11.182,
416
+ "eval_steps_per_second": 2.796,
417
+ "step": 600
418
+ },
419
+ {
420
+ "epoch": 0.12,
421
+ "learning_rate": 0.00019979332452445752,
422
+ "loss": 0.0599,
423
+ "step": 610
424
+ },
425
+ {
426
+ "epoch": 0.12,
427
+ "learning_rate": 0.00019978649516842016,
428
+ "loss": 0.0697,
429
+ "step": 620
430
+ },
431
+ {
432
+ "epoch": 0.13,
433
+ "learning_rate": 0.00019977955492537787,
434
+ "loss": 0.0715,
435
+ "step": 630
436
+ },
437
+ {
438
+ "epoch": 0.13,
439
+ "learning_rate": 0.00019977250380304287,
440
+ "loss": 0.0589,
441
+ "step": 640
442
+ },
443
+ {
444
+ "epoch": 0.13,
445
+ "learning_rate": 0.0001997653418092507,
446
+ "loss": 0.0722,
447
+ "step": 650
448
+ },
449
+ {
450
+ "epoch": 0.13,
451
+ "learning_rate": 0.00019975806895196008,
452
+ "loss": 0.0611,
453
+ "step": 660
454
+ },
455
+ {
456
+ "epoch": 0.13,
457
+ "learning_rate": 0.0001997506852392529,
458
+ "loss": 0.056,
459
+ "step": 670
460
+ },
461
+ {
462
+ "epoch": 0.14,
463
+ "learning_rate": 0.0001997431906793343,
464
+ "loss": 0.0614,
465
+ "step": 680
466
+ },
467
+ {
468
+ "epoch": 0.14,
469
+ "learning_rate": 0.00019973558528053252,
470
+ "loss": 0.0537,
471
+ "step": 690
472
+ },
473
+ {
474
+ "epoch": 0.14,
475
+ "learning_rate": 0.000199727869051299,
476
+ "loss": 0.0762,
477
+ "step": 700
478
+ },
479
+ {
480
+ "epoch": 0.14,
481
+ "eval_loss": 0.07101555913686752,
482
+ "eval_runtime": 665.8897,
483
+ "eval_samples_per_second": 11.05,
484
+ "eval_steps_per_second": 2.763,
485
+ "step": 700
486
+ },
487
+ {
488
+ "epoch": 0.14,
489
+ "learning_rate": 0.00019972004200020832,
490
+ "loss": 0.0634,
491
+ "step": 710
492
+ },
493
+ {
494
+ "epoch": 0.14,
495
+ "learning_rate": 0.0001997121041359583,
496
+ "loss": 0.0697,
497
+ "step": 720
498
+ },
499
+ {
500
+ "epoch": 0.15,
501
+ "learning_rate": 0.00019970405546736978,
502
+ "loss": 0.0605,
503
+ "step": 730
504
+ },
505
+ {
506
+ "epoch": 0.15,
507
+ "learning_rate": 0.00019969589600338678,
508
+ "loss": 0.0598,
509
+ "step": 740
510
+ },
511
+ {
512
+ "epoch": 0.15,
513
+ "learning_rate": 0.00019968762575307649,
514
+ "loss": 0.066,
515
+ "step": 750
516
+ },
517
+ {
518
+ "epoch": 0.15,
519
+ "learning_rate": 0.00019967924472562914,
520
+ "loss": 0.0656,
521
+ "step": 760
522
+ },
523
+ {
524
+ "epoch": 0.16,
525
+ "learning_rate": 0.00019967075293035805,
526
+ "loss": 0.0614,
527
+ "step": 770
528
+ },
529
+ {
530
+ "epoch": 0.16,
531
+ "learning_rate": 0.0001996621503766997,
532
+ "loss": 0.0633,
533
+ "step": 780
534
+ },
535
+ {
536
+ "epoch": 0.16,
537
+ "learning_rate": 0.00019965343707421362,
538
+ "loss": 0.0661,
539
+ "step": 790
540
+ },
541
+ {
542
+ "epoch": 0.16,
543
+ "learning_rate": 0.00019964461303258242,
544
+ "loss": 0.0567,
545
+ "step": 800
546
+ },
547
+ {
548
+ "epoch": 0.16,
549
+ "eval_loss": 0.07119747996330261,
550
+ "eval_runtime": 657.3478,
551
+ "eval_samples_per_second": 11.193,
552
+ "eval_steps_per_second": 2.799,
553
+ "step": 800
554
+ },
555
+ {
556
+ "epoch": 0.16,
557
+ "learning_rate": 0.00019963567826161168,
558
+ "loss": 0.0577,
559
+ "step": 810
560
+ },
561
+ {
562
+ "epoch": 0.17,
563
+ "learning_rate": 0.00019962663277123016,
564
+ "loss": 0.0633,
565
+ "step": 820
566
+ },
567
+ {
568
+ "epoch": 0.17,
569
+ "learning_rate": 0.0001996174765714896,
570
+ "loss": 0.0701,
571
+ "step": 830
572
+ },
573
+ {
574
+ "epoch": 0.17,
575
+ "learning_rate": 0.0001996082096725647,
576
+ "loss": 0.0715,
577
+ "step": 840
578
+ },
579
+ {
580
+ "epoch": 0.17,
581
+ "learning_rate": 0.0001995988320847533,
582
+ "loss": 0.0618,
583
+ "step": 850
584
+ },
585
+ {
586
+ "epoch": 0.17,
587
+ "learning_rate": 0.00019958934381847612,
588
+ "loss": 0.0589,
589
+ "step": 860
590
+ },
591
+ {
592
+ "epoch": 0.18,
593
+ "learning_rate": 0.00019957974488427698,
594
+ "loss": 0.0658,
595
+ "step": 870
596
+ },
597
+ {
598
+ "epoch": 0.18,
599
+ "learning_rate": 0.00019957003529282255,
600
+ "loss": 0.0549,
601
+ "step": 880
602
+ },
603
+ {
604
+ "epoch": 0.18,
605
+ "learning_rate": 0.00019956021505490262,
606
+ "loss": 0.0664,
607
+ "step": 890
608
+ },
609
+ {
610
+ "epoch": 0.18,
611
+ "learning_rate": 0.0001995502841814298,
612
+ "loss": 0.0578,
613
+ "step": 900
614
+ },
615
+ {
616
+ "epoch": 0.18,
617
+ "eval_loss": 0.06790292263031006,
618
+ "eval_runtime": 663.3404,
619
+ "eval_samples_per_second": 11.092,
620
+ "eval_steps_per_second": 2.774,
621
+ "step": 900
622
+ },
623
+ {
624
+ "epoch": 0.18,
625
+ "learning_rate": 0.00019954024268343975,
626
+ "loss": 0.0626,
627
+ "step": 910
628
+ },
629
+ {
630
+ "epoch": 0.19,
631
+ "learning_rate": 0.000199530090572091,
632
+ "loss": 0.0621,
633
+ "step": 920
634
+ },
635
+ {
636
+ "epoch": 0.19,
637
+ "learning_rate": 0.00019951982785866492,
638
+ "loss": 0.0527,
639
+ "step": 930
640
+ },
641
+ {
642
+ "epoch": 0.19,
643
+ "learning_rate": 0.00019950945455456597,
644
+ "loss": 0.0636,
645
+ "step": 940
646
+ },
647
+ {
648
+ "epoch": 0.19,
649
+ "learning_rate": 0.00019949897067132142,
650
+ "loss": 0.0767,
651
+ "step": 950
652
+ },
653
+ {
654
+ "epoch": 0.19,
655
+ "learning_rate": 0.00019948837622058135,
656
+ "loss": 0.0785,
657
+ "step": 960
658
+ },
659
+ {
660
+ "epoch": 0.2,
661
+ "learning_rate": 0.00019947767121411878,
662
+ "loss": 0.0698,
663
+ "step": 970
664
+ },
665
+ {
666
+ "epoch": 0.2,
667
+ "learning_rate": 0.0001994668556638296,
668
+ "loss": 0.0576,
669
+ "step": 980
670
+ },
671
+ {
672
+ "epoch": 0.2,
673
+ "learning_rate": 0.00019945592958173247,
674
+ "loss": 0.0616,
675
+ "step": 990
676
+ },
677
+ {
678
+ "epoch": 0.2,
679
+ "learning_rate": 0.00019944489297996892,
680
+ "loss": 0.0597,
681
+ "step": 1000
682
+ },
683
+ {
684
+ "epoch": 0.2,
685
+ "eval_loss": 0.06911394000053406,
686
+ "eval_runtime": 654.3952,
687
+ "eval_samples_per_second": 11.244,
688
+ "eval_steps_per_second": 2.812,
689
+ "step": 1000
690
+ },
691
+ {
692
+ "epoch": 0.2,
693
+ "learning_rate": 0.00019943374587080333,
694
+ "loss": 0.0674,
695
+ "step": 1010
696
+ },
697
+ {
698
+ "epoch": 0.21,
699
+ "learning_rate": 0.0001994224882666228,
700
+ "loss": 0.0528,
701
+ "step": 1020
702
+ },
703
+ {
704
+ "epoch": 0.21,
705
+ "learning_rate": 0.0001994111201799373,
706
+ "loss": 0.0602,
707
+ "step": 1030
708
+ },
709
+ {
710
+ "epoch": 0.21,
711
+ "learning_rate": 0.00019939964162337946,
712
+ "loss": 0.0568,
713
+ "step": 1040
714
+ },
715
+ {
716
+ "epoch": 0.21,
717
+ "learning_rate": 0.00019938805260970486,
718
+ "loss": 0.0571,
719
+ "step": 1050
720
+ },
721
+ {
722
+ "epoch": 0.21,
723
+ "learning_rate": 0.0001993763531517916,
724
+ "loss": 0.0598,
725
+ "step": 1060
726
+ },
727
+ {
728
+ "epoch": 0.22,
729
+ "learning_rate": 0.00019936454326264068,
730
+ "loss": 0.0636,
731
+ "step": 1070
732
+ },
733
+ {
734
+ "epoch": 0.22,
735
+ "learning_rate": 0.00019935262295537568,
736
+ "loss": 0.0602,
737
+ "step": 1080
738
+ },
739
+ {
740
+ "epoch": 0.22,
741
+ "learning_rate": 0.00019934059224324303,
742
+ "loss": 0.0763,
743
+ "step": 1090
744
+ },
745
+ {
746
+ "epoch": 0.22,
747
+ "learning_rate": 0.00019932845113961172,
748
+ "loss": 0.0538,
749
+ "step": 1100
750
+ },
751
+ {
752
+ "epoch": 0.22,
753
+ "eval_loss": 0.06597457081079483,
754
+ "eval_runtime": 653.2731,
755
+ "eval_samples_per_second": 11.263,
756
+ "eval_steps_per_second": 2.817,
757
+ "step": 1100
758
+ },
759
+ {
760
+ "epoch": 0.22,
761
+ "learning_rate": 0.0001993161996579735,
762
+ "loss": 0.0578,
763
+ "step": 1110
764
+ },
765
+ {
766
+ "epoch": 0.23,
767
+ "learning_rate": 0.00019930383781194272,
768
+ "loss": 0.0561,
769
+ "step": 1120
770
+ },
771
+ {
772
+ "epoch": 0.23,
773
+ "learning_rate": 0.00019929136561525637,
774
+ "loss": 0.0558,
775
+ "step": 1130
776
+ },
777
+ {
778
+ "epoch": 0.23,
779
+ "learning_rate": 0.0001992787830817741,
780
+ "loss": 0.0669,
781
+ "step": 1140
782
+ },
783
+ {
784
+ "epoch": 0.23,
785
+ "learning_rate": 0.00019926609022547818,
786
+ "loss": 0.0616,
787
+ "step": 1150
788
+ },
789
+ {
790
+ "epoch": 0.23,
791
+ "learning_rate": 0.00019925328706047344,
792
+ "loss": 0.0637,
793
+ "step": 1160
794
+ },
795
+ {
796
+ "epoch": 0.24,
797
+ "learning_rate": 0.00019924037360098727,
798
+ "loss": 0.052,
799
+ "step": 1170
800
+ },
801
+ {
802
+ "epoch": 0.24,
803
+ "learning_rate": 0.0001992273498613697,
804
+ "loss": 0.0548,
805
+ "step": 1180
806
+ },
807
+ {
808
+ "epoch": 0.24,
809
+ "learning_rate": 0.00019921421585609325,
810
+ "loss": 0.0545,
811
+ "step": 1190
812
+ },
813
+ {
814
+ "epoch": 0.24,
815
+ "learning_rate": 0.000199200971599753,
816
+ "loss": 0.0498,
817
+ "step": 1200
818
+ },
819
+ {
820
+ "epoch": 0.24,
821
+ "eval_loss": 0.06580409407615662,
822
+ "eval_runtime": 655.2658,
823
+ "eval_samples_per_second": 11.229,
824
+ "eval_steps_per_second": 2.808,
825
+ "step": 1200
826
+ },
827
+ {
828
+ "epoch": 0.24,
829
+ "learning_rate": 0.00019918761710706649,
830
+ "loss": 0.0537,
831
+ "step": 1210
832
+ },
833
+ {
834
+ "epoch": 0.25,
835
+ "learning_rate": 0.00019917415239287386,
836
+ "loss": 0.0579,
837
+ "step": 1220
838
+ },
839
+ {
840
+ "epoch": 0.25,
841
+ "learning_rate": 0.00019916057747213758,
842
+ "loss": 0.053,
843
+ "step": 1230
844
+ },
845
+ {
846
+ "epoch": 0.25,
847
+ "learning_rate": 0.00019914689235994278,
848
+ "loss": 0.0562,
849
+ "step": 1240
850
+ },
851
+ {
852
+ "epoch": 0.25,
853
+ "learning_rate": 0.00019913309707149694,
854
+ "loss": 0.0595,
855
+ "step": 1250
856
+ },
857
+ {
858
+ "epoch": 0.25,
859
+ "learning_rate": 0.00019911919162212988,
860
+ "loss": 0.0515,
861
+ "step": 1260
862
+ },
863
+ {
864
+ "epoch": 0.26,
865
+ "learning_rate": 0.000199105176027294,
866
+ "loss": 0.053,
867
+ "step": 1270
868
+ },
869
+ {
870
+ "epoch": 0.26,
871
+ "learning_rate": 0.00019909105030256402,
872
+ "loss": 0.065,
873
+ "step": 1280
874
+ },
875
+ {
876
+ "epoch": 0.26,
877
+ "learning_rate": 0.000199076814463637,
878
+ "loss": 0.0574,
879
+ "step": 1290
880
+ },
881
+ {
882
+ "epoch": 0.26,
883
+ "learning_rate": 0.0001990624685263325,
884
+ "loss": 0.0595,
885
+ "step": 1300
886
+ },
887
+ {
888
+ "epoch": 0.26,
889
+ "eval_loss": 0.06419895589351654,
890
+ "eval_runtime": 662.3031,
891
+ "eval_samples_per_second": 11.11,
892
+ "eval_steps_per_second": 2.778,
893
+ "step": 1300
894
+ },
895
+ {
896
+ "epoch": 0.26,
897
+ "learning_rate": 0.00019904801250659223,
898
+ "loss": 0.0667,
899
+ "step": 1310
900
+ },
901
+ {
902
+ "epoch": 0.27,
903
+ "learning_rate": 0.00019903344642048043,
904
+ "loss": 0.0562,
905
+ "step": 1320
906
+ },
907
+ {
908
+ "epoch": 0.27,
909
+ "learning_rate": 0.00019901877028418347,
910
+ "loss": 0.0604,
911
+ "step": 1330
912
+ },
913
+ {
914
+ "epoch": 0.27,
915
+ "learning_rate": 0.00019900398411401015,
916
+ "loss": 0.0474,
917
+ "step": 1340
918
+ },
919
+ {
920
+ "epoch": 0.27,
921
+ "learning_rate": 0.0001989890879263915,
922
+ "loss": 0.0444,
923
+ "step": 1350
924
+ },
925
+ {
926
+ "epoch": 0.27,
927
+ "learning_rate": 0.0001989740817378808,
928
+ "loss": 0.0522,
929
+ "step": 1360
930
+ },
931
+ {
932
+ "epoch": 0.28,
933
+ "learning_rate": 0.00019895896556515352,
934
+ "loss": 0.0543,
935
+ "step": 1370
936
+ },
937
+ {
938
+ "epoch": 0.28,
939
+ "learning_rate": 0.00019894373942500743,
940
+ "loss": 0.0447,
941
+ "step": 1380
942
+ },
943
+ {
944
+ "epoch": 0.28,
945
+ "learning_rate": 0.00019892840333436245,
946
+ "loss": 0.0454,
947
+ "step": 1390
948
+ },
949
+ {
950
+ "epoch": 0.28,
951
+ "learning_rate": 0.00019891295731026075,
952
+ "loss": 0.0465,
953
+ "step": 1400
954
+ },
955
+ {
956
+ "epoch": 0.28,
957
+ "eval_loss": 0.0676579549908638,
958
+ "eval_runtime": 660.0804,
959
+ "eval_samples_per_second": 11.147,
960
+ "eval_steps_per_second": 2.788,
961
+ "step": 1400
962
+ },
963
+ {
964
+ "epoch": 0.28,
965
+ "learning_rate": 0.0001988974013698665,
966
+ "loss": 0.0533,
967
+ "step": 1410
968
+ },
969
+ {
970
+ "epoch": 0.29,
971
+ "learning_rate": 0.00019888173553046624,
972
+ "loss": 0.0522,
973
+ "step": 1420
974
+ },
975
+ {
976
+ "epoch": 0.29,
977
+ "learning_rate": 0.00019886595980946842,
978
+ "loss": 0.0592,
979
+ "step": 1430
980
+ },
981
+ {
982
+ "epoch": 0.29,
983
+ "learning_rate": 0.00019885007422440379,
984
+ "loss": 0.0584,
985
+ "step": 1440
986
+ },
987
+ {
988
+ "epoch": 0.29,
989
+ "learning_rate": 0.00019883407879292498,
990
+ "loss": 0.054,
991
+ "step": 1450
992
+ },
993
+ {
994
+ "epoch": 0.29,
995
+ "learning_rate": 0.00019881797353280695,
996
+ "loss": 0.0442,
997
+ "step": 1460
998
+ },
999
+ {
1000
+ "epoch": 0.3,
1001
+ "learning_rate": 0.00019880175846194638,
1002
+ "loss": 0.0659,
1003
+ "step": 1470
1004
+ },
1005
+ {
1006
+ "epoch": 0.3,
1007
+ "learning_rate": 0.00019878543359836223,
1008
+ "loss": 0.0483,
1009
+ "step": 1480
1010
+ },
1011
+ {
1012
+ "epoch": 0.3,
1013
+ "learning_rate": 0.0001987689989601954,
1014
+ "loss": 0.0525,
1015
+ "step": 1490
1016
+ },
1017
+ {
1018
+ "epoch": 0.3,
1019
+ "learning_rate": 0.00019875245456570873,
1020
+ "loss": 0.0533,
1021
+ "step": 1500
1022
+ },
1023
+ {
1024
+ "epoch": 0.3,
1025
+ "eval_loss": 0.06512827426195145,
1026
+ "eval_runtime": 659.6413,
1027
+ "eval_samples_per_second": 11.155,
1028
+ "eval_steps_per_second": 2.789,
1029
+ "step": 1500
1030
+ },
1031
+ {
1032
+ "epoch": 0.3,
1033
+ "learning_rate": 0.00019873580043328705,
1034
+ "loss": 0.0438,
1035
+ "step": 1510
1036
+ },
1037
+ {
1038
+ "epoch": 0.31,
1039
+ "learning_rate": 0.00019871903658143719,
1040
+ "loss": 0.0542,
1041
+ "step": 1520
1042
+ },
1043
+ {
1044
+ "epoch": 0.31,
1045
+ "learning_rate": 0.0001987021630287878,
1046
+ "loss": 0.0555,
1047
+ "step": 1530
1048
+ },
1049
+ {
1050
+ "epoch": 0.31,
1051
+ "learning_rate": 0.00019868517979408954,
1052
+ "loss": 0.0561,
1053
+ "step": 1540
1054
+ },
1055
+ {
1056
+ "epoch": 0.31,
1057
+ "learning_rate": 0.00019866808689621482,
1058
+ "loss": 0.0517,
1059
+ "step": 1550
1060
+ },
1061
+ {
1062
+ "epoch": 0.31,
1063
+ "learning_rate": 0.0001986508843541581,
1064
+ "loss": 0.051,
1065
+ "step": 1560
1066
+ },
1067
+ {
1068
+ "epoch": 0.32,
1069
+ "learning_rate": 0.0001986335721870355,
1070
+ "loss": 0.0525,
1071
+ "step": 1570
1072
+ },
1073
+ {
1074
+ "epoch": 0.32,
1075
+ "learning_rate": 0.0001986161504140851,
1076
+ "loss": 0.0443,
1077
+ "step": 1580
1078
+ },
1079
+ {
1080
+ "epoch": 0.32,
1081
+ "learning_rate": 0.00019859861905466668,
1082
+ "loss": 0.0542,
1083
+ "step": 1590
1084
+ },
1085
+ {
1086
+ "epoch": 0.32,
1087
+ "learning_rate": 0.00019858097812826183,
1088
+ "loss": 0.0593,
1089
+ "step": 1600
1090
+ },
1091
+ {
1092
+ "epoch": 0.32,
1093
+ "eval_loss": 0.06405352801084518,
1094
+ "eval_runtime": 658.5179,
1095
+ "eval_samples_per_second": 11.174,
1096
+ "eval_steps_per_second": 2.794,
1097
+ "step": 1600
1098
+ },
1099
+ {
1100
+ "epoch": 0.32,
1101
+ "learning_rate": 0.0001985632276544739,
1102
+ "loss": 0.0527,
1103
+ "step": 1610
1104
+ },
1105
+ {
1106
+ "epoch": 0.33,
1107
+ "learning_rate": 0.000198545367653028,
1108
+ "loss": 0.0422,
1109
+ "step": 1620
1110
+ },
1111
+ {
1112
+ "epoch": 0.33,
1113
+ "learning_rate": 0.00019852739814377087,
1114
+ "loss": 0.0533,
1115
+ "step": 1630
1116
+ },
1117
+ {
1118
+ "epoch": 0.33,
1119
+ "learning_rate": 0.00019850931914667107,
1120
+ "loss": 0.0507,
1121
+ "step": 1640
1122
+ },
1123
+ {
1124
+ "epoch": 0.33,
1125
+ "learning_rate": 0.00019849113068181869,
1126
+ "loss": 0.0581,
1127
+ "step": 1650
1128
+ },
1129
+ {
1130
+ "epoch": 0.33,
1131
+ "learning_rate": 0.00019847283276942554,
1132
+ "loss": 0.0583,
1133
+ "step": 1660
1134
+ },
1135
+ {
1136
+ "epoch": 0.34,
1137
+ "learning_rate": 0.00019845442542982506,
1138
+ "loss": 0.0562,
1139
+ "step": 1670
1140
+ },
1141
+ {
1142
+ "epoch": 0.34,
1143
+ "learning_rate": 0.00019843590868347225,
1144
+ "loss": 0.0499,
1145
+ "step": 1680
1146
+ },
1147
+ {
1148
+ "epoch": 0.34,
1149
+ "learning_rate": 0.00019841728255094374,
1150
+ "loss": 0.0543,
1151
+ "step": 1690
1152
+ },
1153
+ {
1154
+ "epoch": 0.34,
1155
+ "learning_rate": 0.00019839854705293764,
1156
+ "loss": 0.055,
1157
+ "step": 1700
1158
+ },
1159
+ {
1160
+ "epoch": 0.34,
1161
+ "eval_loss": 0.0653354823589325,
1162
+ "eval_runtime": 674.0253,
1163
+ "eval_samples_per_second": 10.917,
1164
+ "eval_steps_per_second": 2.73,
1165
+ "step": 1700
1166
+ },
1167
+ {
1168
+ "epoch": 0.34,
1169
+ "learning_rate": 0.00019837970221027365,
1170
+ "loss": 0.0422,
1171
+ "step": 1710
1172
+ },
1173
+ {
1174
+ "epoch": 0.35,
1175
+ "learning_rate": 0.00019836074804389296,
1176
+ "loss": 0.0586,
1177
+ "step": 1720
1178
+ },
1179
+ {
1180
+ "epoch": 0.35,
1181
+ "learning_rate": 0.00019834168457485824,
1182
+ "loss": 0.0408,
1183
+ "step": 1730
1184
+ },
1185
+ {
1186
+ "epoch": 0.35,
1187
+ "learning_rate": 0.00019832251182435367,
1188
+ "loss": 0.0587,
1189
+ "step": 1740
1190
+ },
1191
+ {
1192
+ "epoch": 0.35,
1193
+ "learning_rate": 0.00019830322981368478,
1194
+ "loss": 0.0498,
1195
+ "step": 1750
1196
+ },
1197
+ {
1198
+ "epoch": 0.35,
1199
+ "learning_rate": 0.0001982838385642786,
1200
+ "loss": 0.0475,
1201
+ "step": 1760
1202
+ },
1203
+ {
1204
+ "epoch": 0.36,
1205
+ "learning_rate": 0.00019826433809768345,
1206
+ "loss": 0.0571,
1207
+ "step": 1770
1208
+ },
1209
+ {
1210
+ "epoch": 0.36,
1211
+ "learning_rate": 0.00019824472843556914,
1212
+ "loss": 0.0512,
1213
+ "step": 1780
1214
+ },
1215
+ {
1216
+ "epoch": 0.36,
1217
+ "learning_rate": 0.00019822500959972673,
1218
+ "loss": 0.0477,
1219
+ "step": 1790
1220
+ },
1221
+ {
1222
+ "epoch": 0.36,
1223
+ "learning_rate": 0.00019820518161206864,
1224
+ "loss": 0.0546,
1225
+ "step": 1800
1226
+ },
1227
+ {
1228
+ "epoch": 0.36,
1229
+ "eval_loss": 0.06339588016271591,
1230
+ "eval_runtime": 1055.2159,
1231
+ "eval_samples_per_second": 6.973,
1232
+ "eval_steps_per_second": 1.744,
1233
+ "step": 1800
1234
+ },
1235
+ {
1236
+ "epoch": 0.36,
1237
+ "learning_rate": 0.00019818524449462862,
1238
+ "loss": 0.0463,
1239
+ "step": 1810
1240
+ },
1241
+ {
1242
+ "epoch": 0.37,
1243
+ "learning_rate": 0.00019816519826956156,
1244
+ "loss": 0.0571,
1245
+ "step": 1820
1246
+ },
1247
+ {
1248
+ "epoch": 0.37,
1249
+ "learning_rate": 0.00019814504295914372,
1250
+ "loss": 0.0516,
1251
+ "step": 1830
1252
+ },
1253
+ {
1254
+ "epoch": 0.37,
1255
+ "learning_rate": 0.00019812477858577258,
1256
+ "loss": 0.0511,
1257
+ "step": 1840
1258
+ },
1259
+ {
1260
+ "epoch": 0.37,
1261
+ "learning_rate": 0.00019810440517196672,
1262
+ "loss": 0.0471,
1263
+ "step": 1850
1264
+ },
1265
+ {
1266
+ "epoch": 0.37,
1267
+ "learning_rate": 0.00019808392274036596,
1268
+ "loss": 0.0497,
1269
+ "step": 1860
1270
+ },
1271
+ {
1272
+ "epoch": 0.38,
1273
+ "learning_rate": 0.00019806333131373126,
1274
+ "loss": 0.0608,
1275
+ "step": 1870
1276
+ },
1277
+ {
1278
+ "epoch": 0.38,
1279
+ "learning_rate": 0.00019804263091494466,
1280
+ "loss": 0.0431,
1281
+ "step": 1880
1282
+ },
1283
+ {
1284
+ "epoch": 0.38,
1285
+ "learning_rate": 0.00019802182156700938,
1286
+ "loss": 0.0558,
1287
+ "step": 1890
1288
+ },
1289
+ {
1290
+ "epoch": 0.38,
1291
+ "learning_rate": 0.00019800090329304956,
1292
+ "loss": 0.0524,
1293
+ "step": 1900
1294
+ },
1295
+ {
1296
+ "epoch": 0.38,
1297
+ "eval_loss": 0.06149962544441223,
1298
+ "eval_runtime": 1053.8269,
1299
+ "eval_samples_per_second": 6.982,
1300
+ "eval_steps_per_second": 1.746,
1301
+ "step": 1900
1302
+ },
1303
+ {
1304
+ "epoch": 0.38,
1305
+ "learning_rate": 0.00019797987611631059,
1306
+ "loss": 0.0528,
1307
+ "step": 1910
1308
+ },
1309
+ {
1310
+ "epoch": 0.39,
1311
+ "learning_rate": 0.00019795874006015872,
1312
+ "loss": 0.0481,
1313
+ "step": 1920
1314
+ },
1315
+ {
1316
+ "epoch": 0.39,
1317
+ "learning_rate": 0.0001979374951480812,
1318
+ "loss": 0.0491,
1319
+ "step": 1930
1320
+ },
1321
+ {
1322
+ "epoch": 0.39,
1323
+ "learning_rate": 0.00019791614140368633,
1324
+ "loss": 0.049,
1325
+ "step": 1940
1326
+ },
1327
+ {
1328
+ "epoch": 0.39,
1329
+ "learning_rate": 0.00019789467885070327,
1330
+ "loss": 0.0444,
1331
+ "step": 1950
1332
+ },
1333
+ {
1334
+ "epoch": 0.39,
1335
+ "learning_rate": 0.00019787310751298213,
1336
+ "loss": 0.0466,
1337
+ "step": 1960
1338
+ },
1339
+ {
1340
+ "epoch": 0.4,
1341
+ "learning_rate": 0.00019785142741449393,
1342
+ "loss": 0.0498,
1343
+ "step": 1970
1344
+ },
1345
+ {
1346
+ "epoch": 0.4,
1347
+ "learning_rate": 0.00019782963857933048,
1348
+ "loss": 0.0475,
1349
+ "step": 1980
1350
+ },
1351
+ {
1352
+ "epoch": 0.4,
1353
+ "learning_rate": 0.00019780774103170446,
1354
+ "loss": 0.047,
1355
+ "step": 1990
1356
+ },
1357
+ {
1358
+ "epoch": 0.4,
1359
+ "learning_rate": 0.00019778573479594942,
1360
+ "loss": 0.0432,
1361
+ "step": 2000
1362
+ },
1363
+ {
1364
+ "epoch": 0.4,
1365
+ "eval_loss": 0.06317023187875748,
1366
+ "eval_runtime": 1054.8968,
1367
+ "eval_samples_per_second": 6.975,
1368
+ "eval_steps_per_second": 1.744,
1369
+ "step": 2000
1370
+ },
1371
+ {
1372
+ "epoch": 0.4,
1373
+ "learning_rate": 0.00019776361989651956,
1374
+ "loss": 0.0546,
1375
+ "step": 2010
1376
+ },
1377
+ {
1378
+ "epoch": 0.41,
1379
+ "learning_rate": 0.0001977413963579899,
1380
+ "loss": 0.043,
1381
+ "step": 2020
1382
+ },
1383
+ {
1384
+ "epoch": 0.41,
1385
+ "learning_rate": 0.00019771906420505624,
1386
+ "loss": 0.0579,
1387
+ "step": 2030
1388
+ },
1389
+ {
1390
+ "epoch": 0.41,
1391
+ "learning_rate": 0.00019769662346253493,
1392
+ "loss": 0.057,
1393
+ "step": 2040
1394
+ },
1395
+ {
1396
+ "epoch": 0.41,
1397
+ "learning_rate": 0.00019767407415536317,
1398
+ "loss": 0.0476,
1399
+ "step": 2050
1400
+ },
1401
+ {
1402
+ "epoch": 0.41,
1403
+ "learning_rate": 0.00019765141630859865,
1404
+ "loss": 0.0418,
1405
+ "step": 2060
1406
+ },
1407
+ {
1408
+ "epoch": 0.42,
1409
+ "learning_rate": 0.00019762864994741976,
1410
+ "loss": 0.0497,
1411
+ "step": 2070
1412
+ },
1413
+ {
1414
+ "epoch": 0.42,
1415
+ "learning_rate": 0.00019760577509712546,
1416
+ "loss": 0.0493,
1417
+ "step": 2080
1418
+ },
1419
+ {
1420
+ "epoch": 0.42,
1421
+ "learning_rate": 0.00019758279178313525,
1422
+ "loss": 0.0512,
1423
+ "step": 2090
1424
+ },
1425
+ {
1426
+ "epoch": 0.42,
1427
+ "learning_rate": 0.00019755970003098916,
1428
+ "loss": 0.0631,
1429
+ "step": 2100
1430
+ },
1431
+ {
1432
+ "epoch": 0.42,
1433
+ "eval_loss": 0.06188439950346947,
1434
+ "eval_runtime": 1062.072,
1435
+ "eval_samples_per_second": 6.928,
1436
+ "eval_steps_per_second": 1.732,
1437
+ "step": 2100
1438
+ },
1439
+ {
1440
+ "epoch": 0.42,
1441
+ "learning_rate": 0.00019753649986634772,
1442
+ "loss": 0.0543,
1443
+ "step": 2110
1444
+ },
1445
+ {
1446
+ "epoch": 0.43,
1447
+ "learning_rate": 0.00019751319131499194,
1448
+ "loss": 0.0405,
1449
+ "step": 2120
1450
+ },
1451
+ {
1452
+ "epoch": 0.43,
1453
+ "learning_rate": 0.00019748977440282333,
1454
+ "loss": 0.0564,
1455
+ "step": 2130
1456
+ },
1457
+ {
1458
+ "epoch": 0.43,
1459
+ "learning_rate": 0.0001974662491558637,
1460
+ "loss": 0.0396,
1461
+ "step": 2140
1462
+ },
1463
+ {
1464
+ "epoch": 0.43,
1465
+ "learning_rate": 0.00019744261560025533,
1466
+ "loss": 0.039,
1467
+ "step": 2150
1468
+ },
1469
+ {
1470
+ "epoch": 0.43,
1471
+ "learning_rate": 0.00019741887376226083,
1472
+ "loss": 0.0348,
1473
+ "step": 2160
1474
+ },
1475
+ {
1476
+ "epoch": 0.44,
1477
+ "learning_rate": 0.00019739502366826313,
1478
+ "loss": 0.0533,
1479
+ "step": 2170
1480
+ },
1481
+ {
1482
+ "epoch": 0.44,
1483
+ "learning_rate": 0.0001973710653447655,
1484
+ "loss": 0.0417,
1485
+ "step": 2180
1486
+ },
1487
+ {
1488
+ "epoch": 0.44,
1489
+ "learning_rate": 0.0001973469988183914,
1490
+ "loss": 0.0502,
1491
+ "step": 2190
1492
+ },
1493
+ {
1494
+ "epoch": 0.44,
1495
+ "learning_rate": 0.00019732282411588463,
1496
+ "loss": 0.0519,
1497
+ "step": 2200
1498
+ },
1499
+ {
1500
+ "epoch": 0.44,
1501
+ "eval_loss": 0.05978045240044594,
1502
+ "eval_runtime": 1059.6808,
1503
+ "eval_samples_per_second": 6.944,
1504
+ "eval_steps_per_second": 1.736,
1505
+ "step": 2200
1506
+ },
1507
+ {
1508
+ "epoch": 0.44,
1509
+ "learning_rate": 0.00019729854126410913,
1510
+ "loss": 0.0416,
1511
+ "step": 2210
1512
+ },
1513
+ {
1514
+ "epoch": 0.45,
1515
+ "learning_rate": 0.00019727415029004906,
1516
+ "loss": 0.0586,
1517
+ "step": 2220
1518
+ },
1519
+ {
1520
+ "epoch": 0.45,
1521
+ "learning_rate": 0.00019724965122080868,
1522
+ "loss": 0.0535,
1523
+ "step": 2230
1524
+ },
1525
+ {
1526
+ "epoch": 0.45,
1527
+ "learning_rate": 0.0001972250440836124,
1528
+ "loss": 0.0499,
1529
+ "step": 2240
1530
+ },
1531
+ {
1532
+ "epoch": 0.45,
1533
+ "learning_rate": 0.00019720032890580474,
1534
+ "loss": 0.0484,
1535
+ "step": 2250
1536
+ },
1537
+ {
1538
+ "epoch": 0.46,
1539
+ "learning_rate": 0.00019717550571485024,
1540
+ "loss": 0.044,
1541
+ "step": 2260
1542
+ },
1543
+ {
1544
+ "epoch": 0.46,
1545
+ "learning_rate": 0.0001971505745383335,
1546
+ "loss": 0.0496,
1547
+ "step": 2270
1548
+ },
1549
+ {
1550
+ "epoch": 0.46,
1551
+ "learning_rate": 0.00019712553540395908,
1552
+ "loss": 0.05,
1553
+ "step": 2280
1554
+ },
1555
+ {
1556
+ "epoch": 0.46,
1557
+ "learning_rate": 0.0001971003883395516,
1558
+ "loss": 0.0387,
1559
+ "step": 2290
1560
+ },
1561
+ {
1562
+ "epoch": 0.46,
1563
+ "learning_rate": 0.00019707513337305547,
1564
+ "loss": 0.0397,
1565
+ "step": 2300
1566
+ },
1567
+ {
1568
+ "epoch": 0.46,
1569
+ "eval_loss": 0.060694798827171326,
1570
+ "eval_runtime": 1049.503,
1571
+ "eval_samples_per_second": 7.011,
1572
+ "eval_steps_per_second": 1.753,
1573
+ "step": 2300
1574
+ },
1575
+ {
1576
+ "epoch": 0.47,
1577
+ "learning_rate": 0.0001970497705325351,
1578
+ "loss": 0.0475,
1579
+ "step": 2310
1580
+ },
1581
+ {
1582
+ "epoch": 0.47,
1583
+ "learning_rate": 0.00019702429984617484,
1584
+ "loss": 0.0405,
1585
+ "step": 2320
1586
+ },
1587
+ {
1588
+ "epoch": 0.47,
1589
+ "learning_rate": 0.00019699872134227867,
1590
+ "loss": 0.0447,
1591
+ "step": 2330
1592
+ },
1593
+ {
1594
+ "epoch": 0.47,
1595
+ "learning_rate": 0.00019697303504927061,
1596
+ "loss": 0.0608,
1597
+ "step": 2340
1598
+ },
1599
+ {
1600
+ "epoch": 0.47,
1601
+ "learning_rate": 0.00019694724099569434,
1602
+ "loss": 0.0478,
1603
+ "step": 2350
1604
+ },
1605
+ {
1606
+ "epoch": 0.48,
1607
+ "learning_rate": 0.00019692133921021332,
1608
+ "loss": 0.0432,
1609
+ "step": 2360
1610
+ },
1611
+ {
1612
+ "epoch": 0.48,
1613
+ "learning_rate": 0.00019689532972161068,
1614
+ "loss": 0.0492,
1615
+ "step": 2370
1616
+ },
1617
+ {
1618
+ "epoch": 0.48,
1619
+ "learning_rate": 0.00019686921255878932,
1620
+ "loss": 0.0441,
1621
+ "step": 2380
1622
+ },
1623
+ {
1624
+ "epoch": 0.48,
1625
+ "learning_rate": 0.0001968429877507717,
1626
+ "loss": 0.0603,
1627
+ "step": 2390
1628
+ },
1629
+ {
1630
+ "epoch": 0.48,
1631
+ "learning_rate": 0.00019681665532669996,
1632
+ "loss": 0.0467,
1633
+ "step": 2400
1634
+ },
1635
+ {
1636
+ "epoch": 0.48,
1637
+ "eval_loss": 0.06163698434829712,
1638
+ "eval_runtime": 1049.1311,
1639
+ "eval_samples_per_second": 7.013,
1640
+ "eval_steps_per_second": 1.754,
1641
+ "step": 2400
1642
+ },
1643
+ {
1644
+ "epoch": 0.49,
1645
+ "learning_rate": 0.00019679021531583584,
1646
+ "loss": 0.0406,
1647
+ "step": 2410
1648
+ },
1649
+ {
1650
+ "epoch": 0.49,
1651
+ "learning_rate": 0.00019676366774756056,
1652
+ "loss": 0.0484,
1653
+ "step": 2420
1654
+ },
1655
+ {
1656
+ "epoch": 0.49,
1657
+ "learning_rate": 0.00019673701265137495,
1658
+ "loss": 0.0402,
1659
+ "step": 2430
1660
+ },
1661
+ {
1662
+ "epoch": 0.49,
1663
+ "learning_rate": 0.00019671025005689926,
1664
+ "loss": 0.0541,
1665
+ "step": 2440
1666
+ },
1667
+ {
1668
+ "epoch": 0.49,
1669
+ "learning_rate": 0.00019668337999387324,
1670
+ "loss": 0.0438,
1671
+ "step": 2450
1672
+ },
1673
+ {
1674
+ "epoch": 0.5,
1675
+ "learning_rate": 0.00019665640249215605,
1676
+ "loss": 0.045,
1677
+ "step": 2460
1678
+ },
1679
+ {
1680
+ "epoch": 0.5,
1681
+ "learning_rate": 0.0001966293175817262,
1682
+ "loss": 0.0456,
1683
+ "step": 2470
1684
+ },
1685
+ {
1686
+ "epoch": 0.5,
1687
+ "learning_rate": 0.00019660212529268168,
1688
+ "loss": 0.0454,
1689
+ "step": 2480
1690
+ },
1691
+ {
1692
+ "epoch": 0.5,
1693
+ "learning_rate": 0.00019657482565523963,
1694
+ "loss": 0.0464,
1695
+ "step": 2490
1696
+ },
1697
+ {
1698
+ "epoch": 0.5,
1699
+ "learning_rate": 0.00019654741869973663,
1700
+ "loss": 0.049,
1701
+ "step": 2500
1702
+ },
1703
+ {
1704
+ "epoch": 0.5,
1705
+ "eval_loss": 0.06358644366264343,
1706
+ "eval_runtime": 1050.5846,
1707
+ "eval_samples_per_second": 7.004,
1708
+ "eval_steps_per_second": 1.751,
1709
+ "step": 2500
1710
+ },
1711
+ {
1712
+ "epoch": 0.51,
1713
+ "learning_rate": 0.00019651990445662841,
1714
+ "loss": 0.053,
1715
+ "step": 2510
1716
+ },
1717
+ {
1718
+ "epoch": 0.51,
1719
+ "learning_rate": 0.00019649228295649004,
1720
+ "loss": 0.05,
1721
+ "step": 2520
1722
+ },
1723
+ {
1724
+ "epoch": 0.51,
1725
+ "learning_rate": 0.00019646455423001565,
1726
+ "loss": 0.0423,
1727
+ "step": 2530
1728
+ },
1729
+ {
1730
+ "epoch": 0.51,
1731
+ "learning_rate": 0.0001964367183080186,
1732
+ "loss": 0.0556,
1733
+ "step": 2540
1734
+ },
1735
+ {
1736
+ "epoch": 0.51,
1737
+ "learning_rate": 0.00019640877522143134,
1738
+ "loss": 0.0431,
1739
+ "step": 2550
1740
+ },
1741
+ {
1742
+ "epoch": 0.52,
1743
+ "learning_rate": 0.0001963807250013054,
1744
+ "loss": 0.0377,
1745
+ "step": 2560
1746
+ },
1747
+ {
1748
+ "epoch": 0.52,
1749
+ "learning_rate": 0.00019635256767881144,
1750
+ "loss": 0.0432,
1751
+ "step": 2570
1752
+ },
1753
+ {
1754
+ "epoch": 0.52,
1755
+ "learning_rate": 0.00019632430328523902,
1756
+ "loss": 0.0562,
1757
+ "step": 2580
1758
+ },
1759
+ {
1760
+ "epoch": 0.52,
1761
+ "learning_rate": 0.0001962959318519968,
1762
+ "loss": 0.04,
1763
+ "step": 2590
1764
+ },
1765
+ {
1766
+ "epoch": 0.52,
1767
+ "learning_rate": 0.00019626745341061225,
1768
+ "loss": 0.0488,
1769
+ "step": 2600
1770
+ },
1771
+ {
1772
+ "epoch": 0.52,
1773
+ "eval_loss": 0.06040577217936516,
1774
+ "eval_runtime": 1049.7742,
1775
+ "eval_samples_per_second": 7.009,
1776
+ "eval_steps_per_second": 1.753,
1777
+ "step": 2600
1778
+ },
1779
+ {
1780
+ "epoch": 0.53,
1781
+ "learning_rate": 0.0001962388679927319,
1782
+ "loss": 0.0371,
1783
+ "step": 2610
1784
+ },
1785
+ {
1786
+ "epoch": 0.53,
1787
+ "learning_rate": 0.000196210175630121,
1788
+ "loss": 0.0412,
1789
+ "step": 2620
1790
+ },
1791
+ {
1792
+ "epoch": 0.53,
1793
+ "learning_rate": 0.00019618137635466382,
1794
+ "loss": 0.0432,
1795
+ "step": 2630
1796
+ },
1797
+ {
1798
+ "epoch": 0.53,
1799
+ "learning_rate": 0.00019615247019836327,
1800
+ "loss": 0.0501,
1801
+ "step": 2640
1802
+ },
1803
+ {
1804
+ "epoch": 0.53,
1805
+ "learning_rate": 0.00019612345719334116,
1806
+ "loss": 0.0422,
1807
+ "step": 2650
1808
+ },
1809
+ {
1810
+ "epoch": 0.54,
1811
+ "learning_rate": 0.00019609433737183791,
1812
+ "loss": 0.0511,
1813
+ "step": 2660
1814
+ },
1815
+ {
1816
+ "epoch": 0.54,
1817
+ "learning_rate": 0.00019606511076621276,
1818
+ "loss": 0.0372,
1819
+ "step": 2670
1820
+ },
1821
+ {
1822
+ "epoch": 0.54,
1823
+ "learning_rate": 0.00019603577740894354,
1824
+ "loss": 0.0541,
1825
+ "step": 2680
1826
+ },
1827
+ {
1828
+ "epoch": 0.54,
1829
+ "learning_rate": 0.0001960063373326267,
1830
+ "loss": 0.0472,
1831
+ "step": 2690
1832
+ },
1833
+ {
1834
+ "epoch": 0.54,
1835
+ "learning_rate": 0.00019597679056997737,
1836
+ "loss": 0.0449,
1837
+ "step": 2700
1838
+ },
1839
+ {
1840
+ "epoch": 0.54,
1841
+ "eval_loss": 0.05977201834321022,
1842
+ "eval_runtime": 1052.2024,
1843
+ "eval_samples_per_second": 6.993,
1844
+ "eval_steps_per_second": 1.749,
1845
+ "step": 2700
1846
+ },
1847
+ {
1848
+ "epoch": 0.55,
1849
+ "learning_rate": 0.00019594713715382915,
1850
+ "loss": 0.0517,
1851
+ "step": 2710
1852
+ },
1853
+ {
1854
+ "epoch": 0.55,
1855
+ "learning_rate": 0.00019591737711713414,
1856
+ "loss": 0.0476,
1857
+ "step": 2720
1858
+ },
1859
+ {
1860
+ "epoch": 0.55,
1861
+ "learning_rate": 0.00019588751049296298,
1862
+ "loss": 0.048,
1863
+ "step": 2730
1864
+ },
1865
+ {
1866
+ "epoch": 0.55,
1867
+ "learning_rate": 0.00019585753731450478,
1868
+ "loss": 0.0342,
1869
+ "step": 2740
1870
+ },
1871
+ {
1872
+ "epoch": 0.55,
1873
+ "learning_rate": 0.00019582745761506697,
1874
+ "loss": 0.0508,
1875
+ "step": 2750
1876
+ },
1877
+ {
1878
+ "epoch": 0.56,
1879
+ "learning_rate": 0.00019579727142807535,
1880
+ "loss": 0.0407,
1881
+ "step": 2760
1882
+ },
1883
+ {
1884
+ "epoch": 0.56,
1885
+ "learning_rate": 0.0001957669787870742,
1886
+ "loss": 0.0358,
1887
+ "step": 2770
1888
+ },
1889
+ {
1890
+ "epoch": 0.56,
1891
+ "learning_rate": 0.00019573657972572593,
1892
+ "loss": 0.0506,
1893
+ "step": 2780
1894
+ },
1895
+ {
1896
+ "epoch": 0.56,
1897
+ "learning_rate": 0.00019570607427781128,
1898
+ "loss": 0.0476,
1899
+ "step": 2790
1900
+ },
1901
+ {
1902
+ "epoch": 0.56,
1903
+ "learning_rate": 0.0001956754624772292,
1904
+ "loss": 0.0438,
1905
+ "step": 2800
1906
+ },
1907
+ {
1908
+ "epoch": 0.56,
1909
+ "eval_loss": 0.061029911041259766,
1910
+ "eval_runtime": 1062.1739,
1911
+ "eval_samples_per_second": 6.927,
1912
+ "eval_steps_per_second": 1.732,
1913
+ "step": 2800
1914
+ },
1915
+ {
1916
+ "epoch": 0.57,
1917
+ "learning_rate": 0.0001956447443579968,
1918
+ "loss": 0.0481,
1919
+ "step": 2810
1920
+ },
1921
+ {
1922
+ "epoch": 0.57,
1923
+ "learning_rate": 0.00019561391995424941,
1924
+ "loss": 0.0515,
1925
+ "step": 2820
1926
+ },
1927
+ {
1928
+ "epoch": 0.57,
1929
+ "learning_rate": 0.00019558298930024044,
1930
+ "loss": 0.0491,
1931
+ "step": 2830
1932
+ },
1933
+ {
1934
+ "epoch": 0.57,
1935
+ "learning_rate": 0.0001955519524303413,
1936
+ "loss": 0.0367,
1937
+ "step": 2840
1938
+ },
1939
+ {
1940
+ "epoch": 0.57,
1941
+ "learning_rate": 0.0001955208093790415,
1942
+ "loss": 0.0431,
1943
+ "step": 2850
1944
+ },
1945
+ {
1946
+ "epoch": 0.58,
1947
+ "learning_rate": 0.0001954895601809485,
1948
+ "loss": 0.0436,
1949
+ "step": 2860
1950
+ },
1951
+ {
1952
+ "epoch": 0.58,
1953
+ "learning_rate": 0.0001954582048707878,
1954
+ "loss": 0.0442,
1955
+ "step": 2870
1956
+ },
1957
+ {
1958
+ "epoch": 0.58,
1959
+ "learning_rate": 0.00019542674348340267,
1960
+ "loss": 0.0532,
1961
+ "step": 2880
1962
+ },
1963
+ {
1964
+ "epoch": 0.58,
1965
+ "learning_rate": 0.00019539517605375446,
1966
+ "loss": 0.0441,
1967
+ "step": 2890
1968
+ },
1969
+ {
1970
+ "epoch": 0.58,
1971
+ "learning_rate": 0.00019536350261692214,
1972
+ "loss": 0.036,
1973
+ "step": 2900
1974
+ },
1975
+ {
1976
+ "epoch": 0.58,
1977
+ "eval_loss": 0.06326338648796082,
1978
+ "eval_runtime": 1061.8457,
1979
+ "eval_samples_per_second": 6.929,
1980
+ "eval_steps_per_second": 1.733,
1981
+ "step": 2900
1982
+ },
1983
+ {
1984
+ "epoch": 0.59,
1985
+ "learning_rate": 0.00019533172320810265,
1986
+ "loss": 0.0486,
1987
+ "step": 2910
1988
+ },
1989
+ {
1990
+ "epoch": 0.59,
1991
+ "learning_rate": 0.00019529983786261058,
1992
+ "loss": 0.0568,
1993
+ "step": 2920
1994
+ },
1995
+ {
1996
+ "epoch": 0.59,
1997
+ "learning_rate": 0.00019526784661587829,
1998
+ "loss": 0.0434,
1999
+ "step": 2930
2000
+ },
2001
+ {
2002
+ "epoch": 0.59,
2003
+ "learning_rate": 0.0001952357495034558,
2004
+ "loss": 0.0541,
2005
+ "step": 2940
2006
+ },
2007
+ {
2008
+ "epoch": 0.59,
2009
+ "learning_rate": 0.00019520354656101085,
2010
+ "loss": 0.0395,
2011
+ "step": 2950
2012
+ },
2013
+ {
2014
+ "epoch": 0.6,
2015
+ "learning_rate": 0.00019517123782432868,
2016
+ "loss": 0.0458,
2017
+ "step": 2960
2018
+ },
2019
+ {
2020
+ "epoch": 0.6,
2021
+ "learning_rate": 0.00019513882332931212,
2022
+ "loss": 0.0395,
2023
+ "step": 2970
2024
+ },
2025
+ {
2026
+ "epoch": 0.6,
2027
+ "learning_rate": 0.00019510630311198157,
2028
+ "loss": 0.0442,
2029
+ "step": 2980
2030
+ },
2031
+ {
2032
+ "epoch": 0.6,
2033
+ "learning_rate": 0.00019507367720847488,
2034
+ "loss": 0.041,
2035
+ "step": 2990
2036
+ },
2037
+ {
2038
+ "epoch": 0.6,
2039
+ "learning_rate": 0.00019504094565504733,
2040
+ "loss": 0.0464,
2041
+ "step": 3000
2042
+ },
2043
+ {
2044
+ "epoch": 0.6,
2045
+ "eval_loss": 0.06028781458735466,
2046
+ "eval_runtime": 1035.9718,
2047
+ "eval_samples_per_second": 7.103,
2048
+ "eval_steps_per_second": 1.776,
2049
+ "step": 3000
2050
+ },
2051
+ {
2052
+ "epoch": 0.61,
2053
+ "learning_rate": 0.00019500810848807162,
2054
+ "loss": 0.0396,
2055
+ "step": 3010
2056
+ },
2057
+ {
2058
+ "epoch": 0.61,
2059
+ "learning_rate": 0.00019497516574403778,
2060
+ "loss": 0.0491,
2061
+ "step": 3020
2062
+ },
2063
+ {
2064
+ "epoch": 0.61,
2065
+ "learning_rate": 0.00019494211745955324,
2066
+ "loss": 0.0585,
2067
+ "step": 3030
2068
+ },
2069
+ {
2070
+ "epoch": 0.61,
2071
+ "learning_rate": 0.00019490896367134266,
2072
+ "loss": 0.0484,
2073
+ "step": 3040
2074
+ },
2075
+ {
2076
+ "epoch": 0.61,
2077
+ "learning_rate": 0.00019487570441624791,
2078
+ "loss": 0.0507,
2079
+ "step": 3050
2080
+ },
2081
+ {
2082
+ "epoch": 0.62,
2083
+ "learning_rate": 0.0001948423397312281,
2084
+ "loss": 0.0467,
2085
+ "step": 3060
2086
+ },
2087
+ {
2088
+ "epoch": 0.62,
2089
+ "learning_rate": 0.0001948088696533595,
2090
+ "loss": 0.0444,
2091
+ "step": 3070
2092
+ },
2093
+ {
2094
+ "epoch": 0.62,
2095
+ "learning_rate": 0.00019477529421983546,
2096
+ "loss": 0.0498,
2097
+ "step": 3080
2098
+ },
2099
+ {
2100
+ "epoch": 0.62,
2101
+ "learning_rate": 0.0001947416134679665,
2102
+ "loss": 0.0373,
2103
+ "step": 3090
2104
+ },
2105
+ {
2106
+ "epoch": 0.62,
2107
+ "learning_rate": 0.00019470782743518002,
2108
+ "loss": 0.0437,
2109
+ "step": 3100
2110
+ },
2111
+ {
2112
+ "epoch": 0.62,
2113
+ "eval_loss": 0.06118550896644592,
2114
+ "eval_runtime": 1055.3909,
2115
+ "eval_samples_per_second": 6.972,
2116
+ "eval_steps_per_second": 1.743,
2117
+ "step": 3100
2118
+ },
2119
+ {
2120
+ "epoch": 0.63,
2121
+ "learning_rate": 0.00019467393615902055,
2122
+ "loss": 0.0391,
2123
+ "step": 3110
2124
+ },
2125
+ {
2126
+ "epoch": 0.63,
2127
+ "learning_rate": 0.0001946399396771495,
2128
+ "loss": 0.0456,
2129
+ "step": 3120
2130
+ },
2131
+ {
2132
+ "epoch": 0.63,
2133
+ "learning_rate": 0.00019460583802734523,
2134
+ "loss": 0.0368,
2135
+ "step": 3130
2136
+ },
2137
+ {
2138
+ "epoch": 0.63,
2139
+ "learning_rate": 0.0001945716312475029,
2140
+ "loss": 0.0451,
2141
+ "step": 3140
2142
+ },
2143
+ {
2144
+ "epoch": 0.63,
2145
+ "learning_rate": 0.0001945373193756346,
2146
+ "loss": 0.046,
2147
+ "step": 3150
2148
+ },
2149
+ {
2150
+ "epoch": 0.64,
2151
+ "learning_rate": 0.00019450290244986914,
2152
+ "loss": 0.0364,
2153
+ "step": 3160
2154
+ },
2155
+ {
2156
+ "epoch": 0.64,
2157
+ "learning_rate": 0.00019446838050845205,
2158
+ "loss": 0.0576,
2159
+ "step": 3170
2160
+ },
2161
+ {
2162
+ "epoch": 0.64,
2163
+ "learning_rate": 0.00019443375358974555,
2164
+ "loss": 0.0421,
2165
+ "step": 3180
2166
+ },
2167
+ {
2168
+ "epoch": 0.64,
2169
+ "learning_rate": 0.00019439902173222859,
2170
+ "loss": 0.0528,
2171
+ "step": 3190
2172
+ },
2173
+ {
2174
+ "epoch": 0.64,
2175
+ "learning_rate": 0.0001943641849744967,
2176
+ "loss": 0.0389,
2177
+ "step": 3200
2178
+ },
2179
+ {
2180
+ "epoch": 0.64,
2181
+ "eval_loss": 0.06049993634223938,
2182
+ "eval_runtime": 1052.016,
2183
+ "eval_samples_per_second": 6.994,
2184
+ "eval_steps_per_second": 1.749,
2185
+ "step": 3200
2186
+ },
2187
+ {
2188
+ "epoch": 0.65,
2189
+ "learning_rate": 0.00019432924335526194,
2190
+ "loss": 0.0432,
2191
+ "step": 3210
2192
+ },
2193
+ {
2194
+ "epoch": 0.65,
2195
+ "learning_rate": 0.00019429419691335297,
2196
+ "loss": 0.0339,
2197
+ "step": 3220
2198
+ },
2199
+ {
2200
+ "epoch": 0.65,
2201
+ "learning_rate": 0.00019425904568771483,
2202
+ "loss": 0.0434,
2203
+ "step": 3230
2204
+ },
2205
+ {
2206
+ "epoch": 0.65,
2207
+ "learning_rate": 0.00019422378971740907,
2208
+ "loss": 0.0472,
2209
+ "step": 3240
2210
+ },
2211
+ {
2212
+ "epoch": 0.65,
2213
+ "learning_rate": 0.00019418842904161368,
2214
+ "loss": 0.0394,
2215
+ "step": 3250
2216
+ },
2217
+ {
2218
+ "epoch": 0.66,
2219
+ "learning_rate": 0.00019415296369962288,
2220
+ "loss": 0.0404,
2221
+ "step": 3260
2222
+ },
2223
+ {
2224
+ "epoch": 0.66,
2225
+ "learning_rate": 0.00019411739373084732,
2226
+ "loss": 0.0352,
2227
+ "step": 3270
2228
+ },
2229
+ {
2230
+ "epoch": 0.66,
2231
+ "learning_rate": 0.00019408171917481386,
2232
+ "loss": 0.0364,
2233
+ "step": 3280
2234
+ },
2235
+ {
2236
+ "epoch": 0.66,
2237
+ "learning_rate": 0.00019404594007116555,
2238
+ "loss": 0.0433,
2239
+ "step": 3290
2240
+ },
2241
+ {
2242
+ "epoch": 0.66,
2243
+ "learning_rate": 0.00019401005645966167,
2244
+ "loss": 0.0377,
2245
+ "step": 3300
2246
+ },
2247
+ {
2248
+ "epoch": 0.66,
2249
+ "eval_loss": 0.06322025507688522,
2250
+ "eval_runtime": 1050.1226,
2251
+ "eval_samples_per_second": 7.007,
2252
+ "eval_steps_per_second": 1.752,
2253
+ "step": 3300
2254
+ },
2255
+ {
2256
+ "epoch": 0.67,
2257
+ "learning_rate": 0.00019397406838017766,
2258
+ "loss": 0.042,
2259
+ "step": 3310
2260
+ },
2261
+ {
2262
+ "epoch": 0.67,
2263
+ "learning_rate": 0.00019393797587270497,
2264
+ "loss": 0.0326,
2265
+ "step": 3320
2266
+ },
2267
+ {
2268
+ "epoch": 0.67,
2269
+ "learning_rate": 0.00019390177897735114,
2270
+ "loss": 0.0443,
2271
+ "step": 3330
2272
+ },
2273
+ {
2274
+ "epoch": 0.67,
2275
+ "learning_rate": 0.00019386547773433965,
2276
+ "loss": 0.0545,
2277
+ "step": 3340
2278
+ },
2279
+ {
2280
+ "epoch": 0.67,
2281
+ "learning_rate": 0.00019382907218401006,
2282
+ "loss": 0.0422,
2283
+ "step": 3350
2284
+ },
2285
+ {
2286
+ "epoch": 0.68,
2287
+ "learning_rate": 0.00019379256236681775,
2288
+ "loss": 0.0393,
2289
+ "step": 3360
2290
+ },
2291
+ {
2292
+ "epoch": 0.68,
2293
+ "learning_rate": 0.000193755948323334,
2294
+ "loss": 0.0516,
2295
+ "step": 3370
2296
+ },
2297
+ {
2298
+ "epoch": 0.68,
2299
+ "learning_rate": 0.00019371923009424587,
2300
+ "loss": 0.0499,
2301
+ "step": 3380
2302
+ },
2303
+ {
2304
+ "epoch": 0.68,
2305
+ "learning_rate": 0.0001936824077203562,
2306
+ "loss": 0.0491,
2307
+ "step": 3390
2308
+ },
2309
+ {
2310
+ "epoch": 0.68,
2311
+ "learning_rate": 0.0001936454812425836,
2312
+ "loss": 0.0382,
2313
+ "step": 3400
2314
+ },
2315
+ {
2316
+ "epoch": 0.68,
2317
+ "eval_loss": 0.06261178106069565,
2318
+ "eval_runtime": 1054.0723,
2319
+ "eval_samples_per_second": 6.981,
2320
+ "eval_steps_per_second": 1.746,
2321
+ "step": 3400
2322
+ },
2323
+ {
2324
+ "epoch": 0.69,
2325
+ "learning_rate": 0.00019360845070196236,
2326
+ "loss": 0.0431,
2327
+ "step": 3410
2328
+ },
2329
+ {
2330
+ "epoch": 0.69,
2331
+ "learning_rate": 0.0001935713161396424,
2332
+ "loss": 0.0498,
2333
+ "step": 3420
2334
+ },
2335
+ {
2336
+ "epoch": 0.69,
2337
+ "learning_rate": 0.0001935340775968892,
2338
+ "loss": 0.0457,
2339
+ "step": 3430
2340
+ },
2341
+ {
2342
+ "epoch": 0.69,
2343
+ "learning_rate": 0.00019349673511508383,
2344
+ "loss": 0.0362,
2345
+ "step": 3440
2346
+ },
2347
+ {
2348
+ "epoch": 0.69,
2349
+ "learning_rate": 0.00019345928873572282,
2350
+ "loss": 0.0432,
2351
+ "step": 3450
2352
+ },
2353
+ {
2354
+ "epoch": 0.7,
2355
+ "learning_rate": 0.00019342173850041822,
2356
+ "loss": 0.0365,
2357
+ "step": 3460
2358
+ },
2359
+ {
2360
+ "epoch": 0.7,
2361
+ "learning_rate": 0.00019338408445089745,
2362
+ "loss": 0.0358,
2363
+ "step": 3470
2364
+ },
2365
+ {
2366
+ "epoch": 0.7,
2367
+ "learning_rate": 0.0001933463266290033,
2368
+ "loss": 0.039,
2369
+ "step": 3480
2370
+ },
2371
+ {
2372
+ "epoch": 0.7,
2373
+ "learning_rate": 0.00019330846507669382,
2374
+ "loss": 0.0412,
2375
+ "step": 3490
2376
+ },
2377
+ {
2378
+ "epoch": 0.7,
2379
+ "learning_rate": 0.00019327049983604245,
2380
+ "loss": 0.051,
2381
+ "step": 3500
2382
+ },
2383
+ {
2384
+ "epoch": 0.7,
2385
+ "eval_loss": 0.061244383454322815,
2386
+ "eval_runtime": 1045.5529,
2387
+ "eval_samples_per_second": 7.037,
2388
+ "eval_steps_per_second": 1.76,
2389
+ "step": 3500
2390
+ },
2391
+ {
2392
+ "epoch": 0.71,
2393
+ "learning_rate": 0.00019323243094923772,
2394
+ "loss": 0.0387,
2395
+ "step": 3510
2396
+ },
2397
+ {
2398
+ "epoch": 0.71,
2399
+ "learning_rate": 0.00019319425845858341,
2400
+ "loss": 0.0473,
2401
+ "step": 3520
2402
+ },
2403
+ {
2404
+ "epoch": 0.71,
2405
+ "learning_rate": 0.00019315598240649847,
2406
+ "loss": 0.037,
2407
+ "step": 3530
2408
+ },
2409
+ {
2410
+ "epoch": 0.71,
2411
+ "learning_rate": 0.0001931176028355168,
2412
+ "loss": 0.0435,
2413
+ "step": 3540
2414
+ },
2415
+ {
2416
+ "epoch": 0.71,
2417
+ "learning_rate": 0.00019307911978828747,
2418
+ "loss": 0.0364,
2419
+ "step": 3550
2420
+ },
2421
+ {
2422
+ "epoch": 0.72,
2423
+ "learning_rate": 0.0001930405333075745,
2424
+ "loss": 0.0424,
2425
+ "step": 3560
2426
+ },
2427
+ {
2428
+ "epoch": 0.72,
2429
+ "learning_rate": 0.00019300184343625678,
2430
+ "loss": 0.0348,
2431
+ "step": 3570
2432
+ },
2433
+ {
2434
+ "epoch": 0.72,
2435
+ "learning_rate": 0.00019296305021732817,
2436
+ "loss": 0.0395,
2437
+ "step": 3580
2438
+ },
2439
+ {
2440
+ "epoch": 0.72,
2441
+ "learning_rate": 0.00019292415369389734,
2442
+ "loss": 0.0486,
2443
+ "step": 3590
2444
+ },
2445
+ {
2446
+ "epoch": 0.72,
2447
+ "learning_rate": 0.00019288515390918776,
2448
+ "loss": 0.047,
2449
+ "step": 3600
2450
+ },
2451
+ {
2452
+ "epoch": 0.72,
2453
+ "eval_loss": 0.06326541304588318,
2454
+ "eval_runtime": 1046.8433,
2455
+ "eval_samples_per_second": 7.029,
2456
+ "eval_steps_per_second": 1.758,
2457
+ "step": 3600
2458
+ },
2459
+ {
2460
+ "epoch": 0.73,
2461
+ "learning_rate": 0.00019284605090653766,
2462
+ "loss": 0.0403,
2463
+ "step": 3610
2464
+ },
2465
+ {
2466
+ "epoch": 0.73,
2467
+ "learning_rate": 0.00019280684472939994,
2468
+ "loss": 0.0381,
2469
+ "step": 3620
2470
+ },
2471
+ {
2472
+ "epoch": 0.73,
2473
+ "learning_rate": 0.00019276753542134224,
2474
+ "loss": 0.0332,
2475
+ "step": 3630
2476
+ },
2477
+ {
2478
+ "epoch": 0.73,
2479
+ "learning_rate": 0.00019272812302604665,
2480
+ "loss": 0.0435,
2481
+ "step": 3640
2482
+ },
2483
+ {
2484
+ "epoch": 0.73,
2485
+ "learning_rate": 0.00019268860758730997,
2486
+ "loss": 0.0512,
2487
+ "step": 3650
2488
+ },
2489
+ {
2490
+ "epoch": 0.74,
2491
+ "learning_rate": 0.00019264898914904342,
2492
+ "loss": 0.0403,
2493
+ "step": 3660
2494
+ },
2495
+ {
2496
+ "epoch": 0.74,
2497
+ "learning_rate": 0.00019260926775527265,
2498
+ "loss": 0.0336,
2499
+ "step": 3670
2500
+ },
2501
+ {
2502
+ "epoch": 0.74,
2503
+ "learning_rate": 0.00019256944345013785,
2504
+ "loss": 0.0425,
2505
+ "step": 3680
2506
+ },
2507
+ {
2508
+ "epoch": 0.74,
2509
+ "learning_rate": 0.00019252951627789344,
2510
+ "loss": 0.0431,
2511
+ "step": 3690
2512
+ },
2513
+ {
2514
+ "epoch": 0.74,
2515
+ "learning_rate": 0.00019248948628290818,
2516
+ "loss": 0.0451,
2517
+ "step": 3700
2518
+ },
2519
+ {
2520
+ "epoch": 0.74,
2521
+ "eval_loss": 0.06083804368972778,
2522
+ "eval_runtime": 1049.5583,
2523
+ "eval_samples_per_second": 7.011,
2524
+ "eval_steps_per_second": 1.753,
2525
+ "step": 3700
2526
+ },
2527
+ {
2528
+ "epoch": 0.75,
2529
+ "learning_rate": 0.00019244935350966514,
2530
+ "loss": 0.032,
2531
+ "step": 3710
2532
+ },
2533
+ {
2534
+ "epoch": 0.75,
2535
+ "learning_rate": 0.00019240911800276153,
2536
+ "loss": 0.0378,
2537
+ "step": 3720
2538
+ },
2539
+ {
2540
+ "epoch": 0.75,
2541
+ "learning_rate": 0.0001923687798069088,
2542
+ "loss": 0.044,
2543
+ "step": 3730
2544
+ },
2545
+ {
2546
+ "epoch": 0.75,
2547
+ "learning_rate": 0.00019232833896693242,
2548
+ "loss": 0.0363,
2549
+ "step": 3740
2550
+ },
2551
+ {
2552
+ "epoch": 0.75,
2553
+ "learning_rate": 0.00019228779552777202,
2554
+ "loss": 0.0412,
2555
+ "step": 3750
2556
+ },
2557
+ {
2558
+ "epoch": 0.76,
2559
+ "learning_rate": 0.0001922471495344812,
2560
+ "loss": 0.0376,
2561
+ "step": 3760
2562
+ },
2563
+ {
2564
+ "epoch": 0.76,
2565
+ "learning_rate": 0.0001922064010322275,
2566
+ "loss": 0.0451,
2567
+ "step": 3770
2568
+ },
2569
+ {
2570
+ "epoch": 0.76,
2571
+ "learning_rate": 0.00019216555006629237,
2572
+ "loss": 0.0443,
2573
+ "step": 3780
2574
+ },
2575
+ {
2576
+ "epoch": 0.76,
2577
+ "learning_rate": 0.0001921245966820712,
2578
+ "loss": 0.0394,
2579
+ "step": 3790
2580
+ },
2581
+ {
2582
+ "epoch": 0.77,
2583
+ "learning_rate": 0.00019208354092507305,
2584
+ "loss": 0.0472,
2585
+ "step": 3800
2586
+ },
2587
+ {
2588
+ "epoch": 0.77,
2589
+ "eval_loss": 0.06302132457494736,
2590
+ "eval_runtime": 1043.9496,
2591
+ "eval_samples_per_second": 7.048,
2592
+ "eval_steps_per_second": 1.763,
2593
+ "step": 3800
2594
+ },
2595
+ {
2596
+ "epoch": 0.77,
2597
+ "learning_rate": 0.00019204238284092093,
2598
+ "loss": 0.0418,
2599
+ "step": 3810
2600
+ },
2601
+ {
2602
+ "epoch": 0.77,
2603
+ "learning_rate": 0.00019200112247535141,
2604
+ "loss": 0.0416,
2605
+ "step": 3820
2606
+ },
2607
+ {
2608
+ "epoch": 0.77,
2609
+ "learning_rate": 0.00019195975987421472,
2610
+ "loss": 0.0418,
2611
+ "step": 3830
2612
+ },
2613
+ {
2614
+ "epoch": 0.77,
2615
+ "learning_rate": 0.00019191829508347481,
2616
+ "loss": 0.0394,
2617
+ "step": 3840
2618
+ },
2619
+ {
2620
+ "epoch": 0.78,
2621
+ "learning_rate": 0.00019187672814920912,
2622
+ "loss": 0.0405,
2623
+ "step": 3850
2624
+ },
2625
+ {
2626
+ "epoch": 0.78,
2627
+ "learning_rate": 0.00019183505911760855,
2628
+ "loss": 0.0306,
2629
+ "step": 3860
2630
+ },
2631
+ {
2632
+ "epoch": 0.78,
2633
+ "learning_rate": 0.00019179328803497754,
2634
+ "loss": 0.0443,
2635
+ "step": 3870
2636
+ },
2637
+ {
2638
+ "epoch": 0.78,
2639
+ "learning_rate": 0.0001917514149477339,
2640
+ "loss": 0.035,
2641
+ "step": 3880
2642
+ },
2643
+ {
2644
+ "epoch": 0.78,
2645
+ "learning_rate": 0.00019170943990240877,
2646
+ "loss": 0.0371,
2647
+ "step": 3890
2648
+ },
2649
+ {
2650
+ "epoch": 0.79,
2651
+ "learning_rate": 0.0001916673629456466,
2652
+ "loss": 0.0394,
2653
+ "step": 3900
2654
+ },
2655
+ {
2656
+ "epoch": 0.79,
2657
+ "eval_loss": 0.06118810176849365,
2658
+ "eval_runtime": 1046.2783,
2659
+ "eval_samples_per_second": 7.033,
2660
+ "eval_steps_per_second": 1.759,
2661
+ "step": 3900
2662
+ },
2663
+ {
2664
+ "epoch": 0.79,
2665
+ "learning_rate": 0.00019162518412420512,
2666
+ "loss": 0.0337,
2667
+ "step": 3910
2668
+ },
2669
+ {
2670
+ "epoch": 0.79,
2671
+ "learning_rate": 0.00019158290348495524,
2672
+ "loss": 0.0433,
2673
+ "step": 3920
2674
+ },
2675
+ {
2676
+ "epoch": 0.79,
2677
+ "learning_rate": 0.000191540521074881,
2678
+ "loss": 0.0402,
2679
+ "step": 3930
2680
+ },
2681
+ {
2682
+ "epoch": 0.79,
2683
+ "learning_rate": 0.0001914980369410795,
2684
+ "loss": 0.0326,
2685
+ "step": 3940
2686
+ },
2687
+ {
2688
+ "epoch": 0.8,
2689
+ "learning_rate": 0.00019145545113076096,
2690
+ "loss": 0.0501,
2691
+ "step": 3950
2692
+ },
2693
+ {
2694
+ "epoch": 0.8,
2695
+ "learning_rate": 0.00019141276369124855,
2696
+ "loss": 0.0448,
2697
+ "step": 3960
2698
+ },
2699
+ {
2700
+ "epoch": 0.8,
2701
+ "learning_rate": 0.00019136997466997837,
2702
+ "loss": 0.0403,
2703
+ "step": 3970
2704
+ },
2705
+ {
2706
+ "epoch": 0.8,
2707
+ "learning_rate": 0.00019132708411449936,
2708
+ "loss": 0.0427,
2709
+ "step": 3980
2710
+ },
2711
+ {
2712
+ "epoch": 0.8,
2713
+ "learning_rate": 0.0001912840920724734,
2714
+ "loss": 0.0322,
2715
+ "step": 3990
2716
+ },
2717
+ {
2718
+ "epoch": 0.81,
2719
+ "learning_rate": 0.00019124099859167503,
2720
+ "loss": 0.0471,
2721
+ "step": 4000
2722
+ },
2723
+ {
2724
+ "epoch": 0.81,
2725
+ "eval_loss": 0.06139204651117325,
2726
+ "eval_runtime": 1057.3652,
2727
+ "eval_samples_per_second": 6.959,
2728
+ "eval_steps_per_second": 1.74,
2729
+ "step": 4000
2730
+ },
2731
+ {
2732
+ "epoch": 0.81,
2733
+ "learning_rate": 0.00019119780371999162,
2734
+ "loss": 0.0306,
2735
+ "step": 4010
2736
+ },
2737
+ {
2738
+ "epoch": 0.81,
2739
+ "learning_rate": 0.00019115450750542304,
2740
+ "loss": 0.0352,
2741
+ "step": 4020
2742
+ },
2743
+ {
2744
+ "epoch": 0.81,
2745
+ "learning_rate": 0.000191111109996082,
2746
+ "loss": 0.0351,
2747
+ "step": 4030
2748
+ },
2749
+ {
2750
+ "epoch": 0.81,
2751
+ "learning_rate": 0.00019106761124019364,
2752
+ "loss": 0.0295,
2753
+ "step": 4040
2754
+ },
2755
+ {
2756
+ "epoch": 0.82,
2757
+ "learning_rate": 0.00019102401128609557,
2758
+ "loss": 0.0379,
2759
+ "step": 4050
2760
+ },
2761
+ {
2762
+ "epoch": 0.82,
2763
+ "learning_rate": 0.00019098031018223796,
2764
+ "loss": 0.048,
2765
+ "step": 4060
2766
+ },
2767
+ {
2768
+ "epoch": 0.82,
2769
+ "learning_rate": 0.00019093650797718338,
2770
+ "loss": 0.0454,
2771
+ "step": 4070
2772
+ },
2773
+ {
2774
+ "epoch": 0.82,
2775
+ "learning_rate": 0.00019089260471960663,
2776
+ "loss": 0.0463,
2777
+ "step": 4080
2778
+ },
2779
+ {
2780
+ "epoch": 0.82,
2781
+ "learning_rate": 0.0001908486004582949,
2782
+ "loss": 0.0447,
2783
+ "step": 4090
2784
+ },
2785
+ {
2786
+ "epoch": 0.83,
2787
+ "learning_rate": 0.00019080449524214762,
2788
+ "loss": 0.031,
2789
+ "step": 4100
2790
+ },
2791
+ {
2792
+ "epoch": 0.83,
2793
+ "eval_loss": 0.06018984317779541,
2794
+ "eval_runtime": 1049.9509,
2795
+ "eval_samples_per_second": 7.008,
2796
+ "eval_steps_per_second": 1.752,
2797
+ "step": 4100
2798
+ },
2799
+ {
2800
+ "epoch": 0.83,
2801
+ "learning_rate": 0.00019076028912017642,
2802
+ "loss": 0.0292,
2803
+ "step": 4110
2804
+ },
2805
+ {
2806
+ "epoch": 0.83,
2807
+ "learning_rate": 0.00019071598214150494,
2808
+ "loss": 0.0252,
2809
+ "step": 4120
2810
+ },
2811
+ {
2812
+ "epoch": 0.83,
2813
+ "learning_rate": 0.00019067157435536904,
2814
+ "loss": 0.0382,
2815
+ "step": 4130
2816
+ },
2817
+ {
2818
+ "epoch": 0.83,
2819
+ "learning_rate": 0.00019062706581111653,
2820
+ "loss": 0.0303,
2821
+ "step": 4140
2822
+ },
2823
+ {
2824
+ "epoch": 0.84,
2825
+ "learning_rate": 0.0001905824565582072,
2826
+ "loss": 0.0419,
2827
+ "step": 4150
2828
+ },
2829
+ {
2830
+ "epoch": 0.84,
2831
+ "learning_rate": 0.00019053774664621272,
2832
+ "loss": 0.0358,
2833
+ "step": 4160
2834
+ },
2835
+ {
2836
+ "epoch": 0.84,
2837
+ "learning_rate": 0.00019049293612481675,
2838
+ "loss": 0.037,
2839
+ "step": 4170
2840
+ },
2841
+ {
2842
+ "epoch": 0.84,
2843
+ "learning_rate": 0.00019044802504381453,
2844
+ "loss": 0.0306,
2845
+ "step": 4180
2846
+ },
2847
+ {
2848
+ "epoch": 0.84,
2849
+ "learning_rate": 0.00019040301345311326,
2850
+ "loss": 0.0431,
2851
+ "step": 4190
2852
+ },
2853
+ {
2854
+ "epoch": 0.85,
2855
+ "learning_rate": 0.0001903579014027317,
2856
+ "loss": 0.0405,
2857
+ "step": 4200
2858
+ },
2859
+ {
2860
+ "epoch": 0.85,
2861
+ "eval_loss": 0.05838855355978012,
2862
+ "eval_runtime": 1051.4215,
2863
+ "eval_samples_per_second": 6.998,
2864
+ "eval_steps_per_second": 1.75,
2865
+ "step": 4200
2866
+ },
2867
+ {
2868
+ "epoch": 0.85,
2869
+ "learning_rate": 0.00019031268894280023,
2870
+ "loss": 0.0334,
2871
+ "step": 4210
2872
+ },
2873
+ {
2874
+ "epoch": 0.85,
2875
+ "learning_rate": 0.00019026737612356094,
2876
+ "loss": 0.0382,
2877
+ "step": 4220
2878
+ },
2879
+ {
2880
+ "epoch": 0.85,
2881
+ "learning_rate": 0.00019022196299536733,
2882
+ "loss": 0.0371,
2883
+ "step": 4230
2884
+ },
2885
+ {
2886
+ "epoch": 0.85,
2887
+ "learning_rate": 0.00019017644960868445,
2888
+ "loss": 0.0353,
2889
+ "step": 4240
2890
+ },
2891
+ {
2892
+ "epoch": 0.86,
2893
+ "learning_rate": 0.00019013083601408863,
2894
+ "loss": 0.0368,
2895
+ "step": 4250
2896
+ },
2897
+ {
2898
+ "epoch": 0.86,
2899
+ "learning_rate": 0.0001900851222622677,
2900
+ "loss": 0.0427,
2901
+ "step": 4260
2902
+ },
2903
+ {
2904
+ "epoch": 0.86,
2905
+ "learning_rate": 0.00019003930840402072,
2906
+ "loss": 0.031,
2907
+ "step": 4270
2908
+ },
2909
+ {
2910
+ "epoch": 0.86,
2911
+ "learning_rate": 0.00018999339449025796,
2912
+ "loss": 0.0377,
2913
+ "step": 4280
2914
+ },
2915
+ {
2916
+ "epoch": 0.86,
2917
+ "learning_rate": 0.00018994738057200099,
2918
+ "loss": 0.0373,
2919
+ "step": 4290
2920
+ },
2921
+ {
2922
+ "epoch": 0.87,
2923
+ "learning_rate": 0.0001899012667003824,
2924
+ "loss": 0.0421,
2925
+ "step": 4300
2926
+ },
2927
+ {
2928
+ "epoch": 0.87,
2929
+ "eval_loss": 0.05914044752717018,
2930
+ "eval_runtime": 1056.4599,
2931
+ "eval_samples_per_second": 6.965,
2932
+ "eval_steps_per_second": 1.742,
2933
+ "step": 4300
2934
+ },
2935
+ {
2936
+ "epoch": 0.87,
2937
+ "learning_rate": 0.00018985505292664587,
2938
+ "loss": 0.0344,
2939
+ "step": 4310
2940
+ },
2941
+ {
2942
+ "epoch": 0.87,
2943
+ "learning_rate": 0.00018980873930214614,
2944
+ "loss": 0.0355,
2945
+ "step": 4320
2946
+ },
2947
+ {
2948
+ "epoch": 0.87,
2949
+ "learning_rate": 0.00018976232587834886,
2950
+ "loss": 0.0416,
2951
+ "step": 4330
2952
+ },
2953
+ {
2954
+ "epoch": 0.87,
2955
+ "learning_rate": 0.00018971581270683062,
2956
+ "loss": 0.0395,
2957
+ "step": 4340
2958
+ },
2959
+ {
2960
+ "epoch": 0.88,
2961
+ "learning_rate": 0.00018966919983927886,
2962
+ "loss": 0.0366,
2963
+ "step": 4350
2964
+ },
2965
+ {
2966
+ "epoch": 0.88,
2967
+ "learning_rate": 0.00018962248732749175,
2968
+ "loss": 0.0368,
2969
+ "step": 4360
2970
+ },
2971
+ {
2972
+ "epoch": 0.88,
2973
+ "learning_rate": 0.0001895756752233782,
2974
+ "loss": 0.0362,
2975
+ "step": 4370
2976
+ },
2977
+ {
2978
+ "epoch": 0.88,
2979
+ "learning_rate": 0.0001895287635789579,
2980
+ "loss": 0.0395,
2981
+ "step": 4380
2982
+ },
2983
+ {
2984
+ "epoch": 0.88,
2985
+ "learning_rate": 0.00018948175244636097,
2986
+ "loss": 0.0462,
2987
+ "step": 4390
2988
+ },
2989
+ {
2990
+ "epoch": 0.89,
2991
+ "learning_rate": 0.00018943464187782828,
2992
+ "loss": 0.0454,
2993
+ "step": 4400
2994
+ },
2995
+ {
2996
+ "epoch": 0.89,
2997
+ "eval_loss": 0.05928103253245354,
2998
+ "eval_runtime": 1029.6162,
2999
+ "eval_samples_per_second": 7.146,
3000
+ "eval_steps_per_second": 1.787,
3001
+ "step": 4400
3002
+ },
3003
+ {
3004
+ "epoch": 0.89,
3005
+ "learning_rate": 0.000189387431925711,
3006
+ "loss": 0.0326,
3007
+ "step": 4410
3008
+ },
3009
+ {
3010
+ "epoch": 0.89,
3011
+ "learning_rate": 0.0001893401226424709,
3012
+ "loss": 0.031,
3013
+ "step": 4420
3014
+ },
3015
+ {
3016
+ "epoch": 0.89,
3017
+ "learning_rate": 0.00018929271408068011,
3018
+ "loss": 0.0408,
3019
+ "step": 4430
3020
+ }
3021
+ ],
3022
+ "logging_steps": 10,
3023
+ "max_steps": 29802,
3024
+ "num_input_tokens_seen": 0,
3025
+ "num_train_epochs": 6,
3026
+ "save_steps": 10,
3027
+ "total_flos": 1.5320191612818555e+18,
3028
+ "train_batch_size": 4,
3029
+ "trial_name": null,
3030
+ "trial_params": null
3031
+ }
fact/checkpoint-4430/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a6b4f9b8b88e7273aa8b6980a1edceead9b52171cee239349e10dde5db66e6c
3
+ size 4920
fact/eval_results.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.89,
3
+ "eval_loss": 0.06257433444261551,
4
+ "eval_runtime": 534.6938,
5
+ "eval_samples_per_second": 13.761,
6
+ "eval_steps_per_second": 1.721
7
+ }
fact/runs/Jan27_07-40-28_platform/events.out.tfevents.1706341483.platform ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79fe331bd9b256b6386e2fe887f9276fc5311e93a89ebf75c0286adfecb870bc
3
+ size 38345
fact/runs/Jan27_13-44-49_platform/events.out.tfevents.1706363103.platform ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df0f67d8baffc729348f307e02ce1ad162ab19b3095d345e50a45029d95c0f1c
3
+ size 52987
fact/runs/Jan29_08-27-56_platform/events.out.tfevents.1706516887.platform ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a00c8b6d5083706633082f07d098717c5a44d8197824de23176c9e9b058e47f1
3
+ size 5320
fact/runs/Jan29_08-27-56_platform/events.out.tfevents.1706517427.platform ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d622c1a83d43a1db3cf97da07b8ca5fb90c04fdc4d411cb4f6ae41a837c38dcd
3
+ size 311
fact/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
fact/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
fact/tokenizer_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ }
29
+ },
30
+ "bos_token": "<s>",
31
+ "clean_up_tokenization_spaces": false,
32
+ "eos_token": "</s>",
33
+ "legacy": false,
34
+ "model_max_length": 4096,
35
+ "pad_token": "<unk>",
36
+ "padding_side": "right",
37
+ "sp_model_kwargs": {},
38
+ "spaces_between_special_tokens": false,
39
+ "split_special_tokens": false,
40
+ "tokenizer_class": "LlamaTokenizer",
41
+ "unk_token": "<unk>",
42
+ "use_default_system_prompt": false
43
+ }
fact/train_results.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.89,
3
+ "train_loss": 9.867495303766314e-06,
4
+ "train_runtime": 4.4906,
5
+ "train_samples_per_second": 15784.097,
6
+ "train_steps_per_second": 986.506
7
+ }
fact/trainer_log.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
fact/trainer_state.json ADDED
@@ -0,0 +1,3040 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.8920877793436682,
5
+ "eval_steps": 100,
6
+ "global_step": 4431,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 0.0001999999444378697,
14
+ "loss": 0.7972,
15
+ "step": 10
16
+ },
17
+ {
18
+ "epoch": 0.0,
19
+ "learning_rate": 0.0001999997777515405,
20
+ "loss": 0.221,
21
+ "step": 20
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 0.0001999994999411976,
26
+ "loss": 0.1313,
27
+ "step": 30
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 0.00019999911100714978,
32
+ "loss": 0.1169,
33
+ "step": 40
34
+ },
35
+ {
36
+ "epoch": 0.01,
37
+ "learning_rate": 0.00019999861094982923,
38
+ "loss": 0.1353,
39
+ "step": 50
40
+ },
41
+ {
42
+ "epoch": 0.01,
43
+ "learning_rate": 0.00019999799976979157,
44
+ "loss": 0.1355,
45
+ "step": 60
46
+ },
47
+ {
48
+ "epoch": 0.01,
49
+ "learning_rate": 0.00019999727746771603,
50
+ "loss": 0.0991,
51
+ "step": 70
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00019999644404440528,
56
+ "loss": 0.0995,
57
+ "step": 80
58
+ },
59
+ {
60
+ "epoch": 0.02,
61
+ "learning_rate": 0.00019999549950078536,
62
+ "loss": 0.1175,
63
+ "step": 90
64
+ },
65
+ {
66
+ "epoch": 0.02,
67
+ "learning_rate": 0.000199994443837906,
68
+ "loss": 0.0941,
69
+ "step": 100
70
+ },
71
+ {
72
+ "epoch": 0.02,
73
+ "eval_loss": 0.09952569007873535,
74
+ "eval_runtime": 659.9109,
75
+ "eval_samples_per_second": 11.15,
76
+ "eval_steps_per_second": 2.788,
77
+ "step": 100
78
+ },
79
+ {
80
+ "epoch": 0.02,
81
+ "learning_rate": 0.00019999327705694028,
82
+ "loss": 0.09,
83
+ "step": 110
84
+ },
85
+ {
86
+ "epoch": 0.02,
87
+ "learning_rate": 0.00019999199915918467,
88
+ "loss": 0.1044,
89
+ "step": 120
90
+ },
91
+ {
92
+ "epoch": 0.03,
93
+ "learning_rate": 0.00019999061014605934,
94
+ "loss": 0.0978,
95
+ "step": 130
96
+ },
97
+ {
98
+ "epoch": 0.03,
99
+ "learning_rate": 0.00019998911001910778,
100
+ "loss": 0.0963,
101
+ "step": 140
102
+ },
103
+ {
104
+ "epoch": 0.03,
105
+ "learning_rate": 0.000199987498779997,
106
+ "loss": 0.0955,
107
+ "step": 150
108
+ },
109
+ {
110
+ "epoch": 0.03,
111
+ "learning_rate": 0.00019998577643051744,
112
+ "loss": 0.0994,
113
+ "step": 160
114
+ },
115
+ {
116
+ "epoch": 0.03,
117
+ "learning_rate": 0.00019998394297258311,
118
+ "loss": 0.0921,
119
+ "step": 170
120
+ },
121
+ {
122
+ "epoch": 0.04,
123
+ "learning_rate": 0.0001999819984082314,
124
+ "loss": 0.087,
125
+ "step": 180
126
+ },
127
+ {
128
+ "epoch": 0.04,
129
+ "learning_rate": 0.00019997994273962316,
130
+ "loss": 0.0826,
131
+ "step": 190
132
+ },
133
+ {
134
+ "epoch": 0.04,
135
+ "learning_rate": 0.00019997777596904282,
136
+ "loss": 0.0903,
137
+ "step": 200
138
+ },
139
+ {
140
+ "epoch": 0.04,
141
+ "eval_loss": 0.090216726064682,
142
+ "eval_runtime": 649.064,
143
+ "eval_samples_per_second": 11.336,
144
+ "eval_steps_per_second": 2.835,
145
+ "step": 200
146
+ },
147
+ {
148
+ "epoch": 0.04,
149
+ "learning_rate": 0.00019997549809889811,
150
+ "loss": 0.0949,
151
+ "step": 210
152
+ },
153
+ {
154
+ "epoch": 0.04,
155
+ "learning_rate": 0.00019997310913172028,
156
+ "loss": 0.0902,
157
+ "step": 220
158
+ },
159
+ {
160
+ "epoch": 0.05,
161
+ "learning_rate": 0.00019997060907016415,
162
+ "loss": 0.0792,
163
+ "step": 230
164
+ },
165
+ {
166
+ "epoch": 0.05,
167
+ "learning_rate": 0.0001999679979170078,
168
+ "loss": 0.0901,
169
+ "step": 240
170
+ },
171
+ {
172
+ "epoch": 0.05,
173
+ "learning_rate": 0.0001999652756751529,
174
+ "loss": 0.1035,
175
+ "step": 250
176
+ },
177
+ {
178
+ "epoch": 0.05,
179
+ "learning_rate": 0.00019996244234762452,
180
+ "loss": 0.0875,
181
+ "step": 260
182
+ },
183
+ {
184
+ "epoch": 0.05,
185
+ "learning_rate": 0.00019995949793757118,
186
+ "loss": 0.0718,
187
+ "step": 270
188
+ },
189
+ {
190
+ "epoch": 0.06,
191
+ "learning_rate": 0.00019995644244826481,
192
+ "loss": 0.0799,
193
+ "step": 280
194
+ },
195
+ {
196
+ "epoch": 0.06,
197
+ "learning_rate": 0.0001999532758831008,
198
+ "loss": 0.0767,
199
+ "step": 290
200
+ },
201
+ {
202
+ "epoch": 0.06,
203
+ "learning_rate": 0.000199949998245598,
204
+ "loss": 0.089,
205
+ "step": 300
206
+ },
207
+ {
208
+ "epoch": 0.06,
209
+ "eval_loss": 0.08015166223049164,
210
+ "eval_runtime": 646.37,
211
+ "eval_samples_per_second": 11.384,
212
+ "eval_steps_per_second": 2.847,
213
+ "step": 300
214
+ },
215
+ {
216
+ "epoch": 0.06,
217
+ "learning_rate": 0.00019994660953939864,
218
+ "loss": 0.0833,
219
+ "step": 310
220
+ },
221
+ {
222
+ "epoch": 0.06,
223
+ "learning_rate": 0.00019994310976826837,
224
+ "loss": 0.0799,
225
+ "step": 320
226
+ },
227
+ {
228
+ "epoch": 0.07,
229
+ "learning_rate": 0.00019993949893609635,
230
+ "loss": 0.0747,
231
+ "step": 330
232
+ },
233
+ {
234
+ "epoch": 0.07,
235
+ "learning_rate": 0.00019993577704689505,
236
+ "loss": 0.0888,
237
+ "step": 340
238
+ },
239
+ {
240
+ "epoch": 0.07,
241
+ "learning_rate": 0.0001999319441048004,
242
+ "loss": 0.0872,
243
+ "step": 350
244
+ },
245
+ {
246
+ "epoch": 0.07,
247
+ "learning_rate": 0.00019992800011407172,
248
+ "loss": 0.0762,
249
+ "step": 360
250
+ },
251
+ {
252
+ "epoch": 0.07,
253
+ "learning_rate": 0.00019992394507909174,
254
+ "loss": 0.0746,
255
+ "step": 370
256
+ },
257
+ {
258
+ "epoch": 0.08,
259
+ "learning_rate": 0.00019991977900436658,
260
+ "loss": 0.0713,
261
+ "step": 380
262
+ },
263
+ {
264
+ "epoch": 0.08,
265
+ "learning_rate": 0.0001999155018945258,
266
+ "loss": 0.077,
267
+ "step": 390
268
+ },
269
+ {
270
+ "epoch": 0.08,
271
+ "learning_rate": 0.00019991111375432225,
272
+ "loss": 0.0619,
273
+ "step": 400
274
+ },
275
+ {
276
+ "epoch": 0.08,
277
+ "eval_loss": 0.07623383402824402,
278
+ "eval_runtime": 654.0144,
279
+ "eval_samples_per_second": 11.251,
280
+ "eval_steps_per_second": 2.813,
281
+ "step": 400
282
+ },
283
+ {
284
+ "epoch": 0.08,
285
+ "learning_rate": 0.00019990661458863226,
286
+ "loss": 0.0702,
287
+ "step": 410
288
+ },
289
+ {
290
+ "epoch": 0.08,
291
+ "learning_rate": 0.0001999020044024555,
292
+ "loss": 0.0725,
293
+ "step": 420
294
+ },
295
+ {
296
+ "epoch": 0.09,
297
+ "learning_rate": 0.00019989728320091496,
298
+ "loss": 0.065,
299
+ "step": 430
300
+ },
301
+ {
302
+ "epoch": 0.09,
303
+ "learning_rate": 0.00019989245098925708,
304
+ "loss": 0.0632,
305
+ "step": 440
306
+ },
307
+ {
308
+ "epoch": 0.09,
309
+ "learning_rate": 0.00019988750777285162,
310
+ "loss": 0.0762,
311
+ "step": 450
312
+ },
313
+ {
314
+ "epoch": 0.09,
315
+ "learning_rate": 0.00019988245355719166,
316
+ "loss": 0.0736,
317
+ "step": 460
318
+ },
319
+ {
320
+ "epoch": 0.09,
321
+ "learning_rate": 0.0001998772883478937,
322
+ "loss": 0.0662,
323
+ "step": 470
324
+ },
325
+ {
326
+ "epoch": 0.1,
327
+ "learning_rate": 0.0001998720121506975,
328
+ "loss": 0.0765,
329
+ "step": 480
330
+ },
331
+ {
332
+ "epoch": 0.1,
333
+ "learning_rate": 0.00019986662497146622,
334
+ "loss": 0.0694,
335
+ "step": 490
336
+ },
337
+ {
338
+ "epoch": 0.1,
339
+ "learning_rate": 0.00019986112681618634,
340
+ "loss": 0.0722,
341
+ "step": 500
342
+ },
343
+ {
344
+ "epoch": 0.1,
345
+ "eval_loss": 0.075162373483181,
346
+ "eval_runtime": 656.2929,
347
+ "eval_samples_per_second": 11.211,
348
+ "eval_steps_per_second": 2.804,
349
+ "step": 500
350
+ },
351
+ {
352
+ "epoch": 0.1,
353
+ "learning_rate": 0.0001998555176909676,
354
+ "loss": 0.073,
355
+ "step": 510
356
+ },
357
+ {
358
+ "epoch": 0.1,
359
+ "learning_rate": 0.00019984979760204313,
360
+ "loss": 0.0681,
361
+ "step": 520
362
+ },
363
+ {
364
+ "epoch": 0.11,
365
+ "learning_rate": 0.00019984396655576932,
366
+ "loss": 0.057,
367
+ "step": 530
368
+ },
369
+ {
370
+ "epoch": 0.11,
371
+ "learning_rate": 0.0001998380245586259,
372
+ "loss": 0.0689,
373
+ "step": 540
374
+ },
375
+ {
376
+ "epoch": 0.11,
377
+ "learning_rate": 0.00019983197161721583,
378
+ "loss": 0.057,
379
+ "step": 550
380
+ },
381
+ {
382
+ "epoch": 0.11,
383
+ "learning_rate": 0.00019982580773826545,
384
+ "loss": 0.0693,
385
+ "step": 560
386
+ },
387
+ {
388
+ "epoch": 0.11,
389
+ "learning_rate": 0.00019981953292862428,
390
+ "loss": 0.0702,
391
+ "step": 570
392
+ },
393
+ {
394
+ "epoch": 0.12,
395
+ "learning_rate": 0.0001998131471952652,
396
+ "loss": 0.0681,
397
+ "step": 580
398
+ },
399
+ {
400
+ "epoch": 0.12,
401
+ "learning_rate": 0.00019980665054528425,
402
+ "loss": 0.0657,
403
+ "step": 590
404
+ },
405
+ {
406
+ "epoch": 0.12,
407
+ "learning_rate": 0.00019980004298590081,
408
+ "loss": 0.0724,
409
+ "step": 600
410
+ },
411
+ {
412
+ "epoch": 0.12,
413
+ "eval_loss": 0.0736309364438057,
414
+ "eval_runtime": 657.9967,
415
+ "eval_samples_per_second": 11.182,
416
+ "eval_steps_per_second": 2.796,
417
+ "step": 600
418
+ },
419
+ {
420
+ "epoch": 0.12,
421
+ "learning_rate": 0.00019979332452445752,
422
+ "loss": 0.0599,
423
+ "step": 610
424
+ },
425
+ {
426
+ "epoch": 0.12,
427
+ "learning_rate": 0.00019978649516842016,
428
+ "loss": 0.0697,
429
+ "step": 620
430
+ },
431
+ {
432
+ "epoch": 0.13,
433
+ "learning_rate": 0.00019977955492537787,
434
+ "loss": 0.0715,
435
+ "step": 630
436
+ },
437
+ {
438
+ "epoch": 0.13,
439
+ "learning_rate": 0.00019977250380304287,
440
+ "loss": 0.0589,
441
+ "step": 640
442
+ },
443
+ {
444
+ "epoch": 0.13,
445
+ "learning_rate": 0.0001997653418092507,
446
+ "loss": 0.0722,
447
+ "step": 650
448
+ },
449
+ {
450
+ "epoch": 0.13,
451
+ "learning_rate": 0.00019975806895196008,
452
+ "loss": 0.0611,
453
+ "step": 660
454
+ },
455
+ {
456
+ "epoch": 0.13,
457
+ "learning_rate": 0.0001997506852392529,
458
+ "loss": 0.056,
459
+ "step": 670
460
+ },
461
+ {
462
+ "epoch": 0.14,
463
+ "learning_rate": 0.0001997431906793343,
464
+ "loss": 0.0614,
465
+ "step": 680
466
+ },
467
+ {
468
+ "epoch": 0.14,
469
+ "learning_rate": 0.00019973558528053252,
470
+ "loss": 0.0537,
471
+ "step": 690
472
+ },
473
+ {
474
+ "epoch": 0.14,
475
+ "learning_rate": 0.000199727869051299,
476
+ "loss": 0.0762,
477
+ "step": 700
478
+ },
479
+ {
480
+ "epoch": 0.14,
481
+ "eval_loss": 0.07101555913686752,
482
+ "eval_runtime": 665.8897,
483
+ "eval_samples_per_second": 11.05,
484
+ "eval_steps_per_second": 2.763,
485
+ "step": 700
486
+ },
487
+ {
488
+ "epoch": 0.14,
489
+ "learning_rate": 0.00019972004200020832,
490
+ "loss": 0.0634,
491
+ "step": 710
492
+ },
493
+ {
494
+ "epoch": 0.14,
495
+ "learning_rate": 0.0001997121041359583,
496
+ "loss": 0.0697,
497
+ "step": 720
498
+ },
499
+ {
500
+ "epoch": 0.15,
501
+ "learning_rate": 0.00019970405546736978,
502
+ "loss": 0.0605,
503
+ "step": 730
504
+ },
505
+ {
506
+ "epoch": 0.15,
507
+ "learning_rate": 0.00019969589600338678,
508
+ "loss": 0.0598,
509
+ "step": 740
510
+ },
511
+ {
512
+ "epoch": 0.15,
513
+ "learning_rate": 0.00019968762575307649,
514
+ "loss": 0.066,
515
+ "step": 750
516
+ },
517
+ {
518
+ "epoch": 0.15,
519
+ "learning_rate": 0.00019967924472562914,
520
+ "loss": 0.0656,
521
+ "step": 760
522
+ },
523
+ {
524
+ "epoch": 0.16,
525
+ "learning_rate": 0.00019967075293035805,
526
+ "loss": 0.0614,
527
+ "step": 770
528
+ },
529
+ {
530
+ "epoch": 0.16,
531
+ "learning_rate": 0.0001996621503766997,
532
+ "loss": 0.0633,
533
+ "step": 780
534
+ },
535
+ {
536
+ "epoch": 0.16,
537
+ "learning_rate": 0.00019965343707421362,
538
+ "loss": 0.0661,
539
+ "step": 790
540
+ },
541
+ {
542
+ "epoch": 0.16,
543
+ "learning_rate": 0.00019964461303258242,
544
+ "loss": 0.0567,
545
+ "step": 800
546
+ },
547
+ {
548
+ "epoch": 0.16,
549
+ "eval_loss": 0.07119747996330261,
550
+ "eval_runtime": 657.3478,
551
+ "eval_samples_per_second": 11.193,
552
+ "eval_steps_per_second": 2.799,
553
+ "step": 800
554
+ },
555
+ {
556
+ "epoch": 0.16,
557
+ "learning_rate": 0.00019963567826161168,
558
+ "loss": 0.0577,
559
+ "step": 810
560
+ },
561
+ {
562
+ "epoch": 0.17,
563
+ "learning_rate": 0.00019962663277123016,
564
+ "loss": 0.0633,
565
+ "step": 820
566
+ },
567
+ {
568
+ "epoch": 0.17,
569
+ "learning_rate": 0.0001996174765714896,
570
+ "loss": 0.0701,
571
+ "step": 830
572
+ },
573
+ {
574
+ "epoch": 0.17,
575
+ "learning_rate": 0.0001996082096725647,
576
+ "loss": 0.0715,
577
+ "step": 840
578
+ },
579
+ {
580
+ "epoch": 0.17,
581
+ "learning_rate": 0.0001995988320847533,
582
+ "loss": 0.0618,
583
+ "step": 850
584
+ },
585
+ {
586
+ "epoch": 0.17,
587
+ "learning_rate": 0.00019958934381847612,
588
+ "loss": 0.0589,
589
+ "step": 860
590
+ },
591
+ {
592
+ "epoch": 0.18,
593
+ "learning_rate": 0.00019957974488427698,
594
+ "loss": 0.0658,
595
+ "step": 870
596
+ },
597
+ {
598
+ "epoch": 0.18,
599
+ "learning_rate": 0.00019957003529282255,
600
+ "loss": 0.0549,
601
+ "step": 880
602
+ },
603
+ {
604
+ "epoch": 0.18,
605
+ "learning_rate": 0.00019956021505490262,
606
+ "loss": 0.0664,
607
+ "step": 890
608
+ },
609
+ {
610
+ "epoch": 0.18,
611
+ "learning_rate": 0.0001995502841814298,
612
+ "loss": 0.0578,
613
+ "step": 900
614
+ },
615
+ {
616
+ "epoch": 0.18,
617
+ "eval_loss": 0.06790292263031006,
618
+ "eval_runtime": 663.3404,
619
+ "eval_samples_per_second": 11.092,
620
+ "eval_steps_per_second": 2.774,
621
+ "step": 900
622
+ },
623
+ {
624
+ "epoch": 0.18,
625
+ "learning_rate": 0.00019954024268343975,
626
+ "loss": 0.0626,
627
+ "step": 910
628
+ },
629
+ {
630
+ "epoch": 0.19,
631
+ "learning_rate": 0.000199530090572091,
632
+ "loss": 0.0621,
633
+ "step": 920
634
+ },
635
+ {
636
+ "epoch": 0.19,
637
+ "learning_rate": 0.00019951982785866492,
638
+ "loss": 0.0527,
639
+ "step": 930
640
+ },
641
+ {
642
+ "epoch": 0.19,
643
+ "learning_rate": 0.00019950945455456597,
644
+ "loss": 0.0636,
645
+ "step": 940
646
+ },
647
+ {
648
+ "epoch": 0.19,
649
+ "learning_rate": 0.00019949897067132142,
650
+ "loss": 0.0767,
651
+ "step": 950
652
+ },
653
+ {
654
+ "epoch": 0.19,
655
+ "learning_rate": 0.00019948837622058135,
656
+ "loss": 0.0785,
657
+ "step": 960
658
+ },
659
+ {
660
+ "epoch": 0.2,
661
+ "learning_rate": 0.00019947767121411878,
662
+ "loss": 0.0698,
663
+ "step": 970
664
+ },
665
+ {
666
+ "epoch": 0.2,
667
+ "learning_rate": 0.0001994668556638296,
668
+ "loss": 0.0576,
669
+ "step": 980
670
+ },
671
+ {
672
+ "epoch": 0.2,
673
+ "learning_rate": 0.00019945592958173247,
674
+ "loss": 0.0616,
675
+ "step": 990
676
+ },
677
+ {
678
+ "epoch": 0.2,
679
+ "learning_rate": 0.00019944489297996892,
680
+ "loss": 0.0597,
681
+ "step": 1000
682
+ },
683
+ {
684
+ "epoch": 0.2,
685
+ "eval_loss": 0.06911394000053406,
686
+ "eval_runtime": 654.3952,
687
+ "eval_samples_per_second": 11.244,
688
+ "eval_steps_per_second": 2.812,
689
+ "step": 1000
690
+ },
691
+ {
692
+ "epoch": 0.2,
693
+ "learning_rate": 0.00019943374587080333,
694
+ "loss": 0.0674,
695
+ "step": 1010
696
+ },
697
+ {
698
+ "epoch": 0.21,
699
+ "learning_rate": 0.0001994224882666228,
700
+ "loss": 0.0528,
701
+ "step": 1020
702
+ },
703
+ {
704
+ "epoch": 0.21,
705
+ "learning_rate": 0.0001994111201799373,
706
+ "loss": 0.0602,
707
+ "step": 1030
708
+ },
709
+ {
710
+ "epoch": 0.21,
711
+ "learning_rate": 0.00019939964162337946,
712
+ "loss": 0.0568,
713
+ "step": 1040
714
+ },
715
+ {
716
+ "epoch": 0.21,
717
+ "learning_rate": 0.00019938805260970486,
718
+ "loss": 0.0571,
719
+ "step": 1050
720
+ },
721
+ {
722
+ "epoch": 0.21,
723
+ "learning_rate": 0.0001993763531517916,
724
+ "loss": 0.0598,
725
+ "step": 1060
726
+ },
727
+ {
728
+ "epoch": 0.22,
729
+ "learning_rate": 0.00019936454326264068,
730
+ "loss": 0.0636,
731
+ "step": 1070
732
+ },
733
+ {
734
+ "epoch": 0.22,
735
+ "learning_rate": 0.00019935262295537568,
736
+ "loss": 0.0602,
737
+ "step": 1080
738
+ },
739
+ {
740
+ "epoch": 0.22,
741
+ "learning_rate": 0.00019934059224324303,
742
+ "loss": 0.0763,
743
+ "step": 1090
744
+ },
745
+ {
746
+ "epoch": 0.22,
747
+ "learning_rate": 0.00019932845113961172,
748
+ "loss": 0.0538,
749
+ "step": 1100
750
+ },
751
+ {
752
+ "epoch": 0.22,
753
+ "eval_loss": 0.06597457081079483,
754
+ "eval_runtime": 653.2731,
755
+ "eval_samples_per_second": 11.263,
756
+ "eval_steps_per_second": 2.817,
757
+ "step": 1100
758
+ },
759
+ {
760
+ "epoch": 0.22,
761
+ "learning_rate": 0.0001993161996579735,
762
+ "loss": 0.0578,
763
+ "step": 1110
764
+ },
765
+ {
766
+ "epoch": 0.23,
767
+ "learning_rate": 0.00019930383781194272,
768
+ "loss": 0.0561,
769
+ "step": 1120
770
+ },
771
+ {
772
+ "epoch": 0.23,
773
+ "learning_rate": 0.00019929136561525637,
774
+ "loss": 0.0558,
775
+ "step": 1130
776
+ },
777
+ {
778
+ "epoch": 0.23,
779
+ "learning_rate": 0.0001992787830817741,
780
+ "loss": 0.0669,
781
+ "step": 1140
782
+ },
783
+ {
784
+ "epoch": 0.23,
785
+ "learning_rate": 0.00019926609022547818,
786
+ "loss": 0.0616,
787
+ "step": 1150
788
+ },
789
+ {
790
+ "epoch": 0.23,
791
+ "learning_rate": 0.00019925328706047344,
792
+ "loss": 0.0637,
793
+ "step": 1160
794
+ },
795
+ {
796
+ "epoch": 0.24,
797
+ "learning_rate": 0.00019924037360098727,
798
+ "loss": 0.052,
799
+ "step": 1170
800
+ },
801
+ {
802
+ "epoch": 0.24,
803
+ "learning_rate": 0.0001992273498613697,
804
+ "loss": 0.0548,
805
+ "step": 1180
806
+ },
807
+ {
808
+ "epoch": 0.24,
809
+ "learning_rate": 0.00019921421585609325,
810
+ "loss": 0.0545,
811
+ "step": 1190
812
+ },
813
+ {
814
+ "epoch": 0.24,
815
+ "learning_rate": 0.000199200971599753,
816
+ "loss": 0.0498,
817
+ "step": 1200
818
+ },
819
+ {
820
+ "epoch": 0.24,
821
+ "eval_loss": 0.06580409407615662,
822
+ "eval_runtime": 655.2658,
823
+ "eval_samples_per_second": 11.229,
824
+ "eval_steps_per_second": 2.808,
825
+ "step": 1200
826
+ },
827
+ {
828
+ "epoch": 0.24,
829
+ "learning_rate": 0.00019918761710706649,
830
+ "loss": 0.0537,
831
+ "step": 1210
832
+ },
833
+ {
834
+ "epoch": 0.25,
835
+ "learning_rate": 0.00019917415239287386,
836
+ "loss": 0.0579,
837
+ "step": 1220
838
+ },
839
+ {
840
+ "epoch": 0.25,
841
+ "learning_rate": 0.00019916057747213758,
842
+ "loss": 0.053,
843
+ "step": 1230
844
+ },
845
+ {
846
+ "epoch": 0.25,
847
+ "learning_rate": 0.00019914689235994278,
848
+ "loss": 0.0562,
849
+ "step": 1240
850
+ },
851
+ {
852
+ "epoch": 0.25,
853
+ "learning_rate": 0.00019913309707149694,
854
+ "loss": 0.0595,
855
+ "step": 1250
856
+ },
857
+ {
858
+ "epoch": 0.25,
859
+ "learning_rate": 0.00019911919162212988,
860
+ "loss": 0.0515,
861
+ "step": 1260
862
+ },
863
+ {
864
+ "epoch": 0.26,
865
+ "learning_rate": 0.000199105176027294,
866
+ "loss": 0.053,
867
+ "step": 1270
868
+ },
869
+ {
870
+ "epoch": 0.26,
871
+ "learning_rate": 0.00019909105030256402,
872
+ "loss": 0.065,
873
+ "step": 1280
874
+ },
875
+ {
876
+ "epoch": 0.26,
877
+ "learning_rate": 0.000199076814463637,
878
+ "loss": 0.0574,
879
+ "step": 1290
880
+ },
881
+ {
882
+ "epoch": 0.26,
883
+ "learning_rate": 0.0001990624685263325,
884
+ "loss": 0.0595,
885
+ "step": 1300
886
+ },
887
+ {
888
+ "epoch": 0.26,
889
+ "eval_loss": 0.06419895589351654,
890
+ "eval_runtime": 662.3031,
891
+ "eval_samples_per_second": 11.11,
892
+ "eval_steps_per_second": 2.778,
893
+ "step": 1300
894
+ },
895
+ {
896
+ "epoch": 0.26,
897
+ "learning_rate": 0.00019904801250659223,
898
+ "loss": 0.0667,
899
+ "step": 1310
900
+ },
901
+ {
902
+ "epoch": 0.27,
903
+ "learning_rate": 0.00019903344642048043,
904
+ "loss": 0.0562,
905
+ "step": 1320
906
+ },
907
+ {
908
+ "epoch": 0.27,
909
+ "learning_rate": 0.00019901877028418347,
910
+ "loss": 0.0604,
911
+ "step": 1330
912
+ },
913
+ {
914
+ "epoch": 0.27,
915
+ "learning_rate": 0.00019900398411401015,
916
+ "loss": 0.0474,
917
+ "step": 1340
918
+ },
919
+ {
920
+ "epoch": 0.27,
921
+ "learning_rate": 0.0001989890879263915,
922
+ "loss": 0.0444,
923
+ "step": 1350
924
+ },
925
+ {
926
+ "epoch": 0.27,
927
+ "learning_rate": 0.0001989740817378808,
928
+ "loss": 0.0522,
929
+ "step": 1360
930
+ },
931
+ {
932
+ "epoch": 0.28,
933
+ "learning_rate": 0.00019895896556515352,
934
+ "loss": 0.0543,
935
+ "step": 1370
936
+ },
937
+ {
938
+ "epoch": 0.28,
939
+ "learning_rate": 0.00019894373942500743,
940
+ "loss": 0.0447,
941
+ "step": 1380
942
+ },
943
+ {
944
+ "epoch": 0.28,
945
+ "learning_rate": 0.00019892840333436245,
946
+ "loss": 0.0454,
947
+ "step": 1390
948
+ },
949
+ {
950
+ "epoch": 0.28,
951
+ "learning_rate": 0.00019891295731026075,
952
+ "loss": 0.0465,
953
+ "step": 1400
954
+ },
955
+ {
956
+ "epoch": 0.28,
957
+ "eval_loss": 0.0676579549908638,
958
+ "eval_runtime": 660.0804,
959
+ "eval_samples_per_second": 11.147,
960
+ "eval_steps_per_second": 2.788,
961
+ "step": 1400
962
+ },
963
+ {
964
+ "epoch": 0.28,
965
+ "learning_rate": 0.0001988974013698665,
966
+ "loss": 0.0533,
967
+ "step": 1410
968
+ },
969
+ {
970
+ "epoch": 0.29,
971
+ "learning_rate": 0.00019888173553046624,
972
+ "loss": 0.0522,
973
+ "step": 1420
974
+ },
975
+ {
976
+ "epoch": 0.29,
977
+ "learning_rate": 0.00019886595980946842,
978
+ "loss": 0.0592,
979
+ "step": 1430
980
+ },
981
+ {
982
+ "epoch": 0.29,
983
+ "learning_rate": 0.00019885007422440379,
984
+ "loss": 0.0584,
985
+ "step": 1440
986
+ },
987
+ {
988
+ "epoch": 0.29,
989
+ "learning_rate": 0.00019883407879292498,
990
+ "loss": 0.054,
991
+ "step": 1450
992
+ },
993
+ {
994
+ "epoch": 0.29,
995
+ "learning_rate": 0.00019881797353280695,
996
+ "loss": 0.0442,
997
+ "step": 1460
998
+ },
999
+ {
1000
+ "epoch": 0.3,
1001
+ "learning_rate": 0.00019880175846194638,
1002
+ "loss": 0.0659,
1003
+ "step": 1470
1004
+ },
1005
+ {
1006
+ "epoch": 0.3,
1007
+ "learning_rate": 0.00019878543359836223,
1008
+ "loss": 0.0483,
1009
+ "step": 1480
1010
+ },
1011
+ {
1012
+ "epoch": 0.3,
1013
+ "learning_rate": 0.0001987689989601954,
1014
+ "loss": 0.0525,
1015
+ "step": 1490
1016
+ },
1017
+ {
1018
+ "epoch": 0.3,
1019
+ "learning_rate": 0.00019875245456570873,
1020
+ "loss": 0.0533,
1021
+ "step": 1500
1022
+ },
1023
+ {
1024
+ "epoch": 0.3,
1025
+ "eval_loss": 0.06512827426195145,
1026
+ "eval_runtime": 659.6413,
1027
+ "eval_samples_per_second": 11.155,
1028
+ "eval_steps_per_second": 2.789,
1029
+ "step": 1500
1030
+ },
1031
+ {
1032
+ "epoch": 0.3,
1033
+ "learning_rate": 0.00019873580043328705,
1034
+ "loss": 0.0438,
1035
+ "step": 1510
1036
+ },
1037
+ {
1038
+ "epoch": 0.31,
1039
+ "learning_rate": 0.00019871903658143719,
1040
+ "loss": 0.0542,
1041
+ "step": 1520
1042
+ },
1043
+ {
1044
+ "epoch": 0.31,
1045
+ "learning_rate": 0.0001987021630287878,
1046
+ "loss": 0.0555,
1047
+ "step": 1530
1048
+ },
1049
+ {
1050
+ "epoch": 0.31,
1051
+ "learning_rate": 0.00019868517979408954,
1052
+ "loss": 0.0561,
1053
+ "step": 1540
1054
+ },
1055
+ {
1056
+ "epoch": 0.31,
1057
+ "learning_rate": 0.00019866808689621482,
1058
+ "loss": 0.0517,
1059
+ "step": 1550
1060
+ },
1061
+ {
1062
+ "epoch": 0.31,
1063
+ "learning_rate": 0.0001986508843541581,
1064
+ "loss": 0.051,
1065
+ "step": 1560
1066
+ },
1067
+ {
1068
+ "epoch": 0.32,
1069
+ "learning_rate": 0.0001986335721870355,
1070
+ "loss": 0.0525,
1071
+ "step": 1570
1072
+ },
1073
+ {
1074
+ "epoch": 0.32,
1075
+ "learning_rate": 0.0001986161504140851,
1076
+ "loss": 0.0443,
1077
+ "step": 1580
1078
+ },
1079
+ {
1080
+ "epoch": 0.32,
1081
+ "learning_rate": 0.00019859861905466668,
1082
+ "loss": 0.0542,
1083
+ "step": 1590
1084
+ },
1085
+ {
1086
+ "epoch": 0.32,
1087
+ "learning_rate": 0.00019858097812826183,
1088
+ "loss": 0.0593,
1089
+ "step": 1600
1090
+ },
1091
+ {
1092
+ "epoch": 0.32,
1093
+ "eval_loss": 0.06405352801084518,
1094
+ "eval_runtime": 658.5179,
1095
+ "eval_samples_per_second": 11.174,
1096
+ "eval_steps_per_second": 2.794,
1097
+ "step": 1600
1098
+ },
1099
+ {
1100
+ "epoch": 0.32,
1101
+ "learning_rate": 0.0001985632276544739,
1102
+ "loss": 0.0527,
1103
+ "step": 1610
1104
+ },
1105
+ {
1106
+ "epoch": 0.33,
1107
+ "learning_rate": 0.000198545367653028,
1108
+ "loss": 0.0422,
1109
+ "step": 1620
1110
+ },
1111
+ {
1112
+ "epoch": 0.33,
1113
+ "learning_rate": 0.00019852739814377087,
1114
+ "loss": 0.0533,
1115
+ "step": 1630
1116
+ },
1117
+ {
1118
+ "epoch": 0.33,
1119
+ "learning_rate": 0.00019850931914667107,
1120
+ "loss": 0.0507,
1121
+ "step": 1640
1122
+ },
1123
+ {
1124
+ "epoch": 0.33,
1125
+ "learning_rate": 0.00019849113068181869,
1126
+ "loss": 0.0581,
1127
+ "step": 1650
1128
+ },
1129
+ {
1130
+ "epoch": 0.33,
1131
+ "learning_rate": 0.00019847283276942554,
1132
+ "loss": 0.0583,
1133
+ "step": 1660
1134
+ },
1135
+ {
1136
+ "epoch": 0.34,
1137
+ "learning_rate": 0.00019845442542982506,
1138
+ "loss": 0.0562,
1139
+ "step": 1670
1140
+ },
1141
+ {
1142
+ "epoch": 0.34,
1143
+ "learning_rate": 0.00019843590868347225,
1144
+ "loss": 0.0499,
1145
+ "step": 1680
1146
+ },
1147
+ {
1148
+ "epoch": 0.34,
1149
+ "learning_rate": 0.00019841728255094374,
1150
+ "loss": 0.0543,
1151
+ "step": 1690
1152
+ },
1153
+ {
1154
+ "epoch": 0.34,
1155
+ "learning_rate": 0.00019839854705293764,
1156
+ "loss": 0.055,
1157
+ "step": 1700
1158
+ },
1159
+ {
1160
+ "epoch": 0.34,
1161
+ "eval_loss": 0.0653354823589325,
1162
+ "eval_runtime": 674.0253,
1163
+ "eval_samples_per_second": 10.917,
1164
+ "eval_steps_per_second": 2.73,
1165
+ "step": 1700
1166
+ },
1167
+ {
1168
+ "epoch": 0.34,
1169
+ "learning_rate": 0.00019837970221027365,
1170
+ "loss": 0.0422,
1171
+ "step": 1710
1172
+ },
1173
+ {
1174
+ "epoch": 0.35,
1175
+ "learning_rate": 0.00019836074804389296,
1176
+ "loss": 0.0586,
1177
+ "step": 1720
1178
+ },
1179
+ {
1180
+ "epoch": 0.35,
1181
+ "learning_rate": 0.00019834168457485824,
1182
+ "loss": 0.0408,
1183
+ "step": 1730
1184
+ },
1185
+ {
1186
+ "epoch": 0.35,
1187
+ "learning_rate": 0.00019832251182435367,
1188
+ "loss": 0.0587,
1189
+ "step": 1740
1190
+ },
1191
+ {
1192
+ "epoch": 0.35,
1193
+ "learning_rate": 0.00019830322981368478,
1194
+ "loss": 0.0498,
1195
+ "step": 1750
1196
+ },
1197
+ {
1198
+ "epoch": 0.35,
1199
+ "learning_rate": 0.0001982838385642786,
1200
+ "loss": 0.0475,
1201
+ "step": 1760
1202
+ },
1203
+ {
1204
+ "epoch": 0.36,
1205
+ "learning_rate": 0.00019826433809768345,
1206
+ "loss": 0.0571,
1207
+ "step": 1770
1208
+ },
1209
+ {
1210
+ "epoch": 0.36,
1211
+ "learning_rate": 0.00019824472843556914,
1212
+ "loss": 0.0512,
1213
+ "step": 1780
1214
+ },
1215
+ {
1216
+ "epoch": 0.36,
1217
+ "learning_rate": 0.00019822500959972673,
1218
+ "loss": 0.0477,
1219
+ "step": 1790
1220
+ },
1221
+ {
1222
+ "epoch": 0.36,
1223
+ "learning_rate": 0.00019820518161206864,
1224
+ "loss": 0.0546,
1225
+ "step": 1800
1226
+ },
1227
+ {
1228
+ "epoch": 0.36,
1229
+ "eval_loss": 0.06339588016271591,
1230
+ "eval_runtime": 1055.2159,
1231
+ "eval_samples_per_second": 6.973,
1232
+ "eval_steps_per_second": 1.744,
1233
+ "step": 1800
1234
+ },
1235
+ {
1236
+ "epoch": 0.36,
1237
+ "learning_rate": 0.00019818524449462862,
1238
+ "loss": 0.0463,
1239
+ "step": 1810
1240
+ },
1241
+ {
1242
+ "epoch": 0.37,
1243
+ "learning_rate": 0.00019816519826956156,
1244
+ "loss": 0.0571,
1245
+ "step": 1820
1246
+ },
1247
+ {
1248
+ "epoch": 0.37,
1249
+ "learning_rate": 0.00019814504295914372,
1250
+ "loss": 0.0516,
1251
+ "step": 1830
1252
+ },
1253
+ {
1254
+ "epoch": 0.37,
1255
+ "learning_rate": 0.00019812477858577258,
1256
+ "loss": 0.0511,
1257
+ "step": 1840
1258
+ },
1259
+ {
1260
+ "epoch": 0.37,
1261
+ "learning_rate": 0.00019810440517196672,
1262
+ "loss": 0.0471,
1263
+ "step": 1850
1264
+ },
1265
+ {
1266
+ "epoch": 0.37,
1267
+ "learning_rate": 0.00019808392274036596,
1268
+ "loss": 0.0497,
1269
+ "step": 1860
1270
+ },
1271
+ {
1272
+ "epoch": 0.38,
1273
+ "learning_rate": 0.00019806333131373126,
1274
+ "loss": 0.0608,
1275
+ "step": 1870
1276
+ },
1277
+ {
1278
+ "epoch": 0.38,
1279
+ "learning_rate": 0.00019804263091494466,
1280
+ "loss": 0.0431,
1281
+ "step": 1880
1282
+ },
1283
+ {
1284
+ "epoch": 0.38,
1285
+ "learning_rate": 0.00019802182156700938,
1286
+ "loss": 0.0558,
1287
+ "step": 1890
1288
+ },
1289
+ {
1290
+ "epoch": 0.38,
1291
+ "learning_rate": 0.00019800090329304956,
1292
+ "loss": 0.0524,
1293
+ "step": 1900
1294
+ },
1295
+ {
1296
+ "epoch": 0.38,
1297
+ "eval_loss": 0.06149962544441223,
1298
+ "eval_runtime": 1053.8269,
1299
+ "eval_samples_per_second": 6.982,
1300
+ "eval_steps_per_second": 1.746,
1301
+ "step": 1900
1302
+ },
1303
+ {
1304
+ "epoch": 0.38,
1305
+ "learning_rate": 0.00019797987611631059,
1306
+ "loss": 0.0528,
1307
+ "step": 1910
1308
+ },
1309
+ {
1310
+ "epoch": 0.39,
1311
+ "learning_rate": 0.00019795874006015872,
1312
+ "loss": 0.0481,
1313
+ "step": 1920
1314
+ },
1315
+ {
1316
+ "epoch": 0.39,
1317
+ "learning_rate": 0.0001979374951480812,
1318
+ "loss": 0.0491,
1319
+ "step": 1930
1320
+ },
1321
+ {
1322
+ "epoch": 0.39,
1323
+ "learning_rate": 0.00019791614140368633,
1324
+ "loss": 0.049,
1325
+ "step": 1940
1326
+ },
1327
+ {
1328
+ "epoch": 0.39,
1329
+ "learning_rate": 0.00019789467885070327,
1330
+ "loss": 0.0444,
1331
+ "step": 1950
1332
+ },
1333
+ {
1334
+ "epoch": 0.39,
1335
+ "learning_rate": 0.00019787310751298213,
1336
+ "loss": 0.0466,
1337
+ "step": 1960
1338
+ },
1339
+ {
1340
+ "epoch": 0.4,
1341
+ "learning_rate": 0.00019785142741449393,
1342
+ "loss": 0.0498,
1343
+ "step": 1970
1344
+ },
1345
+ {
1346
+ "epoch": 0.4,
1347
+ "learning_rate": 0.00019782963857933048,
1348
+ "loss": 0.0475,
1349
+ "step": 1980
1350
+ },
1351
+ {
1352
+ "epoch": 0.4,
1353
+ "learning_rate": 0.00019780774103170446,
1354
+ "loss": 0.047,
1355
+ "step": 1990
1356
+ },
1357
+ {
1358
+ "epoch": 0.4,
1359
+ "learning_rate": 0.00019778573479594942,
1360
+ "loss": 0.0432,
1361
+ "step": 2000
1362
+ },
1363
+ {
1364
+ "epoch": 0.4,
1365
+ "eval_loss": 0.06317023187875748,
1366
+ "eval_runtime": 1054.8968,
1367
+ "eval_samples_per_second": 6.975,
1368
+ "eval_steps_per_second": 1.744,
1369
+ "step": 2000
1370
+ },
1371
+ {
1372
+ "epoch": 0.4,
1373
+ "learning_rate": 0.00019776361989651956,
1374
+ "loss": 0.0546,
1375
+ "step": 2010
1376
+ },
1377
+ {
1378
+ "epoch": 0.41,
1379
+ "learning_rate": 0.0001977413963579899,
1380
+ "loss": 0.043,
1381
+ "step": 2020
1382
+ },
1383
+ {
1384
+ "epoch": 0.41,
1385
+ "learning_rate": 0.00019771906420505624,
1386
+ "loss": 0.0579,
1387
+ "step": 2030
1388
+ },
1389
+ {
1390
+ "epoch": 0.41,
1391
+ "learning_rate": 0.00019769662346253493,
1392
+ "loss": 0.057,
1393
+ "step": 2040
1394
+ },
1395
+ {
1396
+ "epoch": 0.41,
1397
+ "learning_rate": 0.00019767407415536317,
1398
+ "loss": 0.0476,
1399
+ "step": 2050
1400
+ },
1401
+ {
1402
+ "epoch": 0.41,
1403
+ "learning_rate": 0.00019765141630859865,
1404
+ "loss": 0.0418,
1405
+ "step": 2060
1406
+ },
1407
+ {
1408
+ "epoch": 0.42,
1409
+ "learning_rate": 0.00019762864994741976,
1410
+ "loss": 0.0497,
1411
+ "step": 2070
1412
+ },
1413
+ {
1414
+ "epoch": 0.42,
1415
+ "learning_rate": 0.00019760577509712546,
1416
+ "loss": 0.0493,
1417
+ "step": 2080
1418
+ },
1419
+ {
1420
+ "epoch": 0.42,
1421
+ "learning_rate": 0.00019758279178313525,
1422
+ "loss": 0.0512,
1423
+ "step": 2090
1424
+ },
1425
+ {
1426
+ "epoch": 0.42,
1427
+ "learning_rate": 0.00019755970003098916,
1428
+ "loss": 0.0631,
1429
+ "step": 2100
1430
+ },
1431
+ {
1432
+ "epoch": 0.42,
1433
+ "eval_loss": 0.06188439950346947,
1434
+ "eval_runtime": 1062.072,
1435
+ "eval_samples_per_second": 6.928,
1436
+ "eval_steps_per_second": 1.732,
1437
+ "step": 2100
1438
+ },
1439
+ {
1440
+ "epoch": 0.42,
1441
+ "learning_rate": 0.00019753649986634772,
1442
+ "loss": 0.0543,
1443
+ "step": 2110
1444
+ },
1445
+ {
1446
+ "epoch": 0.43,
1447
+ "learning_rate": 0.00019751319131499194,
1448
+ "loss": 0.0405,
1449
+ "step": 2120
1450
+ },
1451
+ {
1452
+ "epoch": 0.43,
1453
+ "learning_rate": 0.00019748977440282333,
1454
+ "loss": 0.0564,
1455
+ "step": 2130
1456
+ },
1457
+ {
1458
+ "epoch": 0.43,
1459
+ "learning_rate": 0.0001974662491558637,
1460
+ "loss": 0.0396,
1461
+ "step": 2140
1462
+ },
1463
+ {
1464
+ "epoch": 0.43,
1465
+ "learning_rate": 0.00019744261560025533,
1466
+ "loss": 0.039,
1467
+ "step": 2150
1468
+ },
1469
+ {
1470
+ "epoch": 0.43,
1471
+ "learning_rate": 0.00019741887376226083,
1472
+ "loss": 0.0348,
1473
+ "step": 2160
1474
+ },
1475
+ {
1476
+ "epoch": 0.44,
1477
+ "learning_rate": 0.00019739502366826313,
1478
+ "loss": 0.0533,
1479
+ "step": 2170
1480
+ },
1481
+ {
1482
+ "epoch": 0.44,
1483
+ "learning_rate": 0.0001973710653447655,
1484
+ "loss": 0.0417,
1485
+ "step": 2180
1486
+ },
1487
+ {
1488
+ "epoch": 0.44,
1489
+ "learning_rate": 0.0001973469988183914,
1490
+ "loss": 0.0502,
1491
+ "step": 2190
1492
+ },
1493
+ {
1494
+ "epoch": 0.44,
1495
+ "learning_rate": 0.00019732282411588463,
1496
+ "loss": 0.0519,
1497
+ "step": 2200
1498
+ },
1499
+ {
1500
+ "epoch": 0.44,
1501
+ "eval_loss": 0.05978045240044594,
1502
+ "eval_runtime": 1059.6808,
1503
+ "eval_samples_per_second": 6.944,
1504
+ "eval_steps_per_second": 1.736,
1505
+ "step": 2200
1506
+ },
1507
+ {
1508
+ "epoch": 0.44,
1509
+ "learning_rate": 0.00019729854126410913,
1510
+ "loss": 0.0416,
1511
+ "step": 2210
1512
+ },
1513
+ {
1514
+ "epoch": 0.45,
1515
+ "learning_rate": 0.00019727415029004906,
1516
+ "loss": 0.0586,
1517
+ "step": 2220
1518
+ },
1519
+ {
1520
+ "epoch": 0.45,
1521
+ "learning_rate": 0.00019724965122080868,
1522
+ "loss": 0.0535,
1523
+ "step": 2230
1524
+ },
1525
+ {
1526
+ "epoch": 0.45,
1527
+ "learning_rate": 0.0001972250440836124,
1528
+ "loss": 0.0499,
1529
+ "step": 2240
1530
+ },
1531
+ {
1532
+ "epoch": 0.45,
1533
+ "learning_rate": 0.00019720032890580474,
1534
+ "loss": 0.0484,
1535
+ "step": 2250
1536
+ },
1537
+ {
1538
+ "epoch": 0.46,
1539
+ "learning_rate": 0.00019717550571485024,
1540
+ "loss": 0.044,
1541
+ "step": 2260
1542
+ },
1543
+ {
1544
+ "epoch": 0.46,
1545
+ "learning_rate": 0.0001971505745383335,
1546
+ "loss": 0.0496,
1547
+ "step": 2270
1548
+ },
1549
+ {
1550
+ "epoch": 0.46,
1551
+ "learning_rate": 0.00019712553540395908,
1552
+ "loss": 0.05,
1553
+ "step": 2280
1554
+ },
1555
+ {
1556
+ "epoch": 0.46,
1557
+ "learning_rate": 0.0001971003883395516,
1558
+ "loss": 0.0387,
1559
+ "step": 2290
1560
+ },
1561
+ {
1562
+ "epoch": 0.46,
1563
+ "learning_rate": 0.00019707513337305547,
1564
+ "loss": 0.0397,
1565
+ "step": 2300
1566
+ },
1567
+ {
1568
+ "epoch": 0.46,
1569
+ "eval_loss": 0.060694798827171326,
1570
+ "eval_runtime": 1049.503,
1571
+ "eval_samples_per_second": 7.011,
1572
+ "eval_steps_per_second": 1.753,
1573
+ "step": 2300
1574
+ },
1575
+ {
1576
+ "epoch": 0.47,
1577
+ "learning_rate": 0.0001970497705325351,
1578
+ "loss": 0.0475,
1579
+ "step": 2310
1580
+ },
1581
+ {
1582
+ "epoch": 0.47,
1583
+ "learning_rate": 0.00019702429984617484,
1584
+ "loss": 0.0405,
1585
+ "step": 2320
1586
+ },
1587
+ {
1588
+ "epoch": 0.47,
1589
+ "learning_rate": 0.00019699872134227867,
1590
+ "loss": 0.0447,
1591
+ "step": 2330
1592
+ },
1593
+ {
1594
+ "epoch": 0.47,
1595
+ "learning_rate": 0.00019697303504927061,
1596
+ "loss": 0.0608,
1597
+ "step": 2340
1598
+ },
1599
+ {
1600
+ "epoch": 0.47,
1601
+ "learning_rate": 0.00019694724099569434,
1602
+ "loss": 0.0478,
1603
+ "step": 2350
1604
+ },
1605
+ {
1606
+ "epoch": 0.48,
1607
+ "learning_rate": 0.00019692133921021332,
1608
+ "loss": 0.0432,
1609
+ "step": 2360
1610
+ },
1611
+ {
1612
+ "epoch": 0.48,
1613
+ "learning_rate": 0.00019689532972161068,
1614
+ "loss": 0.0492,
1615
+ "step": 2370
1616
+ },
1617
+ {
1618
+ "epoch": 0.48,
1619
+ "learning_rate": 0.00019686921255878932,
1620
+ "loss": 0.0441,
1621
+ "step": 2380
1622
+ },
1623
+ {
1624
+ "epoch": 0.48,
1625
+ "learning_rate": 0.0001968429877507717,
1626
+ "loss": 0.0603,
1627
+ "step": 2390
1628
+ },
1629
+ {
1630
+ "epoch": 0.48,
1631
+ "learning_rate": 0.00019681665532669996,
1632
+ "loss": 0.0467,
1633
+ "step": 2400
1634
+ },
1635
+ {
1636
+ "epoch": 0.48,
1637
+ "eval_loss": 0.06163698434829712,
1638
+ "eval_runtime": 1049.1311,
1639
+ "eval_samples_per_second": 7.013,
1640
+ "eval_steps_per_second": 1.754,
1641
+ "step": 2400
1642
+ },
1643
+ {
1644
+ "epoch": 0.49,
1645
+ "learning_rate": 0.00019679021531583584,
1646
+ "loss": 0.0406,
1647
+ "step": 2410
1648
+ },
1649
+ {
1650
+ "epoch": 0.49,
1651
+ "learning_rate": 0.00019676366774756056,
1652
+ "loss": 0.0484,
1653
+ "step": 2420
1654
+ },
1655
+ {
1656
+ "epoch": 0.49,
1657
+ "learning_rate": 0.00019673701265137495,
1658
+ "loss": 0.0402,
1659
+ "step": 2430
1660
+ },
1661
+ {
1662
+ "epoch": 0.49,
1663
+ "learning_rate": 0.00019671025005689926,
1664
+ "loss": 0.0541,
1665
+ "step": 2440
1666
+ },
1667
+ {
1668
+ "epoch": 0.49,
1669
+ "learning_rate": 0.00019668337999387324,
1670
+ "loss": 0.0438,
1671
+ "step": 2450
1672
+ },
1673
+ {
1674
+ "epoch": 0.5,
1675
+ "learning_rate": 0.00019665640249215605,
1676
+ "loss": 0.045,
1677
+ "step": 2460
1678
+ },
1679
+ {
1680
+ "epoch": 0.5,
1681
+ "learning_rate": 0.0001966293175817262,
1682
+ "loss": 0.0456,
1683
+ "step": 2470
1684
+ },
1685
+ {
1686
+ "epoch": 0.5,
1687
+ "learning_rate": 0.00019660212529268168,
1688
+ "loss": 0.0454,
1689
+ "step": 2480
1690
+ },
1691
+ {
1692
+ "epoch": 0.5,
1693
+ "learning_rate": 0.00019657482565523963,
1694
+ "loss": 0.0464,
1695
+ "step": 2490
1696
+ },
1697
+ {
1698
+ "epoch": 0.5,
1699
+ "learning_rate": 0.00019654741869973663,
1700
+ "loss": 0.049,
1701
+ "step": 2500
1702
+ },
1703
+ {
1704
+ "epoch": 0.5,
1705
+ "eval_loss": 0.06358644366264343,
1706
+ "eval_runtime": 1050.5846,
1707
+ "eval_samples_per_second": 7.004,
1708
+ "eval_steps_per_second": 1.751,
1709
+ "step": 2500
1710
+ },
1711
+ {
1712
+ "epoch": 0.51,
1713
+ "learning_rate": 0.00019651990445662841,
1714
+ "loss": 0.053,
1715
+ "step": 2510
1716
+ },
1717
+ {
1718
+ "epoch": 0.51,
1719
+ "learning_rate": 0.00019649228295649004,
1720
+ "loss": 0.05,
1721
+ "step": 2520
1722
+ },
1723
+ {
1724
+ "epoch": 0.51,
1725
+ "learning_rate": 0.00019646455423001565,
1726
+ "loss": 0.0423,
1727
+ "step": 2530
1728
+ },
1729
+ {
1730
+ "epoch": 0.51,
1731
+ "learning_rate": 0.0001964367183080186,
1732
+ "loss": 0.0556,
1733
+ "step": 2540
1734
+ },
1735
+ {
1736
+ "epoch": 0.51,
1737
+ "learning_rate": 0.00019640877522143134,
1738
+ "loss": 0.0431,
1739
+ "step": 2550
1740
+ },
1741
+ {
1742
+ "epoch": 0.52,
1743
+ "learning_rate": 0.0001963807250013054,
1744
+ "loss": 0.0377,
1745
+ "step": 2560
1746
+ },
1747
+ {
1748
+ "epoch": 0.52,
1749
+ "learning_rate": 0.00019635256767881144,
1750
+ "loss": 0.0432,
1751
+ "step": 2570
1752
+ },
1753
+ {
1754
+ "epoch": 0.52,
1755
+ "learning_rate": 0.00019632430328523902,
1756
+ "loss": 0.0562,
1757
+ "step": 2580
1758
+ },
1759
+ {
1760
+ "epoch": 0.52,
1761
+ "learning_rate": 0.0001962959318519968,
1762
+ "loss": 0.04,
1763
+ "step": 2590
1764
+ },
1765
+ {
1766
+ "epoch": 0.52,
1767
+ "learning_rate": 0.00019626745341061225,
1768
+ "loss": 0.0488,
1769
+ "step": 2600
1770
+ },
1771
+ {
1772
+ "epoch": 0.52,
1773
+ "eval_loss": 0.06040577217936516,
1774
+ "eval_runtime": 1049.7742,
1775
+ "eval_samples_per_second": 7.009,
1776
+ "eval_steps_per_second": 1.753,
1777
+ "step": 2600
1778
+ },
1779
+ {
1780
+ "epoch": 0.53,
1781
+ "learning_rate": 0.0001962388679927319,
1782
+ "loss": 0.0371,
1783
+ "step": 2610
1784
+ },
1785
+ {
1786
+ "epoch": 0.53,
1787
+ "learning_rate": 0.000196210175630121,
1788
+ "loss": 0.0412,
1789
+ "step": 2620
1790
+ },
1791
+ {
1792
+ "epoch": 0.53,
1793
+ "learning_rate": 0.00019618137635466382,
1794
+ "loss": 0.0432,
1795
+ "step": 2630
1796
+ },
1797
+ {
1798
+ "epoch": 0.53,
1799
+ "learning_rate": 0.00019615247019836327,
1800
+ "loss": 0.0501,
1801
+ "step": 2640
1802
+ },
1803
+ {
1804
+ "epoch": 0.53,
1805
+ "learning_rate": 0.00019612345719334116,
1806
+ "loss": 0.0422,
1807
+ "step": 2650
1808
+ },
1809
+ {
1810
+ "epoch": 0.54,
1811
+ "learning_rate": 0.00019609433737183791,
1812
+ "loss": 0.0511,
1813
+ "step": 2660
1814
+ },
1815
+ {
1816
+ "epoch": 0.54,
1817
+ "learning_rate": 0.00019606511076621276,
1818
+ "loss": 0.0372,
1819
+ "step": 2670
1820
+ },
1821
+ {
1822
+ "epoch": 0.54,
1823
+ "learning_rate": 0.00019603577740894354,
1824
+ "loss": 0.0541,
1825
+ "step": 2680
1826
+ },
1827
+ {
1828
+ "epoch": 0.54,
1829
+ "learning_rate": 0.0001960063373326267,
1830
+ "loss": 0.0472,
1831
+ "step": 2690
1832
+ },
1833
+ {
1834
+ "epoch": 0.54,
1835
+ "learning_rate": 0.00019597679056997737,
1836
+ "loss": 0.0449,
1837
+ "step": 2700
1838
+ },
1839
+ {
1840
+ "epoch": 0.54,
1841
+ "eval_loss": 0.05977201834321022,
1842
+ "eval_runtime": 1052.2024,
1843
+ "eval_samples_per_second": 6.993,
1844
+ "eval_steps_per_second": 1.749,
1845
+ "step": 2700
1846
+ },
1847
+ {
1848
+ "epoch": 0.55,
1849
+ "learning_rate": 0.00019594713715382915,
1850
+ "loss": 0.0517,
1851
+ "step": 2710
1852
+ },
1853
+ {
1854
+ "epoch": 0.55,
1855
+ "learning_rate": 0.00019591737711713414,
1856
+ "loss": 0.0476,
1857
+ "step": 2720
1858
+ },
1859
+ {
1860
+ "epoch": 0.55,
1861
+ "learning_rate": 0.00019588751049296298,
1862
+ "loss": 0.048,
1863
+ "step": 2730
1864
+ },
1865
+ {
1866
+ "epoch": 0.55,
1867
+ "learning_rate": 0.00019585753731450478,
1868
+ "loss": 0.0342,
1869
+ "step": 2740
1870
+ },
1871
+ {
1872
+ "epoch": 0.55,
1873
+ "learning_rate": 0.00019582745761506697,
1874
+ "loss": 0.0508,
1875
+ "step": 2750
1876
+ },
1877
+ {
1878
+ "epoch": 0.56,
1879
+ "learning_rate": 0.00019579727142807535,
1880
+ "loss": 0.0407,
1881
+ "step": 2760
1882
+ },
1883
+ {
1884
+ "epoch": 0.56,
1885
+ "learning_rate": 0.0001957669787870742,
1886
+ "loss": 0.0358,
1887
+ "step": 2770
1888
+ },
1889
+ {
1890
+ "epoch": 0.56,
1891
+ "learning_rate": 0.00019573657972572593,
1892
+ "loss": 0.0506,
1893
+ "step": 2780
1894
+ },
1895
+ {
1896
+ "epoch": 0.56,
1897
+ "learning_rate": 0.00019570607427781128,
1898
+ "loss": 0.0476,
1899
+ "step": 2790
1900
+ },
1901
+ {
1902
+ "epoch": 0.56,
1903
+ "learning_rate": 0.0001956754624772292,
1904
+ "loss": 0.0438,
1905
+ "step": 2800
1906
+ },
1907
+ {
1908
+ "epoch": 0.56,
1909
+ "eval_loss": 0.061029911041259766,
1910
+ "eval_runtime": 1062.1739,
1911
+ "eval_samples_per_second": 6.927,
1912
+ "eval_steps_per_second": 1.732,
1913
+ "step": 2800
1914
+ },
1915
+ {
1916
+ "epoch": 0.57,
1917
+ "learning_rate": 0.0001956447443579968,
1918
+ "loss": 0.0481,
1919
+ "step": 2810
1920
+ },
1921
+ {
1922
+ "epoch": 0.57,
1923
+ "learning_rate": 0.00019561391995424941,
1924
+ "loss": 0.0515,
1925
+ "step": 2820
1926
+ },
1927
+ {
1928
+ "epoch": 0.57,
1929
+ "learning_rate": 0.00019558298930024044,
1930
+ "loss": 0.0491,
1931
+ "step": 2830
1932
+ },
1933
+ {
1934
+ "epoch": 0.57,
1935
+ "learning_rate": 0.0001955519524303413,
1936
+ "loss": 0.0367,
1937
+ "step": 2840
1938
+ },
1939
+ {
1940
+ "epoch": 0.57,
1941
+ "learning_rate": 0.0001955208093790415,
1942
+ "loss": 0.0431,
1943
+ "step": 2850
1944
+ },
1945
+ {
1946
+ "epoch": 0.58,
1947
+ "learning_rate": 0.0001954895601809485,
1948
+ "loss": 0.0436,
1949
+ "step": 2860
1950
+ },
1951
+ {
1952
+ "epoch": 0.58,
1953
+ "learning_rate": 0.0001954582048707878,
1954
+ "loss": 0.0442,
1955
+ "step": 2870
1956
+ },
1957
+ {
1958
+ "epoch": 0.58,
1959
+ "learning_rate": 0.00019542674348340267,
1960
+ "loss": 0.0532,
1961
+ "step": 2880
1962
+ },
1963
+ {
1964
+ "epoch": 0.58,
1965
+ "learning_rate": 0.00019539517605375446,
1966
+ "loss": 0.0441,
1967
+ "step": 2890
1968
+ },
1969
+ {
1970
+ "epoch": 0.58,
1971
+ "learning_rate": 0.00019536350261692214,
1972
+ "loss": 0.036,
1973
+ "step": 2900
1974
+ },
1975
+ {
1976
+ "epoch": 0.58,
1977
+ "eval_loss": 0.06326338648796082,
1978
+ "eval_runtime": 1061.8457,
1979
+ "eval_samples_per_second": 6.929,
1980
+ "eval_steps_per_second": 1.733,
1981
+ "step": 2900
1982
+ },
1983
+ {
1984
+ "epoch": 0.59,
1985
+ "learning_rate": 0.00019533172320810265,
1986
+ "loss": 0.0486,
1987
+ "step": 2910
1988
+ },
1989
+ {
1990
+ "epoch": 0.59,
1991
+ "learning_rate": 0.00019529983786261058,
1992
+ "loss": 0.0568,
1993
+ "step": 2920
1994
+ },
1995
+ {
1996
+ "epoch": 0.59,
1997
+ "learning_rate": 0.00019526784661587829,
1998
+ "loss": 0.0434,
1999
+ "step": 2930
2000
+ },
2001
+ {
2002
+ "epoch": 0.59,
2003
+ "learning_rate": 0.0001952357495034558,
2004
+ "loss": 0.0541,
2005
+ "step": 2940
2006
+ },
2007
+ {
2008
+ "epoch": 0.59,
2009
+ "learning_rate": 0.00019520354656101085,
2010
+ "loss": 0.0395,
2011
+ "step": 2950
2012
+ },
2013
+ {
2014
+ "epoch": 0.6,
2015
+ "learning_rate": 0.00019517123782432868,
2016
+ "loss": 0.0458,
2017
+ "step": 2960
2018
+ },
2019
+ {
2020
+ "epoch": 0.6,
2021
+ "learning_rate": 0.00019513882332931212,
2022
+ "loss": 0.0395,
2023
+ "step": 2970
2024
+ },
2025
+ {
2026
+ "epoch": 0.6,
2027
+ "learning_rate": 0.00019510630311198157,
2028
+ "loss": 0.0442,
2029
+ "step": 2980
2030
+ },
2031
+ {
2032
+ "epoch": 0.6,
2033
+ "learning_rate": 0.00019507367720847488,
2034
+ "loss": 0.041,
2035
+ "step": 2990
2036
+ },
2037
+ {
2038
+ "epoch": 0.6,
2039
+ "learning_rate": 0.00019504094565504733,
2040
+ "loss": 0.0464,
2041
+ "step": 3000
2042
+ },
2043
+ {
2044
+ "epoch": 0.6,
2045
+ "eval_loss": 0.06028781458735466,
2046
+ "eval_runtime": 1035.9718,
2047
+ "eval_samples_per_second": 7.103,
2048
+ "eval_steps_per_second": 1.776,
2049
+ "step": 3000
2050
+ },
2051
+ {
2052
+ "epoch": 0.61,
2053
+ "learning_rate": 0.00019500810848807162,
2054
+ "loss": 0.0396,
2055
+ "step": 3010
2056
+ },
2057
+ {
2058
+ "epoch": 0.61,
2059
+ "learning_rate": 0.00019497516574403778,
2060
+ "loss": 0.0491,
2061
+ "step": 3020
2062
+ },
2063
+ {
2064
+ "epoch": 0.61,
2065
+ "learning_rate": 0.00019494211745955324,
2066
+ "loss": 0.0585,
2067
+ "step": 3030
2068
+ },
2069
+ {
2070
+ "epoch": 0.61,
2071
+ "learning_rate": 0.00019490896367134266,
2072
+ "loss": 0.0484,
2073
+ "step": 3040
2074
+ },
2075
+ {
2076
+ "epoch": 0.61,
2077
+ "learning_rate": 0.00019487570441624791,
2078
+ "loss": 0.0507,
2079
+ "step": 3050
2080
+ },
2081
+ {
2082
+ "epoch": 0.62,
2083
+ "learning_rate": 0.0001948423397312281,
2084
+ "loss": 0.0467,
2085
+ "step": 3060
2086
+ },
2087
+ {
2088
+ "epoch": 0.62,
2089
+ "learning_rate": 0.0001948088696533595,
2090
+ "loss": 0.0444,
2091
+ "step": 3070
2092
+ },
2093
+ {
2094
+ "epoch": 0.62,
2095
+ "learning_rate": 0.00019477529421983546,
2096
+ "loss": 0.0498,
2097
+ "step": 3080
2098
+ },
2099
+ {
2100
+ "epoch": 0.62,
2101
+ "learning_rate": 0.0001947416134679665,
2102
+ "loss": 0.0373,
2103
+ "step": 3090
2104
+ },
2105
+ {
2106
+ "epoch": 0.62,
2107
+ "learning_rate": 0.00019470782743518002,
2108
+ "loss": 0.0437,
2109
+ "step": 3100
2110
+ },
2111
+ {
2112
+ "epoch": 0.62,
2113
+ "eval_loss": 0.06118550896644592,
2114
+ "eval_runtime": 1055.3909,
2115
+ "eval_samples_per_second": 6.972,
2116
+ "eval_steps_per_second": 1.743,
2117
+ "step": 3100
2118
+ },
2119
+ {
2120
+ "epoch": 0.63,
2121
+ "learning_rate": 0.00019467393615902055,
2122
+ "loss": 0.0391,
2123
+ "step": 3110
2124
+ },
2125
+ {
2126
+ "epoch": 0.63,
2127
+ "learning_rate": 0.0001946399396771495,
2128
+ "loss": 0.0456,
2129
+ "step": 3120
2130
+ },
2131
+ {
2132
+ "epoch": 0.63,
2133
+ "learning_rate": 0.00019460583802734523,
2134
+ "loss": 0.0368,
2135
+ "step": 3130
2136
+ },
2137
+ {
2138
+ "epoch": 0.63,
2139
+ "learning_rate": 0.0001945716312475029,
2140
+ "loss": 0.0451,
2141
+ "step": 3140
2142
+ },
2143
+ {
2144
+ "epoch": 0.63,
2145
+ "learning_rate": 0.0001945373193756346,
2146
+ "loss": 0.046,
2147
+ "step": 3150
2148
+ },
2149
+ {
2150
+ "epoch": 0.64,
2151
+ "learning_rate": 0.00019450290244986914,
2152
+ "loss": 0.0364,
2153
+ "step": 3160
2154
+ },
2155
+ {
2156
+ "epoch": 0.64,
2157
+ "learning_rate": 0.00019446838050845205,
2158
+ "loss": 0.0576,
2159
+ "step": 3170
2160
+ },
2161
+ {
2162
+ "epoch": 0.64,
2163
+ "learning_rate": 0.00019443375358974555,
2164
+ "loss": 0.0421,
2165
+ "step": 3180
2166
+ },
2167
+ {
2168
+ "epoch": 0.64,
2169
+ "learning_rate": 0.00019439902173222859,
2170
+ "loss": 0.0528,
2171
+ "step": 3190
2172
+ },
2173
+ {
2174
+ "epoch": 0.64,
2175
+ "learning_rate": 0.0001943641849744967,
2176
+ "loss": 0.0389,
2177
+ "step": 3200
2178
+ },
2179
+ {
2180
+ "epoch": 0.64,
2181
+ "eval_loss": 0.06049993634223938,
2182
+ "eval_runtime": 1052.016,
2183
+ "eval_samples_per_second": 6.994,
2184
+ "eval_steps_per_second": 1.749,
2185
+ "step": 3200
2186
+ },
2187
+ {
2188
+ "epoch": 0.65,
2189
+ "learning_rate": 0.00019432924335526194,
2190
+ "loss": 0.0432,
2191
+ "step": 3210
2192
+ },
2193
+ {
2194
+ "epoch": 0.65,
2195
+ "learning_rate": 0.00019429419691335297,
2196
+ "loss": 0.0339,
2197
+ "step": 3220
2198
+ },
2199
+ {
2200
+ "epoch": 0.65,
2201
+ "learning_rate": 0.00019425904568771483,
2202
+ "loss": 0.0434,
2203
+ "step": 3230
2204
+ },
2205
+ {
2206
+ "epoch": 0.65,
2207
+ "learning_rate": 0.00019422378971740907,
2208
+ "loss": 0.0472,
2209
+ "step": 3240
2210
+ },
2211
+ {
2212
+ "epoch": 0.65,
2213
+ "learning_rate": 0.00019418842904161368,
2214
+ "loss": 0.0394,
2215
+ "step": 3250
2216
+ },
2217
+ {
2218
+ "epoch": 0.66,
2219
+ "learning_rate": 0.00019415296369962288,
2220
+ "loss": 0.0404,
2221
+ "step": 3260
2222
+ },
2223
+ {
2224
+ "epoch": 0.66,
2225
+ "learning_rate": 0.00019411739373084732,
2226
+ "loss": 0.0352,
2227
+ "step": 3270
2228
+ },
2229
+ {
2230
+ "epoch": 0.66,
2231
+ "learning_rate": 0.00019408171917481386,
2232
+ "loss": 0.0364,
2233
+ "step": 3280
2234
+ },
2235
+ {
2236
+ "epoch": 0.66,
2237
+ "learning_rate": 0.00019404594007116555,
2238
+ "loss": 0.0433,
2239
+ "step": 3290
2240
+ },
2241
+ {
2242
+ "epoch": 0.66,
2243
+ "learning_rate": 0.00019401005645966167,
2244
+ "loss": 0.0377,
2245
+ "step": 3300
2246
+ },
2247
+ {
2248
+ "epoch": 0.66,
2249
+ "eval_loss": 0.06322025507688522,
2250
+ "eval_runtime": 1050.1226,
2251
+ "eval_samples_per_second": 7.007,
2252
+ "eval_steps_per_second": 1.752,
2253
+ "step": 3300
2254
+ },
2255
+ {
2256
+ "epoch": 0.67,
2257
+ "learning_rate": 0.00019397406838017766,
2258
+ "loss": 0.042,
2259
+ "step": 3310
2260
+ },
2261
+ {
2262
+ "epoch": 0.67,
2263
+ "learning_rate": 0.00019393797587270497,
2264
+ "loss": 0.0326,
2265
+ "step": 3320
2266
+ },
2267
+ {
2268
+ "epoch": 0.67,
2269
+ "learning_rate": 0.00019390177897735114,
2270
+ "loss": 0.0443,
2271
+ "step": 3330
2272
+ },
2273
+ {
2274
+ "epoch": 0.67,
2275
+ "learning_rate": 0.00019386547773433965,
2276
+ "loss": 0.0545,
2277
+ "step": 3340
2278
+ },
2279
+ {
2280
+ "epoch": 0.67,
2281
+ "learning_rate": 0.00019382907218401006,
2282
+ "loss": 0.0422,
2283
+ "step": 3350
2284
+ },
2285
+ {
2286
+ "epoch": 0.68,
2287
+ "learning_rate": 0.00019379256236681775,
2288
+ "loss": 0.0393,
2289
+ "step": 3360
2290
+ },
2291
+ {
2292
+ "epoch": 0.68,
2293
+ "learning_rate": 0.000193755948323334,
2294
+ "loss": 0.0516,
2295
+ "step": 3370
2296
+ },
2297
+ {
2298
+ "epoch": 0.68,
2299
+ "learning_rate": 0.00019371923009424587,
2300
+ "loss": 0.0499,
2301
+ "step": 3380
2302
+ },
2303
+ {
2304
+ "epoch": 0.68,
2305
+ "learning_rate": 0.0001936824077203562,
2306
+ "loss": 0.0491,
2307
+ "step": 3390
2308
+ },
2309
+ {
2310
+ "epoch": 0.68,
2311
+ "learning_rate": 0.0001936454812425836,
2312
+ "loss": 0.0382,
2313
+ "step": 3400
2314
+ },
2315
+ {
2316
+ "epoch": 0.68,
2317
+ "eval_loss": 0.06261178106069565,
2318
+ "eval_runtime": 1054.0723,
2319
+ "eval_samples_per_second": 6.981,
2320
+ "eval_steps_per_second": 1.746,
2321
+ "step": 3400
2322
+ },
2323
+ {
2324
+ "epoch": 0.69,
2325
+ "learning_rate": 0.00019360845070196236,
2326
+ "loss": 0.0431,
2327
+ "step": 3410
2328
+ },
2329
+ {
2330
+ "epoch": 0.69,
2331
+ "learning_rate": 0.0001935713161396424,
2332
+ "loss": 0.0498,
2333
+ "step": 3420
2334
+ },
2335
+ {
2336
+ "epoch": 0.69,
2337
+ "learning_rate": 0.0001935340775968892,
2338
+ "loss": 0.0457,
2339
+ "step": 3430
2340
+ },
2341
+ {
2342
+ "epoch": 0.69,
2343
+ "learning_rate": 0.00019349673511508383,
2344
+ "loss": 0.0362,
2345
+ "step": 3440
2346
+ },
2347
+ {
2348
+ "epoch": 0.69,
2349
+ "learning_rate": 0.00019345928873572282,
2350
+ "loss": 0.0432,
2351
+ "step": 3450
2352
+ },
2353
+ {
2354
+ "epoch": 0.7,
2355
+ "learning_rate": 0.00019342173850041822,
2356
+ "loss": 0.0365,
2357
+ "step": 3460
2358
+ },
2359
+ {
2360
+ "epoch": 0.7,
2361
+ "learning_rate": 0.00019338408445089745,
2362
+ "loss": 0.0358,
2363
+ "step": 3470
2364
+ },
2365
+ {
2366
+ "epoch": 0.7,
2367
+ "learning_rate": 0.0001933463266290033,
2368
+ "loss": 0.039,
2369
+ "step": 3480
2370
+ },
2371
+ {
2372
+ "epoch": 0.7,
2373
+ "learning_rate": 0.00019330846507669382,
2374
+ "loss": 0.0412,
2375
+ "step": 3490
2376
+ },
2377
+ {
2378
+ "epoch": 0.7,
2379
+ "learning_rate": 0.00019327049983604245,
2380
+ "loss": 0.051,
2381
+ "step": 3500
2382
+ },
2383
+ {
2384
+ "epoch": 0.7,
2385
+ "eval_loss": 0.061244383454322815,
2386
+ "eval_runtime": 1045.5529,
2387
+ "eval_samples_per_second": 7.037,
2388
+ "eval_steps_per_second": 1.76,
2389
+ "step": 3500
2390
+ },
2391
+ {
2392
+ "epoch": 0.71,
2393
+ "learning_rate": 0.00019323243094923772,
2394
+ "loss": 0.0387,
2395
+ "step": 3510
2396
+ },
2397
+ {
2398
+ "epoch": 0.71,
2399
+ "learning_rate": 0.00019319425845858341,
2400
+ "loss": 0.0473,
2401
+ "step": 3520
2402
+ },
2403
+ {
2404
+ "epoch": 0.71,
2405
+ "learning_rate": 0.00019315598240649847,
2406
+ "loss": 0.037,
2407
+ "step": 3530
2408
+ },
2409
+ {
2410
+ "epoch": 0.71,
2411
+ "learning_rate": 0.0001931176028355168,
2412
+ "loss": 0.0435,
2413
+ "step": 3540
2414
+ },
2415
+ {
2416
+ "epoch": 0.71,
2417
+ "learning_rate": 0.00019307911978828747,
2418
+ "loss": 0.0364,
2419
+ "step": 3550
2420
+ },
2421
+ {
2422
+ "epoch": 0.72,
2423
+ "learning_rate": 0.0001930405333075745,
2424
+ "loss": 0.0424,
2425
+ "step": 3560
2426
+ },
2427
+ {
2428
+ "epoch": 0.72,
2429
+ "learning_rate": 0.00019300184343625678,
2430
+ "loss": 0.0348,
2431
+ "step": 3570
2432
+ },
2433
+ {
2434
+ "epoch": 0.72,
2435
+ "learning_rate": 0.00019296305021732817,
2436
+ "loss": 0.0395,
2437
+ "step": 3580
2438
+ },
2439
+ {
2440
+ "epoch": 0.72,
2441
+ "learning_rate": 0.00019292415369389734,
2442
+ "loss": 0.0486,
2443
+ "step": 3590
2444
+ },
2445
+ {
2446
+ "epoch": 0.72,
2447
+ "learning_rate": 0.00019288515390918776,
2448
+ "loss": 0.047,
2449
+ "step": 3600
2450
+ },
2451
+ {
2452
+ "epoch": 0.72,
2453
+ "eval_loss": 0.06326541304588318,
2454
+ "eval_runtime": 1046.8433,
2455
+ "eval_samples_per_second": 7.029,
2456
+ "eval_steps_per_second": 1.758,
2457
+ "step": 3600
2458
+ },
2459
+ {
2460
+ "epoch": 0.73,
2461
+ "learning_rate": 0.00019284605090653766,
2462
+ "loss": 0.0403,
2463
+ "step": 3610
2464
+ },
2465
+ {
2466
+ "epoch": 0.73,
2467
+ "learning_rate": 0.00019280684472939994,
2468
+ "loss": 0.0381,
2469
+ "step": 3620
2470
+ },
2471
+ {
2472
+ "epoch": 0.73,
2473
+ "learning_rate": 0.00019276753542134224,
2474
+ "loss": 0.0332,
2475
+ "step": 3630
2476
+ },
2477
+ {
2478
+ "epoch": 0.73,
2479
+ "learning_rate": 0.00019272812302604665,
2480
+ "loss": 0.0435,
2481
+ "step": 3640
2482
+ },
2483
+ {
2484
+ "epoch": 0.73,
2485
+ "learning_rate": 0.00019268860758730997,
2486
+ "loss": 0.0512,
2487
+ "step": 3650
2488
+ },
2489
+ {
2490
+ "epoch": 0.74,
2491
+ "learning_rate": 0.00019264898914904342,
2492
+ "loss": 0.0403,
2493
+ "step": 3660
2494
+ },
2495
+ {
2496
+ "epoch": 0.74,
2497
+ "learning_rate": 0.00019260926775527265,
2498
+ "loss": 0.0336,
2499
+ "step": 3670
2500
+ },
2501
+ {
2502
+ "epoch": 0.74,
2503
+ "learning_rate": 0.00019256944345013785,
2504
+ "loss": 0.0425,
2505
+ "step": 3680
2506
+ },
2507
+ {
2508
+ "epoch": 0.74,
2509
+ "learning_rate": 0.00019252951627789344,
2510
+ "loss": 0.0431,
2511
+ "step": 3690
2512
+ },
2513
+ {
2514
+ "epoch": 0.74,
2515
+ "learning_rate": 0.00019248948628290818,
2516
+ "loss": 0.0451,
2517
+ "step": 3700
2518
+ },
2519
+ {
2520
+ "epoch": 0.74,
2521
+ "eval_loss": 0.06083804368972778,
2522
+ "eval_runtime": 1049.5583,
2523
+ "eval_samples_per_second": 7.011,
2524
+ "eval_steps_per_second": 1.753,
2525
+ "step": 3700
2526
+ },
2527
+ {
2528
+ "epoch": 0.75,
2529
+ "learning_rate": 0.00019244935350966514,
2530
+ "loss": 0.032,
2531
+ "step": 3710
2532
+ },
2533
+ {
2534
+ "epoch": 0.75,
2535
+ "learning_rate": 0.00019240911800276153,
2536
+ "loss": 0.0378,
2537
+ "step": 3720
2538
+ },
2539
+ {
2540
+ "epoch": 0.75,
2541
+ "learning_rate": 0.0001923687798069088,
2542
+ "loss": 0.044,
2543
+ "step": 3730
2544
+ },
2545
+ {
2546
+ "epoch": 0.75,
2547
+ "learning_rate": 0.00019232833896693242,
2548
+ "loss": 0.0363,
2549
+ "step": 3740
2550
+ },
2551
+ {
2552
+ "epoch": 0.75,
2553
+ "learning_rate": 0.00019228779552777202,
2554
+ "loss": 0.0412,
2555
+ "step": 3750
2556
+ },
2557
+ {
2558
+ "epoch": 0.76,
2559
+ "learning_rate": 0.0001922471495344812,
2560
+ "loss": 0.0376,
2561
+ "step": 3760
2562
+ },
2563
+ {
2564
+ "epoch": 0.76,
2565
+ "learning_rate": 0.0001922064010322275,
2566
+ "loss": 0.0451,
2567
+ "step": 3770
2568
+ },
2569
+ {
2570
+ "epoch": 0.76,
2571
+ "learning_rate": 0.00019216555006629237,
2572
+ "loss": 0.0443,
2573
+ "step": 3780
2574
+ },
2575
+ {
2576
+ "epoch": 0.76,
2577
+ "learning_rate": 0.0001921245966820712,
2578
+ "loss": 0.0394,
2579
+ "step": 3790
2580
+ },
2581
+ {
2582
+ "epoch": 0.77,
2583
+ "learning_rate": 0.00019208354092507305,
2584
+ "loss": 0.0472,
2585
+ "step": 3800
2586
+ },
2587
+ {
2588
+ "epoch": 0.77,
2589
+ "eval_loss": 0.06302132457494736,
2590
+ "eval_runtime": 1043.9496,
2591
+ "eval_samples_per_second": 7.048,
2592
+ "eval_steps_per_second": 1.763,
2593
+ "step": 3800
2594
+ },
2595
+ {
2596
+ "epoch": 0.77,
2597
+ "learning_rate": 0.00019204238284092093,
2598
+ "loss": 0.0418,
2599
+ "step": 3810
2600
+ },
2601
+ {
2602
+ "epoch": 0.77,
2603
+ "learning_rate": 0.00019200112247535141,
2604
+ "loss": 0.0416,
2605
+ "step": 3820
2606
+ },
2607
+ {
2608
+ "epoch": 0.77,
2609
+ "learning_rate": 0.00019195975987421472,
2610
+ "loss": 0.0418,
2611
+ "step": 3830
2612
+ },
2613
+ {
2614
+ "epoch": 0.77,
2615
+ "learning_rate": 0.00019191829508347481,
2616
+ "loss": 0.0394,
2617
+ "step": 3840
2618
+ },
2619
+ {
2620
+ "epoch": 0.78,
2621
+ "learning_rate": 0.00019187672814920912,
2622
+ "loss": 0.0405,
2623
+ "step": 3850
2624
+ },
2625
+ {
2626
+ "epoch": 0.78,
2627
+ "learning_rate": 0.00019183505911760855,
2628
+ "loss": 0.0306,
2629
+ "step": 3860
2630
+ },
2631
+ {
2632
+ "epoch": 0.78,
2633
+ "learning_rate": 0.00019179328803497754,
2634
+ "loss": 0.0443,
2635
+ "step": 3870
2636
+ },
2637
+ {
2638
+ "epoch": 0.78,
2639
+ "learning_rate": 0.0001917514149477339,
2640
+ "loss": 0.035,
2641
+ "step": 3880
2642
+ },
2643
+ {
2644
+ "epoch": 0.78,
2645
+ "learning_rate": 0.00019170943990240877,
2646
+ "loss": 0.0371,
2647
+ "step": 3890
2648
+ },
2649
+ {
2650
+ "epoch": 0.79,
2651
+ "learning_rate": 0.0001916673629456466,
2652
+ "loss": 0.0394,
2653
+ "step": 3900
2654
+ },
2655
+ {
2656
+ "epoch": 0.79,
2657
+ "eval_loss": 0.06118810176849365,
2658
+ "eval_runtime": 1046.2783,
2659
+ "eval_samples_per_second": 7.033,
2660
+ "eval_steps_per_second": 1.759,
2661
+ "step": 3900
2662
+ },
2663
+ {
2664
+ "epoch": 0.79,
2665
+ "learning_rate": 0.00019162518412420512,
2666
+ "loss": 0.0337,
2667
+ "step": 3910
2668
+ },
2669
+ {
2670
+ "epoch": 0.79,
2671
+ "learning_rate": 0.00019158290348495524,
2672
+ "loss": 0.0433,
2673
+ "step": 3920
2674
+ },
2675
+ {
2676
+ "epoch": 0.79,
2677
+ "learning_rate": 0.000191540521074881,
2678
+ "loss": 0.0402,
2679
+ "step": 3930
2680
+ },
2681
+ {
2682
+ "epoch": 0.79,
2683
+ "learning_rate": 0.0001914980369410795,
2684
+ "loss": 0.0326,
2685
+ "step": 3940
2686
+ },
2687
+ {
2688
+ "epoch": 0.8,
2689
+ "learning_rate": 0.00019145545113076096,
2690
+ "loss": 0.0501,
2691
+ "step": 3950
2692
+ },
2693
+ {
2694
+ "epoch": 0.8,
2695
+ "learning_rate": 0.00019141276369124855,
2696
+ "loss": 0.0448,
2697
+ "step": 3960
2698
+ },
2699
+ {
2700
+ "epoch": 0.8,
2701
+ "learning_rate": 0.00019136997466997837,
2702
+ "loss": 0.0403,
2703
+ "step": 3970
2704
+ },
2705
+ {
2706
+ "epoch": 0.8,
2707
+ "learning_rate": 0.00019132708411449936,
2708
+ "loss": 0.0427,
2709
+ "step": 3980
2710
+ },
2711
+ {
2712
+ "epoch": 0.8,
2713
+ "learning_rate": 0.0001912840920724734,
2714
+ "loss": 0.0322,
2715
+ "step": 3990
2716
+ },
2717
+ {
2718
+ "epoch": 0.81,
2719
+ "learning_rate": 0.00019124099859167503,
2720
+ "loss": 0.0471,
2721
+ "step": 4000
2722
+ },
2723
+ {
2724
+ "epoch": 0.81,
2725
+ "eval_loss": 0.06139204651117325,
2726
+ "eval_runtime": 1057.3652,
2727
+ "eval_samples_per_second": 6.959,
2728
+ "eval_steps_per_second": 1.74,
2729
+ "step": 4000
2730
+ },
2731
+ {
2732
+ "epoch": 0.81,
2733
+ "learning_rate": 0.00019119780371999162,
2734
+ "loss": 0.0306,
2735
+ "step": 4010
2736
+ },
2737
+ {
2738
+ "epoch": 0.81,
2739
+ "learning_rate": 0.00019115450750542304,
2740
+ "loss": 0.0352,
2741
+ "step": 4020
2742
+ },
2743
+ {
2744
+ "epoch": 0.81,
2745
+ "learning_rate": 0.000191111109996082,
2746
+ "loss": 0.0351,
2747
+ "step": 4030
2748
+ },
2749
+ {
2750
+ "epoch": 0.81,
2751
+ "learning_rate": 0.00019106761124019364,
2752
+ "loss": 0.0295,
2753
+ "step": 4040
2754
+ },
2755
+ {
2756
+ "epoch": 0.82,
2757
+ "learning_rate": 0.00019102401128609557,
2758
+ "loss": 0.0379,
2759
+ "step": 4050
2760
+ },
2761
+ {
2762
+ "epoch": 0.82,
2763
+ "learning_rate": 0.00019098031018223796,
2764
+ "loss": 0.048,
2765
+ "step": 4060
2766
+ },
2767
+ {
2768
+ "epoch": 0.82,
2769
+ "learning_rate": 0.00019093650797718338,
2770
+ "loss": 0.0454,
2771
+ "step": 4070
2772
+ },
2773
+ {
2774
+ "epoch": 0.82,
2775
+ "learning_rate": 0.00019089260471960663,
2776
+ "loss": 0.0463,
2777
+ "step": 4080
2778
+ },
2779
+ {
2780
+ "epoch": 0.82,
2781
+ "learning_rate": 0.0001908486004582949,
2782
+ "loss": 0.0447,
2783
+ "step": 4090
2784
+ },
2785
+ {
2786
+ "epoch": 0.83,
2787
+ "learning_rate": 0.00019080449524214762,
2788
+ "loss": 0.031,
2789
+ "step": 4100
2790
+ },
2791
+ {
2792
+ "epoch": 0.83,
2793
+ "eval_loss": 0.06018984317779541,
2794
+ "eval_runtime": 1049.9509,
2795
+ "eval_samples_per_second": 7.008,
2796
+ "eval_steps_per_second": 1.752,
2797
+ "step": 4100
2798
+ },
2799
+ {
2800
+ "epoch": 0.83,
2801
+ "learning_rate": 0.00019076028912017642,
2802
+ "loss": 0.0292,
2803
+ "step": 4110
2804
+ },
2805
+ {
2806
+ "epoch": 0.83,
2807
+ "learning_rate": 0.00019071598214150494,
2808
+ "loss": 0.0252,
2809
+ "step": 4120
2810
+ },
2811
+ {
2812
+ "epoch": 0.83,
2813
+ "learning_rate": 0.00019067157435536904,
2814
+ "loss": 0.0382,
2815
+ "step": 4130
2816
+ },
2817
+ {
2818
+ "epoch": 0.83,
2819
+ "learning_rate": 0.00019062706581111653,
2820
+ "loss": 0.0303,
2821
+ "step": 4140
2822
+ },
2823
+ {
2824
+ "epoch": 0.84,
2825
+ "learning_rate": 0.0001905824565582072,
2826
+ "loss": 0.0419,
2827
+ "step": 4150
2828
+ },
2829
+ {
2830
+ "epoch": 0.84,
2831
+ "learning_rate": 0.00019053774664621272,
2832
+ "loss": 0.0358,
2833
+ "step": 4160
2834
+ },
2835
+ {
2836
+ "epoch": 0.84,
2837
+ "learning_rate": 0.00019049293612481675,
2838
+ "loss": 0.037,
2839
+ "step": 4170
2840
+ },
2841
+ {
2842
+ "epoch": 0.84,
2843
+ "learning_rate": 0.00019044802504381453,
2844
+ "loss": 0.0306,
2845
+ "step": 4180
2846
+ },
2847
+ {
2848
+ "epoch": 0.84,
2849
+ "learning_rate": 0.00019040301345311326,
2850
+ "loss": 0.0431,
2851
+ "step": 4190
2852
+ },
2853
+ {
2854
+ "epoch": 0.85,
2855
+ "learning_rate": 0.0001903579014027317,
2856
+ "loss": 0.0405,
2857
+ "step": 4200
2858
+ },
2859
+ {
2860
+ "epoch": 0.85,
2861
+ "eval_loss": 0.05838855355978012,
2862
+ "eval_runtime": 1051.4215,
2863
+ "eval_samples_per_second": 6.998,
2864
+ "eval_steps_per_second": 1.75,
2865
+ "step": 4200
2866
+ },
2867
+ {
2868
+ "epoch": 0.85,
2869
+ "learning_rate": 0.00019031268894280023,
2870
+ "loss": 0.0334,
2871
+ "step": 4210
2872
+ },
2873
+ {
2874
+ "epoch": 0.85,
2875
+ "learning_rate": 0.00019026737612356094,
2876
+ "loss": 0.0382,
2877
+ "step": 4220
2878
+ },
2879
+ {
2880
+ "epoch": 0.85,
2881
+ "learning_rate": 0.00019022196299536733,
2882
+ "loss": 0.0371,
2883
+ "step": 4230
2884
+ },
2885
+ {
2886
+ "epoch": 0.85,
2887
+ "learning_rate": 0.00019017644960868445,
2888
+ "loss": 0.0353,
2889
+ "step": 4240
2890
+ },
2891
+ {
2892
+ "epoch": 0.86,
2893
+ "learning_rate": 0.00019013083601408863,
2894
+ "loss": 0.0368,
2895
+ "step": 4250
2896
+ },
2897
+ {
2898
+ "epoch": 0.86,
2899
+ "learning_rate": 0.0001900851222622677,
2900
+ "loss": 0.0427,
2901
+ "step": 4260
2902
+ },
2903
+ {
2904
+ "epoch": 0.86,
2905
+ "learning_rate": 0.00019003930840402072,
2906
+ "loss": 0.031,
2907
+ "step": 4270
2908
+ },
2909
+ {
2910
+ "epoch": 0.86,
2911
+ "learning_rate": 0.00018999339449025796,
2912
+ "loss": 0.0377,
2913
+ "step": 4280
2914
+ },
2915
+ {
2916
+ "epoch": 0.86,
2917
+ "learning_rate": 0.00018994738057200099,
2918
+ "loss": 0.0373,
2919
+ "step": 4290
2920
+ },
2921
+ {
2922
+ "epoch": 0.87,
2923
+ "learning_rate": 0.0001899012667003824,
2924
+ "loss": 0.0421,
2925
+ "step": 4300
2926
+ },
2927
+ {
2928
+ "epoch": 0.87,
2929
+ "eval_loss": 0.05914044752717018,
2930
+ "eval_runtime": 1056.4599,
2931
+ "eval_samples_per_second": 6.965,
2932
+ "eval_steps_per_second": 1.742,
2933
+ "step": 4300
2934
+ },
2935
+ {
2936
+ "epoch": 0.87,
2937
+ "learning_rate": 0.00018985505292664587,
2938
+ "loss": 0.0344,
2939
+ "step": 4310
2940
+ },
2941
+ {
2942
+ "epoch": 0.87,
2943
+ "learning_rate": 0.00018980873930214614,
2944
+ "loss": 0.0355,
2945
+ "step": 4320
2946
+ },
2947
+ {
2948
+ "epoch": 0.87,
2949
+ "learning_rate": 0.00018976232587834886,
2950
+ "loss": 0.0416,
2951
+ "step": 4330
2952
+ },
2953
+ {
2954
+ "epoch": 0.87,
2955
+ "learning_rate": 0.00018971581270683062,
2956
+ "loss": 0.0395,
2957
+ "step": 4340
2958
+ },
2959
+ {
2960
+ "epoch": 0.88,
2961
+ "learning_rate": 0.00018966919983927886,
2962
+ "loss": 0.0366,
2963
+ "step": 4350
2964
+ },
2965
+ {
2966
+ "epoch": 0.88,
2967
+ "learning_rate": 0.00018962248732749175,
2968
+ "loss": 0.0368,
2969
+ "step": 4360
2970
+ },
2971
+ {
2972
+ "epoch": 0.88,
2973
+ "learning_rate": 0.0001895756752233782,
2974
+ "loss": 0.0362,
2975
+ "step": 4370
2976
+ },
2977
+ {
2978
+ "epoch": 0.88,
2979
+ "learning_rate": 0.0001895287635789579,
2980
+ "loss": 0.0395,
2981
+ "step": 4380
2982
+ },
2983
+ {
2984
+ "epoch": 0.88,
2985
+ "learning_rate": 0.00018948175244636097,
2986
+ "loss": 0.0462,
2987
+ "step": 4390
2988
+ },
2989
+ {
2990
+ "epoch": 0.89,
2991
+ "learning_rate": 0.00018943464187782828,
2992
+ "loss": 0.0454,
2993
+ "step": 4400
2994
+ },
2995
+ {
2996
+ "epoch": 0.89,
2997
+ "eval_loss": 0.05928103253245354,
2998
+ "eval_runtime": 1029.6162,
2999
+ "eval_samples_per_second": 7.146,
3000
+ "eval_steps_per_second": 1.787,
3001
+ "step": 4400
3002
+ },
3003
+ {
3004
+ "epoch": 0.89,
3005
+ "learning_rate": 0.000189387431925711,
3006
+ "loss": 0.0326,
3007
+ "step": 4410
3008
+ },
3009
+ {
3010
+ "epoch": 0.89,
3011
+ "learning_rate": 0.0001893401226424709,
3012
+ "loss": 0.031,
3013
+ "step": 4420
3014
+ },
3015
+ {
3016
+ "epoch": 0.89,
3017
+ "learning_rate": 0.00018929271408068011,
3018
+ "loss": 0.0408,
3019
+ "step": 4430
3020
+ },
3021
+ {
3022
+ "epoch": 0.89,
3023
+ "step": 4431,
3024
+ "total_flos": 1.532304306097619e+18,
3025
+ "train_loss": 9.867495303766314e-06,
3026
+ "train_runtime": 4.4906,
3027
+ "train_samples_per_second": 15784.097,
3028
+ "train_steps_per_second": 986.506
3029
+ }
3030
+ ],
3031
+ "logging_steps": 10,
3032
+ "max_steps": 4430,
3033
+ "num_input_tokens_seen": 0,
3034
+ "num_train_epochs": 1,
3035
+ "save_steps": 10,
3036
+ "total_flos": 1.532304306097619e+18,
3037
+ "train_batch_size": 4,
3038
+ "trial_name": null,
3039
+ "trial_params": null
3040
+ }
fact/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3730a0743d0c12624c64096dd83e9b8641c7926889ff750fdc910ea65ecf56c
3
+ size 4920
fact/training_eval_loss.png ADDED
fact/training_loss.png ADDED