Training in progress, step 100, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: deepseek-ai/deepseek-coder-6.7b-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "deepseek-ai/deepseek-coder-6.7b-instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3896db9d54dd275f293ce44b2a8ee7a6bfb32fc05fb5791cc76c7f72277434fd
+size 319876032

last-checkpoint/chat_template.jinja

@@ -0,0 +1,5 @@
+{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
+
+'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
+
+' }}{% endif %}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b5598b7e1c9e8fa69d651f219a643f1e2440b0e949de4361e1a81c428d13bc2
+size 162934053

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58eb677c82fe48af1bff9502069aa25a61218643c822f82fdd45ea0d2ed7be16
+size 14645

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48f40d2f08b23557a788305515f32719dcc8b0b8de8713de4a2d45492066d7e6
+size 1465

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<｜begin▁of▁sentence｜>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|EOT|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<｜end▁of▁sentence｜>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,194 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "32000": {
+      "content": "õ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32001": {
+      "content": "÷",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32002": {
+      "content": "Á",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32003": {
+      "content": "ý",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32004": {
+      "content": "À",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32005": {
+      "content": "ÿ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32006": {
+      "content": "ø",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32007": {
+      "content": "ú",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32008": {
+      "content": "þ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32009": {
+      "content": "ü",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32010": {
+      "content": "ù",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32011": {
+      "content": "ö",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32012": {
+      "content": "û",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32013": {
+      "content": "<｜begin▁of▁sentence｜>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32014": {
+      "content": "<｜end▁of▁sentence｜>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32015": {
+      "content": "<｜fim▁hole｜>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32016": {
+      "content": "<｜fim▁begin｜>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32017": {
+      "content": "<｜fim▁end｜>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32018": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32019": {
+      "content": "<|User|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32020": {
+      "content": "<|Assistant|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32021": {
+      "content": "<|EOT|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<｜begin▁of▁sentence｜>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|EOT|>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "model_max_length": 16384,
+  "pad_token": "<｜end▁of▁sentence｜>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizerFast",
+  "unk_token": null,
+  "use_default_system_prompt": false
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,734 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.0346860908775581,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.000346860908775581,
+      "grad_norm": 0.15707533061504364,
+      "learning_rate": 0.0,
+      "loss": 1.0785,
+      "step": 1
+    },
+    {
+      "epoch": 0.000693721817551162,
+      "grad_norm": 0.22791627049446106,
+      "learning_rate": 6.872852233676976e-07,
+      "loss": 1.4474,
+      "step": 2
+    },
+    {
+      "epoch": 0.001040582726326743,
+      "grad_norm": 0.17501138150691986,
+      "learning_rate": 1.3745704467353952e-06,
+      "loss": 1.3096,
+      "step": 3
+    },
+    {
+      "epoch": 0.001387443635102324,
+      "grad_norm": 0.17246569693088531,
+      "learning_rate": 2.061855670103093e-06,
+      "loss": 1.2703,
+      "step": 4
+    },
+    {
+      "epoch": 0.001734304543877905,
+      "grad_norm": 0.15541404485702515,
+      "learning_rate": 2.7491408934707903e-06,
+      "loss": 1.0471,
+      "step": 5
+    },
+    {
+      "epoch": 0.002081165452653486,
+      "grad_norm": 0.17523345351219177,
+      "learning_rate": 3.4364261168384886e-06,
+      "loss": 1.3458,
+      "step": 6
+    },
+    {
+      "epoch": 0.002428026361429067,
+      "grad_norm": 0.17795690894126892,
+      "learning_rate": 4.123711340206186e-06,
+      "loss": 1.2439,
+      "step": 7
+    },
+    {
+      "epoch": 0.002774887270204648,
+      "grad_norm": 0.19390559196472168,
+      "learning_rate": 4.810996563573884e-06,
+      "loss": 1.2889,
+      "step": 8
+    },
+    {
+      "epoch": 0.003121748178980229,
+      "grad_norm": 0.17732024192810059,
+      "learning_rate": 5.498281786941581e-06,
+      "loss": 1.0659,
+      "step": 9
+    },
+    {
+      "epoch": 0.00346860908775581,
+      "grad_norm": 0.2053704559803009,
+      "learning_rate": 6.185567010309279e-06,
+      "loss": 1.2088,
+      "step": 10
+    },
+    {
+      "epoch": 0.003815469996531391,
+      "grad_norm": 0.2579546272754669,
+      "learning_rate": 6.872852233676977e-06,
+      "loss": 1.2702,
+      "step": 11
+    },
+    {
+      "epoch": 0.004162330905306972,
+      "grad_norm": 0.21516655385494232,
+      "learning_rate": 7.560137457044673e-06,
+      "loss": 1.1968,
+      "step": 12
+    },
+    {
+      "epoch": 0.004509191814082553,
+      "grad_norm": 0.24909715354442596,
+      "learning_rate": 8.247422680412371e-06,
+      "loss": 1.165,
+      "step": 13
+    },
+    {
+      "epoch": 0.004856052722858134,
+      "grad_norm": 0.26021990180015564,
+      "learning_rate": 8.934707903780069e-06,
+      "loss": 1.1969,
+      "step": 14
+    },
+    {
+      "epoch": 0.005202913631633715,
+      "grad_norm": 0.22334130108356476,
+      "learning_rate": 9.621993127147768e-06,
+      "loss": 1.1153,
+      "step": 15
+    },
+    {
+      "epoch": 0.005549774540409296,
+      "grad_norm": 0.20358741283416748,
+      "learning_rate": 1.0309278350515464e-05,
+      "loss": 1.2238,
+      "step": 16
+    },
+    {
+      "epoch": 0.005896635449184877,
+      "grad_norm": 0.19010984897613525,
+      "learning_rate": 1.0996563573883161e-05,
+      "loss": 1.2324,
+      "step": 17
+    },
+    {
+      "epoch": 0.006243496357960458,
+      "grad_norm": 0.19241979718208313,
+      "learning_rate": 1.168384879725086e-05,
+      "loss": 1.1586,
+      "step": 18
+    },
+    {
+      "epoch": 0.0065903572667360385,
+      "grad_norm": 0.1602533459663391,
+      "learning_rate": 1.2371134020618558e-05,
+      "loss": 1.2342,
+      "step": 19
+    },
+    {
+      "epoch": 0.00693721817551162,
+      "grad_norm": 0.16475340723991394,
+      "learning_rate": 1.3058419243986255e-05,
+      "loss": 1.1192,
+      "step": 20
+    },
+    {
+      "epoch": 0.007284079084287201,
+      "grad_norm": 0.1598149687051773,
+      "learning_rate": 1.3745704467353954e-05,
+      "loss": 1.1431,
+      "step": 21
+    },
+    {
+      "epoch": 0.007630939993062782,
+      "grad_norm": 0.1715165227651596,
+      "learning_rate": 1.4432989690721649e-05,
+      "loss": 1.1048,
+      "step": 22
+    },
+    {
+      "epoch": 0.007977800901838363,
+      "grad_norm": 0.33843040466308594,
+      "learning_rate": 1.5120274914089346e-05,
+      "loss": 1.0476,
+      "step": 23
+    },
+    {
+      "epoch": 0.008324661810613945,
+      "grad_norm": 0.11684966087341309,
+      "learning_rate": 1.5807560137457044e-05,
+      "loss": 1.0837,
+      "step": 24
+    },
+    {
+      "epoch": 0.008671522719389525,
+      "grad_norm": 0.13994793593883514,
+      "learning_rate": 1.6494845360824743e-05,
+      "loss": 1.3116,
+      "step": 25
+    },
+    {
+      "epoch": 0.009018383628165106,
+      "grad_norm": 0.10933349281549454,
+      "learning_rate": 1.7182130584192442e-05,
+      "loss": 0.85,
+      "step": 26
+    },
+    {
+      "epoch": 0.009365244536940686,
+      "grad_norm": 0.15082964301109314,
+      "learning_rate": 1.7869415807560138e-05,
+      "loss": 1.019,
+      "step": 27
+    },
+    {
+      "epoch": 0.009712105445716268,
+      "grad_norm": 0.17672114074230194,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 1.1169,
+      "step": 28
+    },
+    {
+      "epoch": 0.010058966354491848,
+      "grad_norm": 0.16814199090003967,
+      "learning_rate": 1.9243986254295536e-05,
+      "loss": 1.0976,
+      "step": 29
+    },
+    {
+      "epoch": 0.01040582726326743,
+      "grad_norm": 0.1540689766407013,
+      "learning_rate": 1.9931271477663232e-05,
+      "loss": 1.1427,
+      "step": 30
+    },
+    {
+      "epoch": 0.010752688172043012,
+      "grad_norm": 0.17817096412181854,
+      "learning_rate": 2.0618556701030927e-05,
+      "loss": 1.0834,
+      "step": 31
+    },
+    {
+      "epoch": 0.011099549080818592,
+      "grad_norm": 0.1669071465730667,
+      "learning_rate": 2.1305841924398627e-05,
+      "loss": 1.0102,
+      "step": 32
+    },
+    {
+      "epoch": 0.011446409989594173,
+      "grad_norm": 0.1506875604391098,
+      "learning_rate": 2.1993127147766322e-05,
+      "loss": 0.9786,
+      "step": 33
+    },
+    {
+      "epoch": 0.011793270898369753,
+      "grad_norm": 0.12475559115409851,
+      "learning_rate": 2.268041237113402e-05,
+      "loss": 1.1224,
+      "step": 34
+    },
+    {
+      "epoch": 0.012140131807145335,
+      "grad_norm": 0.15751752257347107,
+      "learning_rate": 2.336769759450172e-05,
+      "loss": 0.9001,
+      "step": 35
+    },
+    {
+      "epoch": 0.012486992715920915,
+      "grad_norm": 0.1342693269252777,
+      "learning_rate": 2.4054982817869417e-05,
+      "loss": 1.2336,
+      "step": 36
+    },
+    {
+      "epoch": 0.012833853624696497,
+      "grad_norm": 0.1096595898270607,
+      "learning_rate": 2.4742268041237116e-05,
+      "loss": 0.9728,
+      "step": 37
+    },
+    {
+      "epoch": 0.013180714533472077,
+      "grad_norm": 0.10404551774263382,
+      "learning_rate": 2.542955326460481e-05,
+      "loss": 0.9728,
+      "step": 38
+    },
+    {
+      "epoch": 0.013527575442247659,
+      "grad_norm": 0.08569219708442688,
+      "learning_rate": 2.611683848797251e-05,
+      "loss": 0.8819,
+      "step": 39
+    },
+    {
+      "epoch": 0.01387443635102324,
+      "grad_norm": 0.1122739389538765,
+      "learning_rate": 2.6804123711340206e-05,
+      "loss": 1.1467,
+      "step": 40
+    },
+    {
+      "epoch": 0.01422129725979882,
+      "grad_norm": 0.11138486862182617,
+      "learning_rate": 2.749140893470791e-05,
+      "loss": 0.8497,
+      "step": 41
+    },
+    {
+      "epoch": 0.014568158168574402,
+      "grad_norm": 0.11229175329208374,
+      "learning_rate": 2.81786941580756e-05,
+      "loss": 0.9303,
+      "step": 42
+    },
+    {
+      "epoch": 0.014915019077349982,
+      "grad_norm": 0.10118245333433151,
+      "learning_rate": 2.8865979381443297e-05,
+      "loss": 0.87,
+      "step": 43
+    },
+    {
+      "epoch": 0.015261879986125564,
+      "grad_norm": 0.08575966954231262,
+      "learning_rate": 2.9553264604811e-05,
+      "loss": 0.9798,
+      "step": 44
+    },
+    {
+      "epoch": 0.015608740894901144,
+      "grad_norm": 0.1441422998905182,
+      "learning_rate": 3.0240549828178692e-05,
+      "loss": 1.0341,
+      "step": 45
+    },
+    {
+      "epoch": 0.015955601803676726,
+      "grad_norm": 0.10418319702148438,
+      "learning_rate": 3.0927835051546395e-05,
+      "loss": 1.0693,
+      "step": 46
+    },
+    {
+      "epoch": 0.016302462712452306,
+      "grad_norm": 0.10118691623210907,
+      "learning_rate": 3.161512027491409e-05,
+      "loss": 0.8208,
+      "step": 47
+    },
+    {
+      "epoch": 0.01664932362122789,
+      "grad_norm": 0.11025723814964294,
+      "learning_rate": 3.230240549828179e-05,
+      "loss": 0.9943,
+      "step": 48
+    },
+    {
+      "epoch": 0.01699618453000347,
+      "grad_norm": 0.09817607700824738,
+      "learning_rate": 3.2989690721649485e-05,
+      "loss": 1.0994,
+      "step": 49
+    },
+    {
+      "epoch": 0.01734304543877905,
+      "grad_norm": 0.16049663722515106,
+      "learning_rate": 3.3676975945017185e-05,
+      "loss": 0.9386,
+      "step": 50
+    },
+    {
+      "epoch": 0.01768990634755463,
+      "grad_norm": 0.13163086771965027,
+      "learning_rate": 3.4364261168384884e-05,
+      "loss": 1.1124,
+      "step": 51
+    },
+    {
+      "epoch": 0.018036767256330213,
+      "grad_norm": 0.10064269602298737,
+      "learning_rate": 3.5051546391752576e-05,
+      "loss": 1.0587,
+      "step": 52
+    },
+    {
+      "epoch": 0.018383628165105793,
+      "grad_norm": 0.1133924201130867,
+      "learning_rate": 3.5738831615120275e-05,
+      "loss": 0.8193,
+      "step": 53
+    },
+    {
+      "epoch": 0.018730489073881373,
+      "grad_norm": 0.10126468539237976,
+      "learning_rate": 3.6426116838487974e-05,
+      "loss": 1.0768,
+      "step": 54
+    },
+    {
+      "epoch": 0.019077349982656956,
+      "grad_norm": 0.09082435816526413,
+      "learning_rate": 3.7113402061855674e-05,
+      "loss": 0.9013,
+      "step": 55
+    },
+    {
+      "epoch": 0.019424210891432536,
+      "grad_norm": 0.10581611841917038,
+      "learning_rate": 3.7800687285223366e-05,
+      "loss": 1.1382,
+      "step": 56
+    },
+    {
+      "epoch": 0.019771071800208116,
+      "grad_norm": 0.10491088777780533,
+      "learning_rate": 3.848797250859107e-05,
+      "loss": 0.963,
+      "step": 57
+    },
+    {
+      "epoch": 0.020117932708983696,
+      "grad_norm": 0.09974328428506851,
+      "learning_rate": 3.9175257731958764e-05,
+      "loss": 0.9865,
+      "step": 58
+    },
+    {
+      "epoch": 0.02046479361775928,
+      "grad_norm": 0.11049463599920273,
+      "learning_rate": 3.9862542955326463e-05,
+      "loss": 1.0206,
+      "step": 59
+    },
+    {
+      "epoch": 0.02081165452653486,
+      "grad_norm": 0.10978296399116516,
+      "learning_rate": 4.054982817869416e-05,
+      "loss": 1.0222,
+      "step": 60
+    },
+    {
+      "epoch": 0.02115851543531044,
+      "grad_norm": 0.11269783973693848,
+      "learning_rate": 4.1237113402061855e-05,
+      "loss": 0.996,
+      "step": 61
+    },
+    {
+      "epoch": 0.021505376344086023,
+      "grad_norm": 0.09804196655750275,
+      "learning_rate": 4.1924398625429554e-05,
+      "loss": 1.049,
+      "step": 62
+    },
+    {
+      "epoch": 0.021852237252861603,
+      "grad_norm": 0.1048455759882927,
+      "learning_rate": 4.261168384879725e-05,
+      "loss": 0.9979,
+      "step": 63
+    },
+    {
+      "epoch": 0.022199098161637183,
+      "grad_norm": 0.1202562153339386,
+      "learning_rate": 4.329896907216495e-05,
+      "loss": 1.1293,
+      "step": 64
+    },
+    {
+      "epoch": 0.022545959070412763,
+      "grad_norm": 0.11165941506624222,
+      "learning_rate": 4.3986254295532645e-05,
+      "loss": 1.0851,
+      "step": 65
+    },
+    {
+      "epoch": 0.022892819979188347,
+      "grad_norm": 0.10654924064874649,
+      "learning_rate": 4.467353951890035e-05,
+      "loss": 0.7404,
+      "step": 66
+    },
+    {
+      "epoch": 0.023239680887963927,
+      "grad_norm": 0.10879170894622803,
+      "learning_rate": 4.536082474226804e-05,
+      "loss": 0.956,
+      "step": 67
+    },
+    {
+      "epoch": 0.023586541796739507,
+      "grad_norm": 0.11754804849624634,
+      "learning_rate": 4.6048109965635736e-05,
+      "loss": 1.0658,
+      "step": 68
+    },
+    {
+      "epoch": 0.023933402705515087,
+      "grad_norm": 0.11166474968194962,
+      "learning_rate": 4.673539518900344e-05,
+      "loss": 0.9424,
+      "step": 69
+    },
+    {
+      "epoch": 0.02428026361429067,
+      "grad_norm": 0.11637741327285767,
+      "learning_rate": 4.7422680412371134e-05,
+      "loss": 1.097,
+      "step": 70
+    },
+    {
+      "epoch": 0.02462712452306625,
+      "grad_norm": 0.10908675938844681,
+      "learning_rate": 4.810996563573883e-05,
+      "loss": 0.957,
+      "step": 71
+    },
+    {
+      "epoch": 0.02497398543184183,
+      "grad_norm": 0.0941724181175232,
+      "learning_rate": 4.879725085910653e-05,
+      "loss": 1.0049,
+      "step": 72
+    },
+    {
+      "epoch": 0.025320846340617414,
+      "grad_norm": 0.12279584258794785,
+      "learning_rate": 4.948453608247423e-05,
+      "loss": 0.9061,
+      "step": 73
+    },
+    {
+      "epoch": 0.025667707249392994,
+      "grad_norm": 0.10812018066644669,
+      "learning_rate": 5.0171821305841924e-05,
+      "loss": 1.0579,
+      "step": 74
+    },
+    {
+      "epoch": 0.026014568158168574,
+      "grad_norm": 0.10963872075080872,
+      "learning_rate": 5.085910652920962e-05,
+      "loss": 0.9971,
+      "step": 75
+    },
+    {
+      "epoch": 0.026361429066944154,
+      "grad_norm": 0.10136889666318893,
+      "learning_rate": 5.1546391752577315e-05,
+      "loss": 0.9668,
+      "step": 76
+    },
+    {
+      "epoch": 0.026708289975719737,
+      "grad_norm": 0.1343003660440445,
+      "learning_rate": 5.223367697594502e-05,
+      "loss": 0.8968,
+      "step": 77
+    },
+    {
+      "epoch": 0.027055150884495317,
+      "grad_norm": 0.12735772132873535,
+      "learning_rate": 5.292096219931272e-05,
+      "loss": 1.0938,
+      "step": 78
+    },
+    {
+      "epoch": 0.027402011793270897,
+      "grad_norm": 0.11420556902885437,
+      "learning_rate": 5.360824742268041e-05,
+      "loss": 0.9915,
+      "step": 79
+    },
+    {
+      "epoch": 0.02774887270204648,
+      "grad_norm": 0.11709192395210266,
+      "learning_rate": 5.4295532646048105e-05,
+      "loss": 1.1102,
+      "step": 80
+    },
+    {
+      "epoch": 0.02809573361082206,
+      "grad_norm": 0.10012415051460266,
+      "learning_rate": 5.498281786941582e-05,
+      "loss": 0.8977,
+      "step": 81
+    },
+    {
+      "epoch": 0.02844259451959764,
+      "grad_norm": 0.10154993832111359,
+      "learning_rate": 5.567010309278351e-05,
+      "loss": 0.8501,
+      "step": 82
+    },
+    {
+      "epoch": 0.02878945542837322,
+      "grad_norm": 0.11879187077283859,
+      "learning_rate": 5.63573883161512e-05,
+      "loss": 0.914,
+      "step": 83
+    },
+    {
+      "epoch": 0.029136316337148804,
+      "grad_norm": 0.09363780915737152,
+      "learning_rate": 5.70446735395189e-05,
+      "loss": 1.0094,
+      "step": 84
+    },
+    {
+      "epoch": 0.029483177245924384,
+      "grad_norm": 0.1178029254078865,
+      "learning_rate": 5.7731958762886594e-05,
+      "loss": 1.0515,
+      "step": 85
+    },
+    {
+      "epoch": 0.029830038154699964,
+      "grad_norm": 0.10389260202646255,
+      "learning_rate": 5.84192439862543e-05,
+      "loss": 0.9075,
+      "step": 86
+    },
+    {
+      "epoch": 0.030176899063475548,
+      "grad_norm": 0.10177561640739441,
+      "learning_rate": 5.9106529209622e-05,
+      "loss": 0.9255,
+      "step": 87
+    },
+    {
+      "epoch": 0.030523759972251128,
+      "grad_norm": 0.11313007026910782,
+      "learning_rate": 5.979381443298969e-05,
+      "loss": 1.069,
+      "step": 88
+    },
+    {
+      "epoch": 0.030870620881026708,
+      "grad_norm": 0.12532612681388855,
+      "learning_rate": 6.0481099656357384e-05,
+      "loss": 0.994,
+      "step": 89
+    },
+    {
+      "epoch": 0.031217481789802288,
+      "grad_norm": 0.12486390024423599,
+      "learning_rate": 6.116838487972509e-05,
+      "loss": 0.9215,
+      "step": 90
+    },
+    {
+      "epoch": 0.03156434269857787,
+      "grad_norm": 0.12236090749502182,
+      "learning_rate": 6.185567010309279e-05,
+      "loss": 0.8634,
+      "step": 91
+    },
+    {
+      "epoch": 0.03191120360735345,
+      "grad_norm": 0.1063942238688469,
+      "learning_rate": 6.254295532646049e-05,
+      "loss": 0.9696,
+      "step": 92
+    },
+    {
+      "epoch": 0.03225806451612903,
+      "grad_norm": 0.10683036595582962,
+      "learning_rate": 6.323024054982817e-05,
+      "loss": 0.8937,
+      "step": 93
+    },
+    {
+      "epoch": 0.03260492542490461,
+      "grad_norm": 0.11124654859304428,
+      "learning_rate": 6.391752577319587e-05,
+      "loss": 0.9895,
+      "step": 94
+    },
+    {
+      "epoch": 0.03295178633368019,
+      "grad_norm": 0.1119171604514122,
+      "learning_rate": 6.460481099656359e-05,
+      "loss": 0.9361,
+      "step": 95
+    },
+    {
+      "epoch": 0.03329864724245578,
+      "grad_norm": 0.10494361817836761,
+      "learning_rate": 6.529209621993127e-05,
+      "loss": 1.0201,
+      "step": 96
+    },
+    {
+      "epoch": 0.03364550815123136,
+      "grad_norm": 0.13596506416797638,
+      "learning_rate": 6.597938144329897e-05,
+      "loss": 1.0507,
+      "step": 97
+    },
+    {
+      "epoch": 0.03399236906000694,
+      "grad_norm": 0.10757894068956375,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 0.9552,
+      "step": 98
+    },
+    {
+      "epoch": 0.03433922996878252,
+      "grad_norm": 0.13154934346675873,
+      "learning_rate": 6.735395189003437e-05,
+      "loss": 0.9236,
+      "step": 99
+    },
+    {
+      "epoch": 0.0346860908775581,
+      "grad_norm": 0.11746134608983994,
+      "learning_rate": 6.804123711340207e-05,
+      "loss": 0.9037,
+      "step": 100
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 100,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 437,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.46559191072768e+17,
+  "train_batch_size": 60,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d47a50196bdbcae08cec2a692371c54cc783d8c814ea7ce45ed092d4e0f1eeb1
+size 7569