Maskara Phase 2 retrain with targeted augmentation

README.md

@@ -1,221 +1,73 @@
 ---
 library_name: transformers
-license: apache-2.0
-base_model: google-bert/bert-base-uncased
 tags:
-- token-classification
-- pii
-- indian-pii
-- privacy
-- aadhaar
-- pan-card
-- named-entity-recognition
-metrics:
-- precision
-- recall
-- f1
-- accuracy
+- generated_from_trainer
+model-index:
+- name: maskara
+  results: []
 ---
 
-# Maskara
-
-Maskara is a token-classification model for detecting Indian PII (Personally Identifiable Information) in unstructured text. It is built by fine-tuning [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased) on the [`somukandula/maskara-indian-pii-200k`](https://huggingface.co/datasets/somukandula/maskara-indian-pii-200k) dataset.
-
-The model identifies 17 PII entity types using a BIO tagging scheme. Loading the model and running inference requires no code changes from the previous release:
-
-```python
-from transformers import AutoModelForTokenClassification, AutoTokenizer
-
-model = AutoModelForTokenClassification.from_pretrained("somukandula/maskara")
-tokenizer = AutoTokenizer.from_pretrained("somukandula/maskara")
-```
-
-## Supported entities
-
-| Entity | Description | Example |
-|--------|-------------|---------|
-| `AADHAAR` | 12-digit Aadhaar number | `1234 5678 9012` |
-| `ADDRESS` | Indian postal address | `Flat 12, Sector 45, Gurgaon` |
-| `API_KEY` | API key / secret token | `sk_live_abc123xyz` |
-| `CREDIT_CARD` | Credit/debit card number | `4532-1234-5678-9012` |
-| `DATE_OF_BIRTH` | Date of birth | `15/08/1990` |
-| `DRIVER_LICENSE` | Indian driving licence | `MH02 20123456789` |
-| `EMAIL` | Email address | `user@example.com` |
-| `IP_ADDRESS` | IPv4 / IPv6 address | `192.168.1.1` |
-| `PAN_CARD` | 10-character PAN | `ABCDE1234F` |
-| `PASSPORT` | Indian passport number | `A1234567` |
-| `PASSWORD` | Password or passphrase | `P@ssw0rd123` |
-| `PERSON_NAME` | Person's full or partial name | `Rahul Sharma` |
-| `PHONE` | Indian phone/landline | `+91 98765 43210` |
-| `SSN` | Social Security Number | `123-45-6789` |
-| `UPI_ID` | UPI handle | `user@upi` |
-| `USERNAME` | Username / handle | `rahul_s` |
-| `VEHICLE_REG` | Indian vehicle registration | `MH 01 AB 1234` |
-
-**Note:** The `LOCATION` label from the earlier release has been removed. The classifier head now matches exactly the 17 entities above.
-
-## Model architecture
-
-| Property | Value |
-|----------|-------|
-| Base model | `google-bert/bert-base-uncased` |
-| Architecture | `BertForTokenClassification` |
-| Hidden size | 768 |
-| Layers | 12 |
-| Attention heads | 12 |
-| Parameters | ~109M |
-| Max sequence length | 256 |
-| Labels | 35 (O + 17 B- + 17 I-) |
-
-## Training data
-
-The training corpus was generated with an expanded synthetic generator covering:
-
-- **150+** multi-entity templates
-- **80+** single-entity templates
-- **6–10** formatting variations per entity
-- **40%** Hinglish / code-mixed text
-- **15%** hard-negative examples
-- **20%** structured or noisy documents
-
-Formats represented include WhatsApp chats, SMS, emails, customer-support conversations, HTML, Markdown, JSON, CSV, SQL, logs, OCR, bank statements, government forms, Aadhaar letters, PAN documents, resumes and configuration files.
-
-The dataset is published at [`somukandula/maskara-indian-pii-200k`](https://huggingface.co/datasets/somukandula/maskara-indian-pii-200k).
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# maskara
+
+This model was trained from scratch on an unknown dataset.
+It achieves the following results on the evaluation set:
+- eval_loss: 0.5814
+- eval_model_preparation_time: 0.0031
+- eval_precision: 0.8442
+- eval_recall: 0.9495
+- eval_f1: 0.8937
+- eval_accuracy: 0.9504
+- eval_AADHAAR_f1: 0.9775
+- eval_ADDRESS_f1: 0.7162
+- eval_API_KEY_f1: 1.0
+- eval_DATE_OF_BIRTH_f1: 0.7435
+- eval_DRIVER_LICENSE_f1: 0.8299
+- eval_EMAIL_f1: 1.0
+- eval_IP_ADDRESS_f1: 0.8584
+- eval_PAN_CARD_f1: 0.9697
+- eval_PASSWORD_f1: 0.0
+- eval_PERSON_NAME_f1: 0.8484
+- eval_PHONE_f1: 0.9418
+- eval_UPI_ID_f1: 0.9990
+- eval_USERNAME_f1: 0.9029
+- eval_VEHICLE_REG_f1: 0.7440
+- eval_runtime: 6.5348
+- eval_samples_per_second: 397.868
+- eval_steps_per_second: 6.274
+- epoch: 0
+- step: 0
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
 
 ## Training procedure
 
-| Hyperparameter | Value |
-|----------------|-------|
-| Epochs | 5 |
-| Learning rate | 3e-5 |
-| Batch size (train) | 32 |
-| Batch size (eval) | 64 |
-| Max sequence length | 256 |
-| Optimizer | AdamW |
-| Weight decay | 0.01 |
-| LR scheduler | Linear with warmup |
-| Warmup ratio | 0.1 |
-| Precision | bf16 |
-| Early stopping patience | 2 epochs |
-| Seed | 2026 |
-
-The model was trained from `bert-base-uncased` pretrained weights. The previous 2-layer compressed encoder checkpoint was replaced.
-
-## Evaluation methodology
-
-Two independent evaluation splits are used:
-
-1. **Template-disjoint validation** — entire template families are excluded from training to measure generalization rather than memorization.
-2. **Real-world evaluation** — a manually curated held-out set of Indian text from support conversations, forms, OCR, public documents, chat logs and semi-structured text. This split is never used during training.
-
-Metrics are computed at the token BIO level using `seqeval`.
-
-## Results
-
-### Overall
-
-| Split | Precision | Recall | F1 | Accuracy |
-|-------|-----------|--------|------|----------|
-| Template-disjoint eval | 0.7870 | 0.9045 | **0.8417** | 0.9437 |
-| Real-world eval | 0.8320 | 0.9493 | **0.8868** | 0.9423 |
-
-### Per-entity F1 — template-disjoint eval
-
-| Entity | F1 |
-|--------|-----|
-| ADDRESS | 0.9935 |
-| API_KEY | 1.0000 |
-| DATE_OF_BIRTH | 0.9573 |
-| EMAIL | 0.9677 |
-| IP_ADDRESS | 0.9986 |
-| PAN_CARD | 0.8860 |
-| PASSPORT | 0.9983 |
-| PERSON_NAME | 0.9404 |
-| SSN | 1.0000 |
-| UPI_ID | 0.9995 |
-| VEHICLE_REG | 0.8286 |
-| PASSWORD | 0.7374 |
-| PHONE | 0.7018 |
-| AADHAAR | 0.7666 |
-| CREDIT_CARD | 0.5588 |
-| USERNAME | 0.6328 |
-| DRIVER_LICENSE | 0.3872 |
-
-### Per-entity F1 — real-world eval
-
-| Entity | F1 |
-|--------|-----|
-| AADHAAR | 0.9719 |
-| DATE_OF_BIRTH | 0.9877 |
-| DRIVER_LICENSE | 1.0000 |
-| EMAIL | 1.0000 |
-| PAN_CARD | 1.0000 |
-| PHONE | 0.9462 |
-| UPI_ID | 1.0000 |
-| VEHICLE_REG | 1.0000 |
-| PERSON_NAME | 0.8578 |
-| ADDRESS | 0.6957 |
-| API_KEY | 0.8032 |
-| IP_ADDRESS | 0.6969 |
-| USERNAME | 0.6329 |
-| CREDIT_CARD | 0.0000 |
-| PASSWORD | 0.0000 |
-
-## Phase 2 Training Update
-
-This release is a major continued training update to the original Maskara model, not a new model family.
-
-### What changed
-
-- **Architecture upgrade:** replaced the 2-layer, 128-hidden compressed encoder with `bert-base-uncased`, increasing capacity from ~16MB to ~416MB/109M parameters.
-- **Dataset expansion:** generator rewritten with 150+ multi-entity and 80+ single-entity templates, 6–10 formatting variations per entity, 40% Hinglish/code-mixed text, 15% hard negatives and 20% structured/noisy documents.
-- **Label cleanup:** removed the unused `LOCATION` class. The head now outputs exactly the documented 17 entities.
-- **Evaluation overhaul:** replaced the old template-overlapping split with a template-disjoint validation set and a separate real-world held-out set.
-- **Evaluation-first training:** future iterations will use an error-analysis loop (false positives/negatives → categorization → targeted synthetic examples → retrain).
-
-### Current status vs. success criteria
-
-| Criterion | Target | Current | Status |
-|-----------|--------|---------|--------|
-| Real-world F1 | ≥ 90% | 88.68% | **Not met** |
-| Template-disjoint F1 | ≥ 95% | 84.17% | **Not met** |
-| No entity below 80% F1 | ≥ 80% per entity | 9/17 below 80% on template-disjoint; 7/17 below 80% on real-world | **Not met** |
-
-The model improved substantially on most entities, especially `EMAIL`, `PHONE`, `AADHAAR`, `PAN_CARD`, `UPI_ID`, `VEHICLE_REG` and `DATE_OF_BIRTH`, but it still struggles with `CREDIT_CARD`, `PASSWORD`, `DRIVER_LICENSE`, `USERNAME`, `ADDRESS`, `IP_ADDRESS` and some `PHONE`/`AADHAAR` formatting variants. The next iteration of the error-analysis loop should target these weaknesses.
-
-## Known limitations
-
-- Performance on real-world text is below the 90% F1 target; further targeted data augmentation and possibly longer fine-tuning are needed.
-- Credit-card and password detection are close to zero on the real-world eval, indicating insufficient diversity or under-representation in the current generator.
-- Address and IP_ADDRESS spans are often fragmented or mis-bounded due to high format variability.
-- The model is English-centric; while Hinglish coverage was increased, other Indian languages are not explicitly supported.
-- Synthetic data cannot fully capture real-world noise, typos, OCR artifacts and adversarial redaction.
-
-## Release history
-
-| Version | Date | Base model | Notes |
-|---------|------|------------|-------|
-| v1.0 | 2024 | `google/bert_uncased_L-2_H-128_A-2` | Initial 2-layer compressed encoder, 13 entities including `LOCATION`. |
-| v2.0 (Phase 2) | 2026-06-27 | `google-bert/bert-base-uncased` | Architecture upgrade, 17 entities, expanded dataset, template-disjoint + real-world evaluation. |
-
-## Future roadmap
-
-- Close the gap on `CREDIT_CARD`, `PASSWORD`, `DRIVER_LICENSE`, `USERNAME`, `ADDRESS` and `IP_ADDRESS` through targeted synthetic examples identified by the error-analysis loop.
-- Add real-world annotated samples for the weakest entities.
-- Experiment with longer sequences (512), label smoothing, and span-based losses.
-- Evaluate distilled / smaller BERT variants for deployment speed without dropping accuracy.
-
-## Citation
+### Training hyperparameters
 
-```bibtex
-@misc{maskara2026,
-  title={Maskara: Indian PII Detection with BERT},
-  author={Somu Kandula},
-  year={2026},
-  howpublished={\url{https://huggingface.co/somukandula/maskara}}
-}
-```
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 64
+- seed: 42
+- optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 3.0
 
-## License
+### Framework versions
 
-Apache-2.0
+- Transformers 5.12.1
+- Pytorch 2.12.1+cu130
+- Datasets 5.0.0
+- Tokenizers 0.22.2

full-openpii-500k/checkpoints/all_results.json

@@ -0,0 +1,11 @@
+{
+    "epoch": 1.0,
+    "eval_accuracy": 0.9452715942731323,
+    "eval_f1": 0.5242530574333041,
+    "eval_loss": 0.16400153934955597,
+    "eval_precision": 0.4926295883237162,
+    "eval_recall": 0.5602150537634408,
+    "eval_runtime": 10.1647,
+    "eval_samples_per_second": 1967.587,
+    "eval_steps_per_second": 30.793
+}

full-openpii-500k/checkpoints/checkpoint-12000/config.json

@@ -0,0 +1,86 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "id2label": {
+    "0": "O",
+    "1": "B-ADDRESS",
+    "2": "I-ADDRESS",
+    "3": "B-API_KEY",
+    "4": "I-API_KEY",
+    "5": "B-CREDIT_CARD",
+    "6": "I-CREDIT_CARD",
+    "7": "B-DATE_OF_BIRTH",
+    "8": "I-DATE_OF_BIRTH",
+    "9": "B-DRIVER_LICENSE",
+    "10": "I-DRIVER_LICENSE",
+    "11": "B-EMAIL",
+    "12": "I-EMAIL",
+    "13": "B-IP_ADDRESS",
+    "14": "I-IP_ADDRESS",
+    "15": "B-LOCATION",
+    "16": "I-LOCATION",
+    "17": "B-PASSWORD",
+    "18": "I-PASSWORD",
+    "19": "B-PERSON_NAME",
+    "20": "I-PERSON_NAME",
+    "21": "B-PHONE",
+    "22": "I-PHONE",
+    "23": "B-SSN",
+    "24": "I-SSN",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "is_decoder": false,
+  "label2id": {
+    "B-ADDRESS": 1,
+    "B-API_KEY": 3,
+    "B-CREDIT_CARD": 5,
+    "B-DATE_OF_BIRTH": 7,
+    "B-DRIVER_LICENSE": 9,
+    "B-EMAIL": 11,
+    "B-IP_ADDRESS": 13,
+    "B-LOCATION": 15,
+    "B-PASSWORD": 17,
+    "B-PERSON_NAME": 19,
+    "B-PHONE": 21,
+    "B-SSN": 23,
+    "B-USERNAME": 25,
+    "I-ADDRESS": 2,
+    "I-API_KEY": 4,
+    "I-CREDIT_CARD": 6,
+    "I-DATE_OF_BIRTH": 8,
+    "I-DRIVER_LICENSE": 10,
+    "I-EMAIL": 12,
+    "I-IP_ADDRESS": 14,
+    "I-LOCATION": 16,
+    "I-PASSWORD": 18,
+    "I-PERSON_NAME": 20,
+    "I-PHONE": 22,
+    "I-SSN": 24,
+    "I-USERNAME": 26,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

full-openpii-500k/checkpoints/checkpoint-12000/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08f8733edd4aa020e99d3741709b94d614dd9c6ff463049251ba56a29f1caece
+size 17495980

full-openpii-500k/checkpoints/checkpoint-12000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc19cbbcc0b0c07666f6b4e04940765423d3f9dd6fbdf491e92e2f76c557d6c8
+size 35016907

full-openpii-500k/checkpoints/checkpoint-12000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1c8af326e76e6c768daf438a48c8693c64020409368138d2582ae3f52a06110
+size 14645

full-openpii-500k/checkpoints/checkpoint-12000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82fd182149e40dac4a79aec92d2c08d42a7a3b82bcc4de291334033484a5a4c6
+size 1383

full-openpii-500k/checkpoints/checkpoint-12000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:195c57e462b912696de44baac4fa514f9d3f20683803d9342b3be0b79a0b4a46
+size 1465

full-openpii-500k/checkpoints/checkpoint-12000/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 160,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

full-openpii-500k/checkpoints/checkpoint-12000/trainer_state.json

@@ -0,0 +1,526 @@
+{
+  "best_global_step": 12000,
+  "best_metric": 0.520962146523804,
+  "best_model_checkpoint": "/outputs/full-openpii-500k/checkpoints/checkpoint-12000",
+  "epoch": 0.827300930713547,
+  "eval_steps": 2000,
+  "global_step": 12000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013788348845225784,
+      "grad_norm": 1.3536444902420044,
+      "learning_rate": 1.972561185798001e-05,
+      "loss": 0.7454490661621094,
+      "step": 200
+    },
+    {
+      "epoch": 0.027576697690451568,
+      "grad_norm": 1.3048033714294434,
+      "learning_rate": 1.944984488107549e-05,
+      "loss": 0.5720141220092774,
+      "step": 400
+    },
+    {
+      "epoch": 0.04136504653567735,
+      "grad_norm": 1.9288572072982788,
+      "learning_rate": 1.9174077904170978e-05,
+      "loss": 0.4927207946777344,
+      "step": 600
+    },
+    {
+      "epoch": 0.055153395380903136,
+      "grad_norm": 1.7550076246261597,
+      "learning_rate": 1.889831092726646e-05,
+      "loss": 0.4657698440551758,
+      "step": 800
+    },
+    {
+      "epoch": 0.06894174422612892,
+      "grad_norm": 1.7711143493652344,
+      "learning_rate": 1.8622543950361945e-05,
+      "loss": 0.42969669342041017,
+      "step": 1000
+    },
+    {
+      "epoch": 0.0827300930713547,
+      "grad_norm": 2.50980544090271,
+      "learning_rate": 1.834677697345743e-05,
+      "loss": 0.41879753112792967,
+      "step": 1200
+    },
+    {
+      "epoch": 0.09651844191658049,
+      "grad_norm": 1.705670952796936,
+      "learning_rate": 1.8071009996552916e-05,
+      "loss": 0.385702018737793,
+      "step": 1400
+    },
+    {
+      "epoch": 0.11030679076180627,
+      "grad_norm": 1.445986032485962,
+      "learning_rate": 1.77952430196484e-05,
+      "loss": 0.36213504791259765,
+      "step": 1600
+    },
+    {
+      "epoch": 0.12409513960703206,
+      "grad_norm": 1.4295554161071777,
+      "learning_rate": 1.7519476042743883e-05,
+      "loss": 0.351446533203125,
+      "step": 1800
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "grad_norm": 1.4451907873153687,
+      "learning_rate": 1.7243709065839366e-05,
+      "loss": 0.34252952575683593,
+      "step": 2000
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "eval_accuracy": 0.9116923546949569,
+      "eval_f1": 0.3809941841325579,
+      "eval_loss": 0.2862004041671753,
+      "eval_precision": 0.37910138123055087,
+      "eval_recall": 0.38290598290598293,
+      "eval_runtime": 11.153,
+      "eval_samples_per_second": 1793.238,
+      "eval_steps_per_second": 28.064,
+      "step": 2000
+    },
+    {
+      "epoch": 0.15167183729748362,
+      "grad_norm": 1.3401026725769043,
+      "learning_rate": 1.6967942088934853e-05,
+      "loss": 0.33029396057128907,
+      "step": 2200
+    },
+    {
+      "epoch": 0.1654601861427094,
+      "grad_norm": 1.4941850900650024,
+      "learning_rate": 1.6692175112030337e-05,
+      "loss": 0.31315877914428714,
+      "step": 2400
+    },
+    {
+      "epoch": 0.1792485349879352,
+      "grad_norm": 1.2174286842346191,
+      "learning_rate": 1.641640813512582e-05,
+      "loss": 0.31084712982177737,
+      "step": 2600
+    },
+    {
+      "epoch": 0.19303688383316098,
+      "grad_norm": 1.6435197591781616,
+      "learning_rate": 1.6140641158221304e-05,
+      "loss": 0.30205060958862306,
+      "step": 2800
+    },
+    {
+      "epoch": 0.20682523267838676,
+      "grad_norm": 2.0023410320281982,
+      "learning_rate": 1.586487418131679e-05,
+      "loss": 0.2959835815429688,
+      "step": 3000
+    },
+    {
+      "epoch": 0.22061358152361255,
+      "grad_norm": 1.1106364727020264,
+      "learning_rate": 1.558910720441227e-05,
+      "loss": 0.28498205184936526,
+      "step": 3200
+    },
+    {
+      "epoch": 0.23440193036883833,
+      "grad_norm": 1.951019525527954,
+      "learning_rate": 1.5313340227507758e-05,
+      "loss": 0.2866662406921387,
+      "step": 3400
+    },
+    {
+      "epoch": 0.2481902792140641,
+      "grad_norm": 1.2675732374191284,
+      "learning_rate": 1.5037573250603242e-05,
+      "loss": 0.2802707290649414,
+      "step": 3600
+    },
+    {
+      "epoch": 0.2619786280592899,
+      "grad_norm": 1.129682183265686,
+      "learning_rate": 1.4761806273698727e-05,
+      "loss": 0.27155515670776365,
+      "step": 3800
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "grad_norm": 1.1867386102676392,
+      "learning_rate": 1.448603929679421e-05,
+      "loss": 0.2704596519470215,
+      "step": 4000
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "eval_accuracy": 0.9294057690079118,
+      "eval_f1": 0.4474882331063317,
+      "eval_loss": 0.2208142876625061,
+      "eval_precision": 0.4183392082104405,
+      "eval_recall": 0.48100358422939066,
+      "eval_runtime": 10.0377,
+      "eval_samples_per_second": 1992.497,
+      "eval_steps_per_second": 31.183,
+      "step": 4000
+    },
+    {
+      "epoch": 0.28955532574974147,
+      "grad_norm": 1.4080299139022827,
+      "learning_rate": 1.4210272319889696e-05,
+      "loss": 0.2664038848876953,
+      "step": 4200
+    },
+    {
+      "epoch": 0.30334367459496725,
+      "grad_norm": 1.4007683992385864,
+      "learning_rate": 1.393450534298518e-05,
+      "loss": 0.26422006607055665,
+      "step": 4400
+    },
+    {
+      "epoch": 0.31713202344019303,
+      "grad_norm": 1.7308765649795532,
+      "learning_rate": 1.3658738366080664e-05,
+      "loss": 0.25533161163330076,
+      "step": 4600
+    },
+    {
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.7256940603256226,
+      "learning_rate": 1.3382971389176146e-05,
+      "loss": 0.25111486434936525,
+      "step": 4800
+    },
+    {
+      "epoch": 0.3447087211306446,
+      "grad_norm": 1.039595365524292,
+      "learning_rate": 1.3107204412271633e-05,
+      "loss": 0.24599828720092773,
+      "step": 5000
+    },
+    {
+      "epoch": 0.3584970699758704,
+      "grad_norm": 1.8612524271011353,
+      "learning_rate": 1.2831437435367115e-05,
+      "loss": 0.2489796257019043,
+      "step": 5200
+    },
+    {
+      "epoch": 0.37228541882109617,
+      "grad_norm": 2.6072161197662354,
+      "learning_rate": 1.25556704584626e-05,
+      "loss": 0.2444438934326172,
+      "step": 5400
+    },
+    {
+      "epoch": 0.38607376766632195,
+      "grad_norm": 1.265944242477417,
+      "learning_rate": 1.2279903481558084e-05,
+      "loss": 0.2422557830810547,
+      "step": 5600
+    },
+    {
+      "epoch": 0.39986211651154774,
+      "grad_norm": 1.2678699493408203,
+      "learning_rate": 1.200413650465357e-05,
+      "loss": 0.24262113571166993,
+      "step": 5800
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "grad_norm": 1.5601742267608643,
+      "learning_rate": 1.1728369527749053e-05,
+      "loss": 0.2445354461669922,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "eval_accuracy": 0.9363128285626995,
+      "eval_f1": 0.4803237047343094,
+      "eval_loss": 0.19360145926475525,
+      "eval_precision": 0.4471989161695149,
+      "eval_recall": 0.5187482768127929,
+      "eval_runtime": 10.0804,
+      "eval_samples_per_second": 1984.049,
+      "eval_steps_per_second": 31.05,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4274388142019993,
+      "grad_norm": 1.5976165533065796,
+      "learning_rate": 1.1452602550844538e-05,
+      "loss": 0.2337088966369629,
+      "step": 6200
+    },
+    {
+      "epoch": 0.4412271630472251,
+      "grad_norm": 1.1755609512329102,
+      "learning_rate": 1.1176835573940022e-05,
+      "loss": 0.22395843505859375,
+      "step": 6400
+    },
+    {
+      "epoch": 0.4550155118924509,
+      "grad_norm": 1.0502196550369263,
+      "learning_rate": 1.0901068597035507e-05,
+      "loss": 0.22551630020141603,
+      "step": 6600
+    },
+    {
+      "epoch": 0.46880386073767666,
+      "grad_norm": 1.2917273044586182,
+      "learning_rate": 1.062530162013099e-05,
+      "loss": 0.23265909194946288,
+      "step": 6800
+    },
+    {
+      "epoch": 0.48259220958290244,
+      "grad_norm": 1.1052443981170654,
+      "learning_rate": 1.0349534643226476e-05,
+      "loss": 0.22997581481933593,
+      "step": 7000
+    },
+    {
+      "epoch": 0.4963805584281282,
+      "grad_norm": 1.381700873374939,
+      "learning_rate": 1.007376766632196e-05,
+      "loss": 0.2284539031982422,
+      "step": 7200
+    },
+    {
+      "epoch": 0.5101689072733541,
+      "grad_norm": 1.7055002450942993,
+      "learning_rate": 9.798000689417443e-06,
+      "loss": 0.22296590805053712,
+      "step": 7400
+    },
+    {
+      "epoch": 0.5239572561185798,
+      "grad_norm": 1.1947736740112305,
+      "learning_rate": 9.522233712512928e-06,
+      "loss": 0.2213929557800293,
+      "step": 7600
+    },
+    {
+      "epoch": 0.5377456049638056,
+      "grad_norm": 1.5072541236877441,
+      "learning_rate": 9.246466735608412e-06,
+      "loss": 0.22671804428100586,
+      "step": 7800
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "grad_norm": 1.1856580972671509,
+      "learning_rate": 8.970699758703895e-06,
+      "loss": 0.21764352798461914,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "eval_accuracy": 0.9401936873821388,
+      "eval_f1": 0.5002960941321868,
+      "eval_loss": 0.18017448484897614,
+      "eval_precision": 0.4692571483771252,
+      "eval_recall": 0.5357320099255584,
+      "eval_runtime": 10.0232,
+      "eval_samples_per_second": 1995.367,
+      "eval_steps_per_second": 31.227,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5653223026542572,
+      "grad_norm": 1.173108696937561,
+      "learning_rate": 8.69493278179938e-06,
+      "loss": 0.21765981674194335,
+      "step": 8200
+    },
+    {
+      "epoch": 0.5791106514994829,
+      "grad_norm": 1.3492612838745117,
+      "learning_rate": 8.419165804894864e-06,
+      "loss": 0.21885225296020508,
+      "step": 8400
+    },
+    {
+      "epoch": 0.5928990003447088,
+      "grad_norm": 1.2368924617767334,
+      "learning_rate": 8.143398827990349e-06,
+      "loss": 0.21894161224365236,
+      "step": 8600
+    },
+    {
+      "epoch": 0.6066873491899345,
+      "grad_norm": 1.0420321226119995,
+      "learning_rate": 7.867631851085833e-06,
+      "loss": 0.21339195251464843,
+      "step": 8800
+    },
+    {
+      "epoch": 0.6204756980351603,
+      "grad_norm": 1.312456727027893,
+      "learning_rate": 7.591864874181317e-06,
+      "loss": 0.21306278228759765,
+      "step": 9000
+    },
+    {
+      "epoch": 0.6342640468803861,
+      "grad_norm": 1.2957308292388916,
+      "learning_rate": 7.3160978972768015e-06,
+      "loss": 0.21283523559570314,
+      "step": 9200
+    },
+    {
+      "epoch": 0.6480523957256119,
+      "grad_norm": 3.096158981323242,
+      "learning_rate": 7.040330920372286e-06,
+      "loss": 0.22200897216796875,
+      "step": 9400
+    },
+    {
+      "epoch": 0.6618407445708376,
+      "grad_norm": 1.420737624168396,
+      "learning_rate": 6.76456394346777e-06,
+      "loss": 0.2175071907043457,
+      "step": 9600
+    },
+    {
+      "epoch": 0.6756290934160635,
+      "grad_norm": 1.5053138732910156,
+      "learning_rate": 6.488796966563255e-06,
+      "loss": 0.21140506744384766,
+      "step": 9800
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "grad_norm": 1.7005326747894287,
+      "learning_rate": 6.213029989658739e-06,
+      "loss": 0.20936330795288086,
+      "step": 10000
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "eval_accuracy": 0.9434556104550099,
+      "eval_f1": 0.5163540194718488,
+      "eval_loss": 0.17051653563976288,
+      "eval_precision": 0.4873125015293513,
+      "eval_recall": 0.5490763716570168,
+      "eval_runtime": 10.4802,
+      "eval_samples_per_second": 1908.354,
+      "eval_steps_per_second": 29.866,
+      "step": 10000
+    },
+    {
+      "epoch": 0.703205791106515,
+      "grad_norm": 1.1808063983917236,
+      "learning_rate": 5.937263012754223e-06,
+      "loss": 0.2099304962158203,
+      "step": 10200
+    },
+    {
+      "epoch": 0.7169941399517408,
+      "grad_norm": 1.238837718963623,
+      "learning_rate": 5.661496035849707e-06,
+      "loss": 0.20580806732177734,
+      "step": 10400
+    },
+    {
+      "epoch": 0.7307824887969666,
+      "grad_norm": 1.7494562864303589,
+      "learning_rate": 5.3857290589451915e-06,
+      "loss": 0.20552108764648438,
+      "step": 10600
+    },
+    {
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1073572635650635,
+      "learning_rate": 5.109962082040676e-06,
+      "loss": 0.20681476593017578,
+      "step": 10800
+    },
+    {
+      "epoch": 0.7583591864874182,
+      "grad_norm": 1.3458033800125122,
+      "learning_rate": 4.83419510513616e-06,
+      "loss": 0.20359535217285157,
+      "step": 11000
+    },
+    {
+      "epoch": 0.7721475353326439,
+      "grad_norm": 1.1731774806976318,
+      "learning_rate": 4.558428128231645e-06,
+      "loss": 0.2056349754333496,
+      "step": 11200
+    },
+    {
+      "epoch": 0.7859358841778697,
+      "grad_norm": 1.3708990812301636,
+      "learning_rate": 4.282661151327129e-06,
+      "loss": 0.2058660125732422,
+      "step": 11400
+    },
+    {
+      "epoch": 0.7997242330230955,
+      "grad_norm": 1.1624313592910767,
+      "learning_rate": 4.0068941744226135e-06,
+      "loss": 0.20310092926025392,
+      "step": 11600
+    },
+    {
+      "epoch": 0.8135125818683213,
+      "grad_norm": 1.3882473707199097,
+      "learning_rate": 3.7311271975180975e-06,
+      "loss": 0.2082367706298828,
+      "step": 11800
+    },
+    {
+      "epoch": 0.827300930713547,
+      "grad_norm": 1.7915494441986084,
+      "learning_rate": 3.455360220613582e-06,
+      "loss": 0.20367862701416015,
+      "step": 12000
+    },
+    {
+      "epoch": 0.827300930713547,
+      "eval_accuracy": 0.9442049177737276,
+      "eval_f1": 0.520962146523804,
+      "eval_loss": 0.16711591184139252,
+      "eval_precision": 0.4862475207302793,
+      "eval_recall": 0.5610146126275158,
+      "eval_runtime": 9.7345,
+      "eval_samples_per_second": 2054.558,
+      "eval_steps_per_second": 32.154,
+      "step": 12000
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 14505,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 2000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 77640709565952.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

full-openpii-500k/checkpoints/checkpoint-12000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec4df483fc65e86a8e85fec1834084cd6c3c69b50ccd36f2eeb0fabf3798ed8f
+size 5201

full-openpii-500k/checkpoints/checkpoint-14000/config.json

@@ -0,0 +1,86 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "id2label": {
+    "0": "O",
+    "1": "B-ADDRESS",
+    "2": "I-ADDRESS",
+    "3": "B-API_KEY",
+    "4": "I-API_KEY",
+    "5": "B-CREDIT_CARD",
+    "6": "I-CREDIT_CARD",
+    "7": "B-DATE_OF_BIRTH",
+    "8": "I-DATE_OF_BIRTH",
+    "9": "B-DRIVER_LICENSE",
+    "10": "I-DRIVER_LICENSE",
+    "11": "B-EMAIL",
+    "12": "I-EMAIL",
+    "13": "B-IP_ADDRESS",
+    "14": "I-IP_ADDRESS",
+    "15": "B-LOCATION",
+    "16": "I-LOCATION",
+    "17": "B-PASSWORD",
+    "18": "I-PASSWORD",
+    "19": "B-PERSON_NAME",
+    "20": "I-PERSON_NAME",
+    "21": "B-PHONE",
+    "22": "I-PHONE",
+    "23": "B-SSN",
+    "24": "I-SSN",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "is_decoder": false,
+  "label2id": {
+    "B-ADDRESS": 1,
+    "B-API_KEY": 3,
+    "B-CREDIT_CARD": 5,
+    "B-DATE_OF_BIRTH": 7,
+    "B-DRIVER_LICENSE": 9,
+    "B-EMAIL": 11,
+    "B-IP_ADDRESS": 13,
+    "B-LOCATION": 15,
+    "B-PASSWORD": 17,
+    "B-PERSON_NAME": 19,
+    "B-PHONE": 21,
+    "B-SSN": 23,
+    "B-USERNAME": 25,
+    "I-ADDRESS": 2,
+    "I-API_KEY": 4,
+    "I-CREDIT_CARD": 6,
+    "I-DATE_OF_BIRTH": 8,
+    "I-DRIVER_LICENSE": 10,
+    "I-EMAIL": 12,
+    "I-IP_ADDRESS": 14,
+    "I-LOCATION": 16,
+    "I-PASSWORD": 18,
+    "I-PERSON_NAME": 20,
+    "I-PHONE": 22,
+    "I-SSN": 24,
+    "I-USERNAME": 26,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

full-openpii-500k/checkpoints/checkpoint-14000/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50666c013ab44091fcdd9d2c93df85f0dc18ffcd34e787cf509c3ce8977c854b
+size 17495980

full-openpii-500k/checkpoints/checkpoint-14000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37ed5f219c0e6434a763cf994f6ac61c35b03b2de0490e39c08653896cd03f72
+size 35016907

full-openpii-500k/checkpoints/checkpoint-14000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64886e7552c59cf7a302bde3f7657f9fc9a73e7f37d504e244252d13bd8ef3aa
+size 14645

full-openpii-500k/checkpoints/checkpoint-14000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a2f73f26499c130880fa0abb201fb2049daf225e8fcfae2e1e94a0b685604f9
+size 1383

full-openpii-500k/checkpoints/checkpoint-14000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8212933720dae023f46fb90420cc13f4175484bff4ae39b89e6959909a8a5e3
+size 1465

full-openpii-500k/checkpoints/checkpoint-14000/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 160,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

full-openpii-500k/checkpoints/checkpoint-14000/trainer_state.json

@@ -0,0 +1,608 @@
+{
+  "best_global_step": 14000,
+  "best_metric": 0.5233849845820377,
+  "best_model_checkpoint": "/outputs/full-openpii-500k/checkpoints/checkpoint-14000",
+  "epoch": 0.9651844191658049,
+  "eval_steps": 2000,
+  "global_step": 14000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013788348845225784,
+      "grad_norm": 1.3536444902420044,
+      "learning_rate": 1.972561185798001e-05,
+      "loss": 0.7454490661621094,
+      "step": 200
+    },
+    {
+      "epoch": 0.027576697690451568,
+      "grad_norm": 1.3048033714294434,
+      "learning_rate": 1.944984488107549e-05,
+      "loss": 0.5720141220092774,
+      "step": 400
+    },
+    {
+      "epoch": 0.04136504653567735,
+      "grad_norm": 1.9288572072982788,
+      "learning_rate": 1.9174077904170978e-05,
+      "loss": 0.4927207946777344,
+      "step": 600
+    },
+    {
+      "epoch": 0.055153395380903136,
+      "grad_norm": 1.7550076246261597,
+      "learning_rate": 1.889831092726646e-05,
+      "loss": 0.4657698440551758,
+      "step": 800
+    },
+    {
+      "epoch": 0.06894174422612892,
+      "grad_norm": 1.7711143493652344,
+      "learning_rate": 1.8622543950361945e-05,
+      "loss": 0.42969669342041017,
+      "step": 1000
+    },
+    {
+      "epoch": 0.0827300930713547,
+      "grad_norm": 2.50980544090271,
+      "learning_rate": 1.834677697345743e-05,
+      "loss": 0.41879753112792967,
+      "step": 1200
+    },
+    {
+      "epoch": 0.09651844191658049,
+      "grad_norm": 1.705670952796936,
+      "learning_rate": 1.8071009996552916e-05,
+      "loss": 0.385702018737793,
+      "step": 1400
+    },
+    {
+      "epoch": 0.11030679076180627,
+      "grad_norm": 1.445986032485962,
+      "learning_rate": 1.77952430196484e-05,
+      "loss": 0.36213504791259765,
+      "step": 1600
+    },
+    {
+      "epoch": 0.12409513960703206,
+      "grad_norm": 1.4295554161071777,
+      "learning_rate": 1.7519476042743883e-05,
+      "loss": 0.351446533203125,
+      "step": 1800
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "grad_norm": 1.4451907873153687,
+      "learning_rate": 1.7243709065839366e-05,
+      "loss": 0.34252952575683593,
+      "step": 2000
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "eval_accuracy": 0.9116923546949569,
+      "eval_f1": 0.3809941841325579,
+      "eval_loss": 0.2862004041671753,
+      "eval_precision": 0.37910138123055087,
+      "eval_recall": 0.38290598290598293,
+      "eval_runtime": 11.153,
+      "eval_samples_per_second": 1793.238,
+      "eval_steps_per_second": 28.064,
+      "step": 2000
+    },
+    {
+      "epoch": 0.15167183729748362,
+      "grad_norm": 1.3401026725769043,
+      "learning_rate": 1.6967942088934853e-05,
+      "loss": 0.33029396057128907,
+      "step": 2200
+    },
+    {
+      "epoch": 0.1654601861427094,
+      "grad_norm": 1.4941850900650024,
+      "learning_rate": 1.6692175112030337e-05,
+      "loss": 0.31315877914428714,
+      "step": 2400
+    },
+    {
+      "epoch": 0.1792485349879352,
+      "grad_norm": 1.2174286842346191,
+      "learning_rate": 1.641640813512582e-05,
+      "loss": 0.31084712982177737,
+      "step": 2600
+    },
+    {
+      "epoch": 0.19303688383316098,
+      "grad_norm": 1.6435197591781616,
+      "learning_rate": 1.6140641158221304e-05,
+      "loss": 0.30205060958862306,
+      "step": 2800
+    },
+    {
+      "epoch": 0.20682523267838676,
+      "grad_norm": 2.0023410320281982,
+      "learning_rate": 1.586487418131679e-05,
+      "loss": 0.2959835815429688,
+      "step": 3000
+    },
+    {
+      "epoch": 0.22061358152361255,
+      "grad_norm": 1.1106364727020264,
+      "learning_rate": 1.558910720441227e-05,
+      "loss": 0.28498205184936526,
+      "step": 3200
+    },
+    {
+      "epoch": 0.23440193036883833,
+      "grad_norm": 1.951019525527954,
+      "learning_rate": 1.5313340227507758e-05,
+      "loss": 0.2866662406921387,
+      "step": 3400
+    },
+    {
+      "epoch": 0.2481902792140641,
+      "grad_norm": 1.2675732374191284,
+      "learning_rate": 1.5037573250603242e-05,
+      "loss": 0.2802707290649414,
+      "step": 3600
+    },
+    {
+      "epoch": 0.2619786280592899,
+      "grad_norm": 1.129682183265686,
+      "learning_rate": 1.4761806273698727e-05,
+      "loss": 0.27155515670776365,
+      "step": 3800
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "grad_norm": 1.1867386102676392,
+      "learning_rate": 1.448603929679421e-05,
+      "loss": 0.2704596519470215,
+      "step": 4000
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "eval_accuracy": 0.9294057690079118,
+      "eval_f1": 0.4474882331063317,
+      "eval_loss": 0.2208142876625061,
+      "eval_precision": 0.4183392082104405,
+      "eval_recall": 0.48100358422939066,
+      "eval_runtime": 10.0377,
+      "eval_samples_per_second": 1992.497,
+      "eval_steps_per_second": 31.183,
+      "step": 4000
+    },
+    {
+      "epoch": 0.28955532574974147,
+      "grad_norm": 1.4080299139022827,
+      "learning_rate": 1.4210272319889696e-05,
+      "loss": 0.2664038848876953,
+      "step": 4200
+    },
+    {
+      "epoch": 0.30334367459496725,
+      "grad_norm": 1.4007683992385864,
+      "learning_rate": 1.393450534298518e-05,
+      "loss": 0.26422006607055665,
+      "step": 4400
+    },
+    {
+      "epoch": 0.31713202344019303,
+      "grad_norm": 1.7308765649795532,
+      "learning_rate": 1.3658738366080664e-05,
+      "loss": 0.25533161163330076,
+      "step": 4600
+    },
+    {
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.7256940603256226,
+      "learning_rate": 1.3382971389176146e-05,
+      "loss": 0.25111486434936525,
+      "step": 4800
+    },
+    {
+      "epoch": 0.3447087211306446,
+      "grad_norm": 1.039595365524292,
+      "learning_rate": 1.3107204412271633e-05,
+      "loss": 0.24599828720092773,
+      "step": 5000
+    },
+    {
+      "epoch": 0.3584970699758704,
+      "grad_norm": 1.8612524271011353,
+      "learning_rate": 1.2831437435367115e-05,
+      "loss": 0.2489796257019043,
+      "step": 5200
+    },
+    {
+      "epoch": 0.37228541882109617,
+      "grad_norm": 2.6072161197662354,
+      "learning_rate": 1.25556704584626e-05,
+      "loss": 0.2444438934326172,
+      "step": 5400
+    },
+    {
+      "epoch": 0.38607376766632195,
+      "grad_norm": 1.265944242477417,
+      "learning_rate": 1.2279903481558084e-05,
+      "loss": 0.2422557830810547,
+      "step": 5600
+    },
+    {
+      "epoch": 0.39986211651154774,
+      "grad_norm": 1.2678699493408203,
+      "learning_rate": 1.200413650465357e-05,
+      "loss": 0.24262113571166993,
+      "step": 5800
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "grad_norm": 1.5601742267608643,
+      "learning_rate": 1.1728369527749053e-05,
+      "loss": 0.2445354461669922,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "eval_accuracy": 0.9363128285626995,
+      "eval_f1": 0.4803237047343094,
+      "eval_loss": 0.19360145926475525,
+      "eval_precision": 0.4471989161695149,
+      "eval_recall": 0.5187482768127929,
+      "eval_runtime": 10.0804,
+      "eval_samples_per_second": 1984.049,
+      "eval_steps_per_second": 31.05,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4274388142019993,
+      "grad_norm": 1.5976165533065796,
+      "learning_rate": 1.1452602550844538e-05,
+      "loss": 0.2337088966369629,
+      "step": 6200
+    },
+    {
+      "epoch": 0.4412271630472251,
+      "grad_norm": 1.1755609512329102,
+      "learning_rate": 1.1176835573940022e-05,
+      "loss": 0.22395843505859375,
+      "step": 6400
+    },
+    {
+      "epoch": 0.4550155118924509,
+      "grad_norm": 1.0502196550369263,
+      "learning_rate": 1.0901068597035507e-05,
+      "loss": 0.22551630020141603,
+      "step": 6600
+    },
+    {
+      "epoch": 0.46880386073767666,
+      "grad_norm": 1.2917273044586182,
+      "learning_rate": 1.062530162013099e-05,
+      "loss": 0.23265909194946288,
+      "step": 6800
+    },
+    {
+      "epoch": 0.48259220958290244,
+      "grad_norm": 1.1052443981170654,
+      "learning_rate": 1.0349534643226476e-05,
+      "loss": 0.22997581481933593,
+      "step": 7000
+    },
+    {
+      "epoch": 0.4963805584281282,
+      "grad_norm": 1.381700873374939,
+      "learning_rate": 1.007376766632196e-05,
+      "loss": 0.2284539031982422,
+      "step": 7200
+    },
+    {
+      "epoch": 0.5101689072733541,
+      "grad_norm": 1.7055002450942993,
+      "learning_rate": 9.798000689417443e-06,
+      "loss": 0.22296590805053712,
+      "step": 7400
+    },
+    {
+      "epoch": 0.5239572561185798,
+      "grad_norm": 1.1947736740112305,
+      "learning_rate": 9.522233712512928e-06,
+      "loss": 0.2213929557800293,
+      "step": 7600
+    },
+    {
+      "epoch": 0.5377456049638056,
+      "grad_norm": 1.5072541236877441,
+      "learning_rate": 9.246466735608412e-06,
+      "loss": 0.22671804428100586,
+      "step": 7800
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "grad_norm": 1.1856580972671509,
+      "learning_rate": 8.970699758703895e-06,
+      "loss": 0.21764352798461914,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "eval_accuracy": 0.9401936873821388,
+      "eval_f1": 0.5002960941321868,
+      "eval_loss": 0.18017448484897614,
+      "eval_precision": 0.4692571483771252,
+      "eval_recall": 0.5357320099255584,
+      "eval_runtime": 10.0232,
+      "eval_samples_per_second": 1995.367,
+      "eval_steps_per_second": 31.227,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5653223026542572,
+      "grad_norm": 1.173108696937561,
+      "learning_rate": 8.69493278179938e-06,
+      "loss": 0.21765981674194335,
+      "step": 8200
+    },
+    {
+      "epoch": 0.5791106514994829,
+      "grad_norm": 1.3492612838745117,
+      "learning_rate": 8.419165804894864e-06,
+      "loss": 0.21885225296020508,
+      "step": 8400
+    },
+    {
+      "epoch": 0.5928990003447088,
+      "grad_norm": 1.2368924617767334,
+      "learning_rate": 8.143398827990349e-06,
+      "loss": 0.21894161224365236,
+      "step": 8600
+    },
+    {
+      "epoch": 0.6066873491899345,
+      "grad_norm": 1.0420321226119995,
+      "learning_rate": 7.867631851085833e-06,
+      "loss": 0.21339195251464843,
+      "step": 8800
+    },
+    {
+      "epoch": 0.6204756980351603,
+      "grad_norm": 1.312456727027893,
+      "learning_rate": 7.591864874181317e-06,
+      "loss": 0.21306278228759765,
+      "step": 9000
+    },
+    {
+      "epoch": 0.6342640468803861,
+      "grad_norm": 1.2957308292388916,
+      "learning_rate": 7.3160978972768015e-06,
+      "loss": 0.21283523559570314,
+      "step": 9200
+    },
+    {
+      "epoch": 0.6480523957256119,
+      "grad_norm": 3.096158981323242,
+      "learning_rate": 7.040330920372286e-06,
+      "loss": 0.22200897216796875,
+      "step": 9400
+    },
+    {
+      "epoch": 0.6618407445708376,
+      "grad_norm": 1.420737624168396,
+      "learning_rate": 6.76456394346777e-06,
+      "loss": 0.2175071907043457,
+      "step": 9600
+    },
+    {
+      "epoch": 0.6756290934160635,
+      "grad_norm": 1.5053138732910156,
+      "learning_rate": 6.488796966563255e-06,
+      "loss": 0.21140506744384766,
+      "step": 9800
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "grad_norm": 1.7005326747894287,
+      "learning_rate": 6.213029989658739e-06,
+      "loss": 0.20936330795288086,
+      "step": 10000
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "eval_accuracy": 0.9434556104550099,
+      "eval_f1": 0.5163540194718488,
+      "eval_loss": 0.17051653563976288,
+      "eval_precision": 0.4873125015293513,
+      "eval_recall": 0.5490763716570168,
+      "eval_runtime": 10.4802,
+      "eval_samples_per_second": 1908.354,
+      "eval_steps_per_second": 29.866,
+      "step": 10000
+    },
+    {
+      "epoch": 0.703205791106515,
+      "grad_norm": 1.1808063983917236,
+      "learning_rate": 5.937263012754223e-06,
+      "loss": 0.2099304962158203,
+      "step": 10200
+    },
+    {
+      "epoch": 0.7169941399517408,
+      "grad_norm": 1.238837718963623,
+      "learning_rate": 5.661496035849707e-06,
+      "loss": 0.20580806732177734,
+      "step": 10400
+    },
+    {
+      "epoch": 0.7307824887969666,
+      "grad_norm": 1.7494562864303589,
+      "learning_rate": 5.3857290589451915e-06,
+      "loss": 0.20552108764648438,
+      "step": 10600
+    },
+    {
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1073572635650635,
+      "learning_rate": 5.109962082040676e-06,
+      "loss": 0.20681476593017578,
+      "step": 10800
+    },
+    {
+      "epoch": 0.7583591864874182,
+      "grad_norm": 1.3458033800125122,
+      "learning_rate": 4.83419510513616e-06,
+      "loss": 0.20359535217285157,
+      "step": 11000
+    },
+    {
+      "epoch": 0.7721475353326439,
+      "grad_norm": 1.1731774806976318,
+      "learning_rate": 4.558428128231645e-06,
+      "loss": 0.2056349754333496,
+      "step": 11200
+    },
+    {
+      "epoch": 0.7859358841778697,
+      "grad_norm": 1.3708990812301636,
+      "learning_rate": 4.282661151327129e-06,
+      "loss": 0.2058660125732422,
+      "step": 11400
+    },
+    {
+      "epoch": 0.7997242330230955,
+      "grad_norm": 1.1624313592910767,
+      "learning_rate": 4.0068941744226135e-06,
+      "loss": 0.20310092926025392,
+      "step": 11600
+    },
+    {
+      "epoch": 0.8135125818683213,
+      "grad_norm": 1.3882473707199097,
+      "learning_rate": 3.7311271975180975e-06,
+      "loss": 0.2082367706298828,
+      "step": 11800
+    },
+    {
+      "epoch": 0.827300930713547,
+      "grad_norm": 1.7915494441986084,
+      "learning_rate": 3.455360220613582e-06,
+      "loss": 0.20367862701416015,
+      "step": 12000
+    },
+    {
+      "epoch": 0.827300930713547,
+      "eval_accuracy": 0.9442049177737276,
+      "eval_f1": 0.520962146523804,
+      "eval_loss": 0.16711591184139252,
+      "eval_precision": 0.4862475207302793,
+      "eval_recall": 0.5610146126275158,
+      "eval_runtime": 9.7345,
+      "eval_samples_per_second": 2054.558,
+      "eval_steps_per_second": 32.154,
+      "step": 12000
+    },
+    {
+      "epoch": 0.8410892795587729,
+      "grad_norm": 1.900614857673645,
+      "learning_rate": 3.179593243709066e-06,
+      "loss": 0.20414718627929687,
+      "step": 12200
+    },
+    {
+      "epoch": 0.8548776284039986,
+      "grad_norm": 1.5320205688476562,
+      "learning_rate": 2.9038262668045503e-06,
+      "loss": 0.19842819213867188,
+      "step": 12400
+    },
+    {
+      "epoch": 0.8686659772492245,
+      "grad_norm": 1.0634764432907104,
+      "learning_rate": 2.6280592899000347e-06,
+      "loss": 0.20455713272094728,
+      "step": 12600
+    },
+    {
+      "epoch": 0.8824543260944502,
+      "grad_norm": 1.5051591396331787,
+      "learning_rate": 2.352292312995519e-06,
+      "loss": 0.20511449813842775,
+      "step": 12800
+    },
+    {
+      "epoch": 0.896242674939676,
+      "grad_norm": 1.2883975505828857,
+      "learning_rate": 2.076525336091003e-06,
+      "loss": 0.1981792640686035,
+      "step": 13000
+    },
+    {
+      "epoch": 0.9100310237849017,
+      "grad_norm": 1.901268720626831,
+      "learning_rate": 1.8007583591864874e-06,
+      "loss": 0.20567977905273438,
+      "step": 13200
+    },
+    {
+      "epoch": 0.9238193726301276,
+      "grad_norm": 1.828506350517273,
+      "learning_rate": 1.5249913822819716e-06,
+      "loss": 0.20068155288696288,
+      "step": 13400
+    },
+    {
+      "epoch": 0.9376077214753533,
+      "grad_norm": 1.401857614517212,
+      "learning_rate": 1.2492244053774563e-06,
+      "loss": 0.20199392318725587,
+      "step": 13600
+    },
+    {
+      "epoch": 0.9513960703205792,
+      "grad_norm": 1.8156548738479614,
+      "learning_rate": 9.734574284729404e-07,
+      "loss": 0.20472015380859376,
+      "step": 13800
+    },
+    {
+      "epoch": 0.9651844191658049,
+      "grad_norm": 1.086504578590393,
+      "learning_rate": 6.976904515684247e-07,
+      "loss": 0.20377262115478514,
+      "step": 14000
+    },
+    {
+      "epoch": 0.9651844191658049,
+      "eval_accuracy": 0.9452518410046249,
+      "eval_f1": 0.5233849845820377,
+      "eval_loss": 0.16398483514785767,
+      "eval_precision": 0.4918641026262822,
+      "eval_recall": 0.5592224979321754,
+      "eval_runtime": 10.6768,
+      "eval_samples_per_second": 1873.214,
+      "eval_steps_per_second": 29.316,
+      "step": 14000
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 14505,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 2000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 90601040517312.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

full-openpii-500k/checkpoints/checkpoint-14000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec4df483fc65e86a8e85fec1834084cd6c3c69b50ccd36f2eeb0fabf3798ed8f
+size 5201

full-openpii-500k/checkpoints/checkpoint-14505/config.json

@@ -0,0 +1,86 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "id2label": {
+    "0": "O",
+    "1": "B-ADDRESS",
+    "2": "I-ADDRESS",
+    "3": "B-API_KEY",
+    "4": "I-API_KEY",
+    "5": "B-CREDIT_CARD",
+    "6": "I-CREDIT_CARD",
+    "7": "B-DATE_OF_BIRTH",
+    "8": "I-DATE_OF_BIRTH",
+    "9": "B-DRIVER_LICENSE",
+    "10": "I-DRIVER_LICENSE",
+    "11": "B-EMAIL",
+    "12": "I-EMAIL",
+    "13": "B-IP_ADDRESS",
+    "14": "I-IP_ADDRESS",
+    "15": "B-LOCATION",
+    "16": "I-LOCATION",
+    "17": "B-PASSWORD",
+    "18": "I-PASSWORD",
+    "19": "B-PERSON_NAME",
+    "20": "I-PERSON_NAME",
+    "21": "B-PHONE",
+    "22": "I-PHONE",
+    "23": "B-SSN",
+    "24": "I-SSN",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "is_decoder": false,
+  "label2id": {
+    "B-ADDRESS": 1,
+    "B-API_KEY": 3,
+    "B-CREDIT_CARD": 5,
+    "B-DATE_OF_BIRTH": 7,
+    "B-DRIVER_LICENSE": 9,
+    "B-EMAIL": 11,
+    "B-IP_ADDRESS": 13,
+    "B-LOCATION": 15,
+    "B-PASSWORD": 17,
+    "B-PERSON_NAME": 19,
+    "B-PHONE": 21,
+    "B-SSN": 23,
+    "B-USERNAME": 25,
+    "I-ADDRESS": 2,
+    "I-API_KEY": 4,
+    "I-CREDIT_CARD": 6,
+    "I-DATE_OF_BIRTH": 8,
+    "I-DRIVER_LICENSE": 10,
+    "I-EMAIL": 12,
+    "I-IP_ADDRESS": 14,
+    "I-LOCATION": 16,
+    "I-PASSWORD": 18,
+    "I-PERSON_NAME": 20,
+    "I-PHONE": 22,
+    "I-SSN": 24,
+    "I-USERNAME": 26,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

full-openpii-500k/checkpoints/checkpoint-14505/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b976a7b1be395ceb87e3d25eb691cc692395118bec627e6437d605d9486215f
+size 17495980

full-openpii-500k/checkpoints/checkpoint-14505/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9846988659b22d7a1047f51e320b7806dbd6c0b710f9b39c669362365c00eacb
+size 35016907

full-openpii-500k/checkpoints/checkpoint-14505/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4314895b7c09719e927ae3a1ac2c9ffb527f589a35c91e3c13cec087327fd75b
+size 14645

full-openpii-500k/checkpoints/checkpoint-14505/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34c753ff65cefab0b3e8b4692a2b1ce7d836c2e3e2e8e5dba14f1b0e440b60db
+size 1383

full-openpii-500k/checkpoints/checkpoint-14505/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20af6ef61c48447733cd6c1e7b36644273b7c42271ba15bb981c9d63bea70ab8
+size 1465

full-openpii-500k/checkpoints/checkpoint-14505/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 160,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

full-openpii-500k/checkpoints/checkpoint-14505/trainer_state.json

@@ -0,0 +1,634 @@
+{
+  "best_global_step": 14505,
+  "best_metric": 0.5242530574333041,
+  "best_model_checkpoint": "/outputs/full-openpii-500k/checkpoints/checkpoint-14505",
+  "epoch": 1.0,
+  "eval_steps": 2000,
+  "global_step": 14505,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013788348845225784,
+      "grad_norm": 1.3536444902420044,
+      "learning_rate": 1.972561185798001e-05,
+      "loss": 0.7454490661621094,
+      "step": 200
+    },
+    {
+      "epoch": 0.027576697690451568,
+      "grad_norm": 1.3048033714294434,
+      "learning_rate": 1.944984488107549e-05,
+      "loss": 0.5720141220092774,
+      "step": 400
+    },
+    {
+      "epoch": 0.04136504653567735,
+      "grad_norm": 1.9288572072982788,
+      "learning_rate": 1.9174077904170978e-05,
+      "loss": 0.4927207946777344,
+      "step": 600
+    },
+    {
+      "epoch": 0.055153395380903136,
+      "grad_norm": 1.7550076246261597,
+      "learning_rate": 1.889831092726646e-05,
+      "loss": 0.4657698440551758,
+      "step": 800
+    },
+    {
+      "epoch": 0.06894174422612892,
+      "grad_norm": 1.7711143493652344,
+      "learning_rate": 1.8622543950361945e-05,
+      "loss": 0.42969669342041017,
+      "step": 1000
+    },
+    {
+      "epoch": 0.0827300930713547,
+      "grad_norm": 2.50980544090271,
+      "learning_rate": 1.834677697345743e-05,
+      "loss": 0.41879753112792967,
+      "step": 1200
+    },
+    {
+      "epoch": 0.09651844191658049,
+      "grad_norm": 1.705670952796936,
+      "learning_rate": 1.8071009996552916e-05,
+      "loss": 0.385702018737793,
+      "step": 1400
+    },
+    {
+      "epoch": 0.11030679076180627,
+      "grad_norm": 1.445986032485962,
+      "learning_rate": 1.77952430196484e-05,
+      "loss": 0.36213504791259765,
+      "step": 1600
+    },
+    {
+      "epoch": 0.12409513960703206,
+      "grad_norm": 1.4295554161071777,
+      "learning_rate": 1.7519476042743883e-05,
+      "loss": 0.351446533203125,
+      "step": 1800
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "grad_norm": 1.4451907873153687,
+      "learning_rate": 1.7243709065839366e-05,
+      "loss": 0.34252952575683593,
+      "step": 2000
+    },
+    {
+      "epoch": 0.13788348845225784,
+      "eval_accuracy": 0.9116923546949569,
+      "eval_f1": 0.3809941841325579,
+      "eval_loss": 0.2862004041671753,
+      "eval_precision": 0.37910138123055087,
+      "eval_recall": 0.38290598290598293,
+      "eval_runtime": 11.153,
+      "eval_samples_per_second": 1793.238,
+      "eval_steps_per_second": 28.064,
+      "step": 2000
+    },
+    {
+      "epoch": 0.15167183729748362,
+      "grad_norm": 1.3401026725769043,
+      "learning_rate": 1.6967942088934853e-05,
+      "loss": 0.33029396057128907,
+      "step": 2200
+    },
+    {
+      "epoch": 0.1654601861427094,
+      "grad_norm": 1.4941850900650024,
+      "learning_rate": 1.6692175112030337e-05,
+      "loss": 0.31315877914428714,
+      "step": 2400
+    },
+    {
+      "epoch": 0.1792485349879352,
+      "grad_norm": 1.2174286842346191,
+      "learning_rate": 1.641640813512582e-05,
+      "loss": 0.31084712982177737,
+      "step": 2600
+    },
+    {
+      "epoch": 0.19303688383316098,
+      "grad_norm": 1.6435197591781616,
+      "learning_rate": 1.6140641158221304e-05,
+      "loss": 0.30205060958862306,
+      "step": 2800
+    },
+    {
+      "epoch": 0.20682523267838676,
+      "grad_norm": 2.0023410320281982,
+      "learning_rate": 1.586487418131679e-05,
+      "loss": 0.2959835815429688,
+      "step": 3000
+    },
+    {
+      "epoch": 0.22061358152361255,
+      "grad_norm": 1.1106364727020264,
+      "learning_rate": 1.558910720441227e-05,
+      "loss": 0.28498205184936526,
+      "step": 3200
+    },
+    {
+      "epoch": 0.23440193036883833,
+      "grad_norm": 1.951019525527954,
+      "learning_rate": 1.5313340227507758e-05,
+      "loss": 0.2866662406921387,
+      "step": 3400
+    },
+    {
+      "epoch": 0.2481902792140641,
+      "grad_norm": 1.2675732374191284,
+      "learning_rate": 1.5037573250603242e-05,
+      "loss": 0.2802707290649414,
+      "step": 3600
+    },
+    {
+      "epoch": 0.2619786280592899,
+      "grad_norm": 1.129682183265686,
+      "learning_rate": 1.4761806273698727e-05,
+      "loss": 0.27155515670776365,
+      "step": 3800
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "grad_norm": 1.1867386102676392,
+      "learning_rate": 1.448603929679421e-05,
+      "loss": 0.2704596519470215,
+      "step": 4000
+    },
+    {
+      "epoch": 0.2757669769045157,
+      "eval_accuracy": 0.9294057690079118,
+      "eval_f1": 0.4474882331063317,
+      "eval_loss": 0.2208142876625061,
+      "eval_precision": 0.4183392082104405,
+      "eval_recall": 0.48100358422939066,
+      "eval_runtime": 10.0377,
+      "eval_samples_per_second": 1992.497,
+      "eval_steps_per_second": 31.183,
+      "step": 4000
+    },
+    {
+      "epoch": 0.28955532574974147,
+      "grad_norm": 1.4080299139022827,
+      "learning_rate": 1.4210272319889696e-05,
+      "loss": 0.2664038848876953,
+      "step": 4200
+    },
+    {
+      "epoch": 0.30334367459496725,
+      "grad_norm": 1.4007683992385864,
+      "learning_rate": 1.393450534298518e-05,
+      "loss": 0.26422006607055665,
+      "step": 4400
+    },
+    {
+      "epoch": 0.31713202344019303,
+      "grad_norm": 1.7308765649795532,
+      "learning_rate": 1.3658738366080664e-05,
+      "loss": 0.25533161163330076,
+      "step": 4600
+    },
+    {
+      "epoch": 0.3309203722854188,
+      "grad_norm": 1.7256940603256226,
+      "learning_rate": 1.3382971389176146e-05,
+      "loss": 0.25111486434936525,
+      "step": 4800
+    },
+    {
+      "epoch": 0.3447087211306446,
+      "grad_norm": 1.039595365524292,
+      "learning_rate": 1.3107204412271633e-05,
+      "loss": 0.24599828720092773,
+      "step": 5000
+    },
+    {
+      "epoch": 0.3584970699758704,
+      "grad_norm": 1.8612524271011353,
+      "learning_rate": 1.2831437435367115e-05,
+      "loss": 0.2489796257019043,
+      "step": 5200
+    },
+    {
+      "epoch": 0.37228541882109617,
+      "grad_norm": 2.6072161197662354,
+      "learning_rate": 1.25556704584626e-05,
+      "loss": 0.2444438934326172,
+      "step": 5400
+    },
+    {
+      "epoch": 0.38607376766632195,
+      "grad_norm": 1.265944242477417,
+      "learning_rate": 1.2279903481558084e-05,
+      "loss": 0.2422557830810547,
+      "step": 5600
+    },
+    {
+      "epoch": 0.39986211651154774,
+      "grad_norm": 1.2678699493408203,
+      "learning_rate": 1.200413650465357e-05,
+      "loss": 0.24262113571166993,
+      "step": 5800
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "grad_norm": 1.5601742267608643,
+      "learning_rate": 1.1728369527749053e-05,
+      "loss": 0.2445354461669922,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4136504653567735,
+      "eval_accuracy": 0.9363128285626995,
+      "eval_f1": 0.4803237047343094,
+      "eval_loss": 0.19360145926475525,
+      "eval_precision": 0.4471989161695149,
+      "eval_recall": 0.5187482768127929,
+      "eval_runtime": 10.0804,
+      "eval_samples_per_second": 1984.049,
+      "eval_steps_per_second": 31.05,
+      "step": 6000
+    },
+    {
+      "epoch": 0.4274388142019993,
+      "grad_norm": 1.5976165533065796,
+      "learning_rate": 1.1452602550844538e-05,
+      "loss": 0.2337088966369629,
+      "step": 6200
+    },
+    {
+      "epoch": 0.4412271630472251,
+      "grad_norm": 1.1755609512329102,
+      "learning_rate": 1.1176835573940022e-05,
+      "loss": 0.22395843505859375,
+      "step": 6400
+    },
+    {
+      "epoch": 0.4550155118924509,
+      "grad_norm": 1.0502196550369263,
+      "learning_rate": 1.0901068597035507e-05,
+      "loss": 0.22551630020141603,
+      "step": 6600
+    },
+    {
+      "epoch": 0.46880386073767666,
+      "grad_norm": 1.2917273044586182,
+      "learning_rate": 1.062530162013099e-05,
+      "loss": 0.23265909194946288,
+      "step": 6800
+    },
+    {
+      "epoch": 0.48259220958290244,
+      "grad_norm": 1.1052443981170654,
+      "learning_rate": 1.0349534643226476e-05,
+      "loss": 0.22997581481933593,
+      "step": 7000
+    },
+    {
+      "epoch": 0.4963805584281282,
+      "grad_norm": 1.381700873374939,
+      "learning_rate": 1.007376766632196e-05,
+      "loss": 0.2284539031982422,
+      "step": 7200
+    },
+    {
+      "epoch": 0.5101689072733541,
+      "grad_norm": 1.7055002450942993,
+      "learning_rate": 9.798000689417443e-06,
+      "loss": 0.22296590805053712,
+      "step": 7400
+    },
+    {
+      "epoch": 0.5239572561185798,
+      "grad_norm": 1.1947736740112305,
+      "learning_rate": 9.522233712512928e-06,
+      "loss": 0.2213929557800293,
+      "step": 7600
+    },
+    {
+      "epoch": 0.5377456049638056,
+      "grad_norm": 1.5072541236877441,
+      "learning_rate": 9.246466735608412e-06,
+      "loss": 0.22671804428100586,
+      "step": 7800
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "grad_norm": 1.1856580972671509,
+      "learning_rate": 8.970699758703895e-06,
+      "loss": 0.21764352798461914,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5515339538090314,
+      "eval_accuracy": 0.9401936873821388,
+      "eval_f1": 0.5002960941321868,
+      "eval_loss": 0.18017448484897614,
+      "eval_precision": 0.4692571483771252,
+      "eval_recall": 0.5357320099255584,
+      "eval_runtime": 10.0232,
+      "eval_samples_per_second": 1995.367,
+      "eval_steps_per_second": 31.227,
+      "step": 8000
+    },
+    {
+      "epoch": 0.5653223026542572,
+      "grad_norm": 1.173108696937561,
+      "learning_rate": 8.69493278179938e-06,
+      "loss": 0.21765981674194335,
+      "step": 8200
+    },
+    {
+      "epoch": 0.5791106514994829,
+      "grad_norm": 1.3492612838745117,
+      "learning_rate": 8.419165804894864e-06,
+      "loss": 0.21885225296020508,
+      "step": 8400
+    },
+    {
+      "epoch": 0.5928990003447088,
+      "grad_norm": 1.2368924617767334,
+      "learning_rate": 8.143398827990349e-06,
+      "loss": 0.21894161224365236,
+      "step": 8600
+    },
+    {
+      "epoch": 0.6066873491899345,
+      "grad_norm": 1.0420321226119995,
+      "learning_rate": 7.867631851085833e-06,
+      "loss": 0.21339195251464843,
+      "step": 8800
+    },
+    {
+      "epoch": 0.6204756980351603,
+      "grad_norm": 1.312456727027893,
+      "learning_rate": 7.591864874181317e-06,
+      "loss": 0.21306278228759765,
+      "step": 9000
+    },
+    {
+      "epoch": 0.6342640468803861,
+      "grad_norm": 1.2957308292388916,
+      "learning_rate": 7.3160978972768015e-06,
+      "loss": 0.21283523559570314,
+      "step": 9200
+    },
+    {
+      "epoch": 0.6480523957256119,
+      "grad_norm": 3.096158981323242,
+      "learning_rate": 7.040330920372286e-06,
+      "loss": 0.22200897216796875,
+      "step": 9400
+    },
+    {
+      "epoch": 0.6618407445708376,
+      "grad_norm": 1.420737624168396,
+      "learning_rate": 6.76456394346777e-06,
+      "loss": 0.2175071907043457,
+      "step": 9600
+    },
+    {
+      "epoch": 0.6756290934160635,
+      "grad_norm": 1.5053138732910156,
+      "learning_rate": 6.488796966563255e-06,
+      "loss": 0.21140506744384766,
+      "step": 9800
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "grad_norm": 1.7005326747894287,
+      "learning_rate": 6.213029989658739e-06,
+      "loss": 0.20936330795288086,
+      "step": 10000
+    },
+    {
+      "epoch": 0.6894174422612892,
+      "eval_accuracy": 0.9434556104550099,
+      "eval_f1": 0.5163540194718488,
+      "eval_loss": 0.17051653563976288,
+      "eval_precision": 0.4873125015293513,
+      "eval_recall": 0.5490763716570168,
+      "eval_runtime": 10.4802,
+      "eval_samples_per_second": 1908.354,
+      "eval_steps_per_second": 29.866,
+      "step": 10000
+    },
+    {
+      "epoch": 0.703205791106515,
+      "grad_norm": 1.1808063983917236,
+      "learning_rate": 5.937263012754223e-06,
+      "loss": 0.2099304962158203,
+      "step": 10200
+    },
+    {
+      "epoch": 0.7169941399517408,
+      "grad_norm": 1.238837718963623,
+      "learning_rate": 5.661496035849707e-06,
+      "loss": 0.20580806732177734,
+      "step": 10400
+    },
+    {
+      "epoch": 0.7307824887969666,
+      "grad_norm": 1.7494562864303589,
+      "learning_rate": 5.3857290589451915e-06,
+      "loss": 0.20552108764648438,
+      "step": 10600
+    },
+    {
+      "epoch": 0.7445708376421923,
+      "grad_norm": 2.1073572635650635,
+      "learning_rate": 5.109962082040676e-06,
+      "loss": 0.20681476593017578,
+      "step": 10800
+    },
+    {
+      "epoch": 0.7583591864874182,
+      "grad_norm": 1.3458033800125122,
+      "learning_rate": 4.83419510513616e-06,
+      "loss": 0.20359535217285157,
+      "step": 11000
+    },
+    {
+      "epoch": 0.7721475353326439,
+      "grad_norm": 1.1731774806976318,
+      "learning_rate": 4.558428128231645e-06,
+      "loss": 0.2056349754333496,
+      "step": 11200
+    },
+    {
+      "epoch": 0.7859358841778697,
+      "grad_norm": 1.3708990812301636,
+      "learning_rate": 4.282661151327129e-06,
+      "loss": 0.2058660125732422,
+      "step": 11400
+    },
+    {
+      "epoch": 0.7997242330230955,
+      "grad_norm": 1.1624313592910767,
+      "learning_rate": 4.0068941744226135e-06,
+      "loss": 0.20310092926025392,
+      "step": 11600
+    },
+    {
+      "epoch": 0.8135125818683213,
+      "grad_norm": 1.3882473707199097,
+      "learning_rate": 3.7311271975180975e-06,
+      "loss": 0.2082367706298828,
+      "step": 11800
+    },
+    {
+      "epoch": 0.827300930713547,
+      "grad_norm": 1.7915494441986084,
+      "learning_rate": 3.455360220613582e-06,
+      "loss": 0.20367862701416015,
+      "step": 12000
+    },
+    {
+      "epoch": 0.827300930713547,
+      "eval_accuracy": 0.9442049177737276,
+      "eval_f1": 0.520962146523804,
+      "eval_loss": 0.16711591184139252,
+      "eval_precision": 0.4862475207302793,
+      "eval_recall": 0.5610146126275158,
+      "eval_runtime": 9.7345,
+      "eval_samples_per_second": 2054.558,
+      "eval_steps_per_second": 32.154,
+      "step": 12000
+    },
+    {
+      "epoch": 0.8410892795587729,
+      "grad_norm": 1.900614857673645,
+      "learning_rate": 3.179593243709066e-06,
+      "loss": 0.20414718627929687,
+      "step": 12200
+    },
+    {
+      "epoch": 0.8548776284039986,
+      "grad_norm": 1.5320205688476562,
+      "learning_rate": 2.9038262668045503e-06,
+      "loss": 0.19842819213867188,
+      "step": 12400
+    },
+    {
+      "epoch": 0.8686659772492245,
+      "grad_norm": 1.0634764432907104,
+      "learning_rate": 2.6280592899000347e-06,
+      "loss": 0.20455713272094728,
+      "step": 12600
+    },
+    {
+      "epoch": 0.8824543260944502,
+      "grad_norm": 1.5051591396331787,
+      "learning_rate": 2.352292312995519e-06,
+      "loss": 0.20511449813842775,
+      "step": 12800
+    },
+    {
+      "epoch": 0.896242674939676,
+      "grad_norm": 1.2883975505828857,
+      "learning_rate": 2.076525336091003e-06,
+      "loss": 0.1981792640686035,
+      "step": 13000
+    },
+    {
+      "epoch": 0.9100310237849017,
+      "grad_norm": 1.901268720626831,
+      "learning_rate": 1.8007583591864874e-06,
+      "loss": 0.20567977905273438,
+      "step": 13200
+    },
+    {
+      "epoch": 0.9238193726301276,
+      "grad_norm": 1.828506350517273,
+      "learning_rate": 1.5249913822819716e-06,
+      "loss": 0.20068155288696288,
+      "step": 13400
+    },
+    {
+      "epoch": 0.9376077214753533,
+      "grad_norm": 1.401857614517212,
+      "learning_rate": 1.2492244053774563e-06,
+      "loss": 0.20199392318725587,
+      "step": 13600
+    },
+    {
+      "epoch": 0.9513960703205792,
+      "grad_norm": 1.8156548738479614,
+      "learning_rate": 9.734574284729404e-07,
+      "loss": 0.20472015380859376,
+      "step": 13800
+    },
+    {
+      "epoch": 0.9651844191658049,
+      "grad_norm": 1.086504578590393,
+      "learning_rate": 6.976904515684247e-07,
+      "loss": 0.20377262115478514,
+      "step": 14000
+    },
+    {
+      "epoch": 0.9651844191658049,
+      "eval_accuracy": 0.9452518410046249,
+      "eval_f1": 0.5233849845820377,
+      "eval_loss": 0.16398483514785767,
+      "eval_precision": 0.4918641026262822,
+      "eval_recall": 0.5592224979321754,
+      "eval_runtime": 10.6768,
+      "eval_samples_per_second": 1873.214,
+      "eval_steps_per_second": 29.316,
+      "step": 14000
+    },
+    {
+      "epoch": 0.9789727680110307,
+      "grad_norm": 1.6399117708206177,
+      "learning_rate": 4.21923474663909e-07,
+      "loss": 0.2011069679260254,
+      "step": 14200
+    },
+    {
+      "epoch": 0.9927611168562565,
+      "grad_norm": 1.01554536819458,
+      "learning_rate": 1.4615649775939334e-07,
+      "loss": 0.20439926147460938,
+      "step": 14400
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9452715942731323,
+      "eval_f1": 0.5242530574333041,
+      "eval_loss": 0.16400153934955597,
+      "eval_precision": 0.4926295883237162,
+      "eval_recall": 0.5602150537634408,
+      "eval_runtime": 10.1161,
+      "eval_samples_per_second": 1977.053,
+      "eval_steps_per_second": 30.941,
+      "step": 14505
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 14505,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 2000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 93817250377032.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

full-openpii-500k/checkpoints/checkpoint-14505/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec4df483fc65e86a8e85fec1834084cd6c3c69b50ccd36f2eeb0fabf3798ed8f
+size 5201

full-openpii-500k/checkpoints/eval_results.json

@@ -0,0 +1,11 @@
+{
+    "epoch": 1.0,
+    "eval_accuracy": 0.9452715942731323,
+    "eval_f1": 0.5242530574333041,
+    "eval_loss": 0.16400153934955597,
+    "eval_precision": 0.4926295883237162,
+    "eval_recall": 0.5602150537634408,
+    "eval_runtime": 10.1647,
+    "eval_samples_per_second": 1967.587,
+    "eval_steps_per_second": 30.793
+}

full-openpii-500k/final/config.json

@@ -0,0 +1,86 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "id2label": {
+    "0": "O",
+    "1": "B-ADDRESS",
+    "2": "I-ADDRESS",
+    "3": "B-API_KEY",
+    "4": "I-API_KEY",
+    "5": "B-CREDIT_CARD",
+    "6": "I-CREDIT_CARD",
+    "7": "B-DATE_OF_BIRTH",
+    "8": "I-DATE_OF_BIRTH",
+    "9": "B-DRIVER_LICENSE",
+    "10": "I-DRIVER_LICENSE",
+    "11": "B-EMAIL",
+    "12": "I-EMAIL",
+    "13": "B-IP_ADDRESS",
+    "14": "I-IP_ADDRESS",
+    "15": "B-LOCATION",
+    "16": "I-LOCATION",
+    "17": "B-PASSWORD",
+    "18": "I-PASSWORD",
+    "19": "B-PERSON_NAME",
+    "20": "I-PERSON_NAME",
+    "21": "B-PHONE",
+    "22": "I-PHONE",
+    "23": "B-SSN",
+    "24": "I-SSN",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "is_decoder": false,
+  "label2id": {
+    "B-ADDRESS": 1,
+    "B-API_KEY": 3,
+    "B-CREDIT_CARD": 5,
+    "B-DATE_OF_BIRTH": 7,
+    "B-DRIVER_LICENSE": 9,
+    "B-EMAIL": 11,
+    "B-IP_ADDRESS": 13,
+    "B-LOCATION": 15,
+    "B-PASSWORD": 17,
+    "B-PERSON_NAME": 19,
+    "B-PHONE": 21,
+    "B-SSN": 23,
+    "B-USERNAME": 25,
+    "I-ADDRESS": 2,
+    "I-API_KEY": 4,
+    "I-CREDIT_CARD": 6,
+    "I-DATE_OF_BIRTH": 8,
+    "I-DRIVER_LICENSE": 10,
+    "I-EMAIL": 12,
+    "I-IP_ADDRESS": 14,
+    "I-LOCATION": 16,
+    "I-PASSWORD": 18,
+    "I-PERSON_NAME": 20,
+    "I-PHONE": 22,
+    "I-SSN": 24,
+    "I-USERNAME": 26,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

full-openpii-500k/final/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b976a7b1be395ceb87e3d25eb691cc692395118bec627e6437d605d9486215f
+size 17495980

full-openpii-500k/final/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 160,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

full-openpii-500k/final/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec4df483fc65e86a8e85fec1834084cd6c3c69b50ccd36f2eeb0fabf3798ed8f
+size 5201

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f4385a310803f053a1106cf02c04c6b1149a5f0842b2ec00094ad527827476c
+oid sha256:0587beffa1064e66ea7502f3cf8b70706fa4c54dee54e28db2e96de318313646
 size 435697572

pilot-10k/checkpoints/all_results.json

@@ -0,0 +1,11 @@
+{
+    "epoch": 1.0,
+    "eval_accuracy": 0.8564406534624838,
+    "eval_f1": 0.15016077170418007,
+    "eval_loss": 0.5771309733390808,
+    "eval_precision": 0.17763408139977177,
+    "eval_recall": 0.13004734057365636,
+    "eval_runtime": 2.2907,
+    "eval_samples_per_second": 873.108,
+    "eval_steps_per_second": 13.97
+}

pilot-10k/checkpoints/checkpoint-250/config.json

@@ -0,0 +1,86 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "id2label": {
+    "0": "O",
+    "1": "B-ADDRESS",
+    "2": "I-ADDRESS",
+    "3": "B-API_KEY",
+    "4": "I-API_KEY",
+    "5": "B-CREDIT_CARD",
+    "6": "I-CREDIT_CARD",
+    "7": "B-DATE_OF_BIRTH",
+    "8": "I-DATE_OF_BIRTH",
+    "9": "B-DRIVER_LICENSE",
+    "10": "I-DRIVER_LICENSE",
+    "11": "B-EMAIL",
+    "12": "I-EMAIL",
+    "13": "B-IP_ADDRESS",
+    "14": "I-IP_ADDRESS",
+    "15": "B-LOCATION",
+    "16": "I-LOCATION",
+    "17": "B-PASSWORD",
+    "18": "I-PASSWORD",
+    "19": "B-PERSON_NAME",
+    "20": "I-PERSON_NAME",
+    "21": "B-PHONE",
+    "22": "I-PHONE",
+    "23": "B-SSN",
+    "24": "I-SSN",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "is_decoder": false,
+  "label2id": {
+    "B-ADDRESS": 1,
+    "B-API_KEY": 3,
+    "B-CREDIT_CARD": 5,
+    "B-DATE_OF_BIRTH": 7,
+    "B-DRIVER_LICENSE": 9,
+    "B-EMAIL": 11,
+    "B-IP_ADDRESS": 13,
+    "B-LOCATION": 15,
+    "B-PASSWORD": 17,
+    "B-PERSON_NAME": 19,
+    "B-PHONE": 21,
+    "B-SSN": 23,
+    "B-USERNAME": 25,
+    "I-ADDRESS": 2,
+    "I-API_KEY": 4,
+    "I-CREDIT_CARD": 6,
+    "I-DATE_OF_BIRTH": 8,
+    "I-DRIVER_LICENSE": 10,
+    "I-EMAIL": 12,
+    "I-IP_ADDRESS": 14,
+    "I-LOCATION": 16,
+    "I-PASSWORD": 18,
+    "I-PERSON_NAME": 20,
+    "I-PHONE": 22,
+    "I-SSN": 24,
+    "I-USERNAME": 26,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

pilot-10k/checkpoints/checkpoint-250/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06281efec76671367d966cd7c8a0b3d07d78f505b6c44e2afbb3558fb4f6fd24
+size 17495980

pilot-10k/checkpoints/checkpoint-250/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5a691dbe327f1e7b4a5f365fae200c1ad405c38acea87667669bec6e6676fca
+size 35016907

pilot-10k/checkpoints/checkpoint-250/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff9ccac4440cabc60152c5f65ef3ec8bfa66181ad04cb8dd113775c7a36ce886
+size 14645

pilot-10k/checkpoints/checkpoint-250/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d6fca631a6bcdfa2416587314d206a68f40e27a07bc674b76e72a93db4e5058
+size 1383

pilot-10k/checkpoints/checkpoint-250/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f01b4104e0a40b46267323affa2177c38dfae9ac349ae035c024b8624a4c215
+size 1465

pilot-10k/checkpoints/checkpoint-250/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 160,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

pilot-10k/checkpoints/checkpoint-250/trainer_state.json

@@ -0,0 +1,116 @@
+{
+  "best_global_step": 250,
+  "best_metric": 0.14514024787997393,
+  "best_model_checkpoint": "/outputs/pilot-10k/checkpoints/checkpoint-250",
+  "epoch": 0.7987220447284346,
+  "eval_steps": 250,
+  "global_step": 250,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.07987220447284345,
+      "grad_norm": 2.651143789291382,
+      "learning_rate": 1.8466453674121408e-05,
+      "loss": 1.015383529663086,
+      "step": 25
+    },
+    {
+      "epoch": 0.1597444089456869,
+      "grad_norm": 2.463940143585205,
+      "learning_rate": 1.6869009584664538e-05,
+      "loss": 0.9280670928955078,
+      "step": 50
+    },
+    {
+      "epoch": 0.23961661341853036,
+      "grad_norm": 1.9472101926803589,
+      "learning_rate": 1.527156549520767e-05,
+      "loss": 0.8376800537109375,
+      "step": 75
+    },
+    {
+      "epoch": 0.3194888178913738,
+      "grad_norm": 3.205868721008301,
+      "learning_rate": 1.36741214057508e-05,
+      "loss": 0.7408914184570312,
+      "step": 100
+    },
+    {
+      "epoch": 0.3993610223642173,
+      "grad_norm": 1.723464012145996,
+      "learning_rate": 1.207667731629393e-05,
+      "loss": 0.730751724243164,
+      "step": 125
+    },
+    {
+      "epoch": 0.4792332268370607,
+      "grad_norm": 1.4765894412994385,
+      "learning_rate": 1.0479233226837063e-05,
+      "loss": 0.712818603515625,
+      "step": 150
+    },
+    {
+      "epoch": 0.5591054313099042,
+      "grad_norm": 1.5612858533859253,
+      "learning_rate": 8.881789137380193e-06,
+      "loss": 0.6849630737304687,
+      "step": 175
+    },
+    {
+      "epoch": 0.6389776357827476,
+      "grad_norm": 2.112462282180786,
+      "learning_rate": 7.284345047923324e-06,
+      "loss": 0.6824495697021484,
+      "step": 200
+    },
+    {
+      "epoch": 0.7188498402555911,
+      "grad_norm": 1.9042291641235352,
+      "learning_rate": 5.6869009584664534e-06,
+      "loss": 0.6615296173095703,
+      "step": 225
+    },
+    {
+      "epoch": 0.7987220447284346,
+      "grad_norm": 1.3791943788528442,
+      "learning_rate": 4.089456869009585e-06,
+      "loss": 0.6428241729736328,
+      "step": 250
+    },
+    {
+      "epoch": 0.7987220447284346,
+      "eval_accuracy": 0.8551395113488507,
+      "eval_f1": 0.14514024787997393,
+      "eval_loss": 0.5809560418128967,
+      "eval_precision": 0.17512790240062967,
+      "eval_recall": 0.1239209133945976,
+      "eval_runtime": 2.0647,
+      "eval_samples_per_second": 968.683,
+      "eval_steps_per_second": 15.499,
+      "step": 250
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 313,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 250,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1640532656256.0,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

pilot-10k/checkpoints/checkpoint-250/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:092b1532cd7ec24f5a56e0bf3ba33a34d942d8a4222f6bedff5a41e39b346bb2
+size 5201