Upload folder using huggingface_hub

README.md

@@ -0,0 +1,178 @@
+---
+language: en
+library_name: transformers
+license: gpl-3.0
+tags:
+  - mist
+  - chemistry
+  - molecular-property-prediction
+---
+
+# MIST: Molecular Insight SMILES Transformers
+
+MIST is a family of molecular foundation models for molecular property prediction.
+The models were pre-trained on SMILES strings from the [Enamine REAL Space](https://enamine.net/compound-collections/real-compounds/real-space-navigator) dataset using the Masked Language Modeling (MLM) objective, then fine-tuned for downstream prediction tasks.
+
+## Model Details
+
+- **Architecture**: Encoder-only transformer [``RoBERTa-PreLayerNorm``](https://huggingface.co/docs/transformers/en/model_doc/roberta-prelayernorm)
+- **Pre-training**: Masked Language Modeling on molecular SMILES
+- **Tokenization**: [``Smirk``](https://eeg.engin.umich.edu/smirk/) tokenizer
+
+
+### Quick Start
+
+```python
+from transformers import AutoModel
+
+# Load the model
+model = AutoModel.from_pretrained(
+    "path/to/model",
+    trust_remote_code=True
+)
+
+# Make predictions
+smiles_batch = [
+    "CCO",           # Ethanol
+    "CC(=O)O",       # Acetic acid
+    "c1ccccc1"       # Benzene
+]
+results = model.predict(smiles_batch)
+```
+
+### Setting Up Your Environment
+
+Create a virtual environment and install dependencies:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+pip install -r requirements.txt
+```
+
+> **Note**: SMIRK tokenizers require Rust to be installed. See the [Rust installation guide](https://www.rust-lang.org/tools/install) for details.
+
+
+## Model Inputs and Outputs
+
+### Inputs
+- **SMILES strings**: Standard SMILES notation for molecular structures
+- **Batch size**: Variable, automatically padded during inference
+
+### Outputs
+- **Predictions**: Task-specific numerical or categorical predictions
+- **Format**: Dictionary with channel names and predicted values (if channels are configured), or raw tensor output
+
+
+## Provided Models
+
+### Pre-trained
+- `mist-1.8B-dh61satt`: Flagship MIST model (MIST-1.8B)
+- `mist-28M-ti624ev1`: Smaller MIST model (MIST-28M).
+
+Below is a full list of finetuned variants hosted on HuggingFace:
+### MoleculeNet Benchmark Models
+
+| Folder                       | Encoder  | Dataset                              |
+| ---------------------------- | :------: | ------------------------------------ |
+| mist-1.8B-fbdn8e35-bbbp      | MIST-1.8B| MoleculeNet BBBP                     |
+| mist-1.8B-1a4puhg2-hiv       | MIST-1.8B| MoleculeNet HIV                      |
+| mist-1.8B-m50jgolp-bace      | MIST-1.8B| MoleculeNet BACE                     |
+| mist-1.8B-uop1z0dc-tox21     | MIST-1.8B| MoleculeNet Tox21                    |
+| mist-1.8B-lu1l5ieh-clintox   | MIST-1.8B| MoleculeNet ClinTox                  |
+| mist-1.8B-l1wfo7oa-sider     | MIST-1.8B| MoleculeNet SIDER.                   |
+| mist-1.8B-hxiygjsm-esol      | MIST-1.8B| MoleculeNet ESOL                     |
+| mist-1.8B-iwqj2cld-freesolv  | MIST-1.8B| MoleculeNet FreeSolv                 |
+| mist-1.8B-jvt4azpz-lipo      | MIST-1.8B| MoleculeNet Lipophilicity            |
+| mist-1.8B-8nd1ot5j-qm8       | MIST-1.8B| MoleculeNet QM8                      |
+| mist-28M-3xpfhv48-bbbp       | MIST-28M | MoleculeNet BBBP                     |
+| mist-28M-8fh43gke-hiv        | MIST-28M | MoleculeNet HIV                      |
+| mist-28M-8loj3bab-bace       | MIST-28M | MoleculeNet BACE                     |
+| mist-28M-kw4ks27p-tox21      | MIST-28M | MoleculeNet Tox21                    |
+| mist-28M-97vfcykk-clintox    | MIST-28M | MoleculeNet ClinTox                  |
+| mist-28M-z8qo16uy-sider      | MIST-28M | MoleculeNet SIDER                    |
+| mist-28M-kcwb9le5-esol       | MIST-28M | MoleculeNet ESOL                     |
+| mist-28M-0uiq7o7m-freesolv   | MIST-28M | MoleculeNet FreeSolv                 |
+| mist-28M-xzr5ulva-lipo       | MIST-28M | MoleculeNet Lipophilicity            |
+| mist-28M-gzwqzpcr-qm8        | MIST-28M | MoleculeNet QM8                      |
+
+
+#### QM9 Benchmark Models
+The single target (MIST-1.8B encoder) models for properties in QM9 are available.
+
+| Folder                       | Encoder  | Target                                                            |
+| ---------------------------- | :------: | ----------------------------------------------------------------- |
+| mist-1.8B-ez05expv-mu        | MIST-1.8B| μ - Dipole moment (unit: D)                                       |
+| mist-1.8B-rcwary93-alpha     | MIST-1.8B| α - Isotropic polarizability (unit: Bohr^3)                       |
+| mist-1.8B-jmjosq12-homo      | MIST-1.8B| HOMO - Highest occupied molecular orbital energy (unit: Hartree)  |
+| mist-1.8B-n14wshc9-lumo      | MIST-1.8B| LUMO - Lowest unoccupied molecular orbital energy (unit: Hartree) |
+| mist-1.8B-kayun6v3-gap       | MIST-1.8B| Gap - Gap between HOMO and LUMO (unit: Hartree)                   |
+| mist-1.8B-xxe7t35e-r2        | MIST-1.8B| \<R2\> - Electronic spatial extent (unit: Bohr^2)                 |
+| mist-1.8B-6nmcwyrp-zpve      | MIST-1.8B| ZPVE - Zero point vibrational energy (unit: Hartree)              |
+| mist-1.8B-a7akimjj-u0        | MIST-1.8B| U0 - Internal energy at 0K (unit: Hartree)                        |
+| mist-1.8B-85f24xkj-u298      | MIST-1.8B| U298 - Internal energy at 298.15K (unit: Hartree)                 |
+| mist-1.8B-3fbbz4is-h298      | MIST-1.8B| H298 - Enthalpy at 298.15K (unit: Hartree)                        |
+| mist-1.8B-09sntn03-g298      | MIST-1.8B| G298 - Free energy at 298.15K (unit: Hartree)                     |
+| mist-1.8B-j356b3nf-cv        | MIST-1.8B| Cv - Heat capacity at 298.15K (unit: cal/(mol*K))                 |
+
+- `mist-ti624ev1-moleculenet`: Contains MoleculeNet benchmark MIST-28M models trained as part of doi:10.5281/zenodo.13761263
+
+
+### Finetuned Single Task Models
+
+These models consist of a MIST-encoder and task network finetuned on a single dataset used in the applications demonstrated in the manuscript.
+
+| Folder                    | Encoder  | Dataset                                                     |
+| ------------------------- | :------: | ----------------------------------------------------------- |
+| mist-26.9M-48kpooqf-odour | MIST-28M | Olfaction                                                   |
+| mist-26.9M-6hk5coof-dn    | MIST-28M | Donor Number                                                |
+| mist-26.9M-0vxdbm36-kt    | MIST-28M | Kamlet-Taft Solvochromatic Parameters                       |
+| mist-26.9M-b302p09x-bp    | MIST-28M | Boiling Point (Part of Characteristic Temperatures Dataset) |
+| mist-26.9M-cyuo2xb6-fp    | MIST-28M | Flash Point (Part of Characteristic Temperatures Dataset)   |
+| mist-26.9M-y3ge5pf9-mp    | MIST-28M | Melting Point (Part of Characteristic Temperatures Dataset) |
+
+### Finetuned Multi-Task Models
+These are additional multi-target finetuned models consisting of a MIST encoder and task network.
+| Folder                     | Encoder  | Dataset                                                     |
+| -------------------------- | :------: | ----------------------------------------------------------- |
+| mist-26.9M-kkgx0omx-qm9    | MIST-28M | QM9 Dataset with SMILES randomization                       |
+| mist-28M-ttqcvt6fs-toxcast | MIST-28M | ToxCast                                                     |
+| mist-28M-yr1urd2c-muv      | MIST-28M | Maximum Unbiased Validation (MUV)                           |
+
+### Finetuned Mixture Models
+
+These models consist of a MIST-encoder and physics informed task network for mixture property prediction.
+| Folder                           | Encoder  | Dataset                                                     |
+| -------------------------------- | :------: | ----------------------------------------------------------- |
+| mist-conductivity-28M-2mpg8dcd   | MIST-28M | Ionic Conductivity                                          |
+| mist-mixtures-zffffbex           | MIST-28M | Excess Density, Molar Volume and Molar Enthalpy             |
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@online{MIST,
+  title = {Foundation Models for Discovery and Exploration in Chemical Space},
+  author = {Wadell, Alexius and Bhutani, Anoushka and Azumah, Victor and Ellis-Mohr, Austin R. and Kelly, Celia and Zhao, Hancheng and Nayak, Anuj K. and Hegazy, Kareem and Brace, Alexander and Lin, Hongyi and Emani, Murali and Vishwanath, Venkatram and Gering, Kevin and Alkan, Melisa and Gibbs, Tom and Wells, Jack and Varshney, Lav R. and Ramsundar, Bharath and Duraisamy, Karthik and Mahoney, Michael W. and Ramanathan, Arvind and Viswanathan, Venkatasubramanian},
+  date = {2025-10-20},
+  eprint = {2510.18900},
+  eprinttype = {arXiv},
+  eprintclass = {physics},
+  doi = {10.48550/arXiv.2510.18900},
+  url = {http://arxiv.org/abs/2510.18900},  
+}
+```
+
+## License and Notice
+
+Model weights are provided as-is for research purposes only, without guarantees of correctness, fitness for purpose, or warranties of any kind.
+
+**Restrictions:**
+- Research use only
+- No redistribution without permission
+- No commercial use without licensing agreement
+
+For questions, issues, or licensing inquiries, please contact [venkvis@umich.edu](mailto:venkvis@umich.edu).
+
+<hr>

config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "RobertaPreLayerNormForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "enable_token_counter": true,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 2304,
+  "initializer_range": 0.02,
+  "intermediate_size": 9216,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 2048,
+  "model_type": "roberta-prelayernorm",
+  "num_attention_heads": 18,
+  "num_hidden_layers": 28,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 165
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77edb147919e841f3bd5c8c78a7bc515d22ce02e16e4ea084a1801ba107c58b6
+size 4949035880

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34cd4f65d0da211d50d2f6822b5a4667d2d421686534009f7508cdcd5b94284e
+size 2230604724

model.safetensors.index.json

@@ -0,0 +1,468 @@
+{
+  "metadata": {
+    "total_parameters": 1794894501,
+    "total_size": 7179578004
+  },
+  "weight_map": {
+    "lm_head.bias": "model-00002-of-00002.safetensors",
+    "lm_head.dense.bias": "model-00002-of-00002.safetensors",
+    "lm_head.dense.weight": "model-00002-of-00002.safetensors",
+    "lm_head.layer_norm.bias": "model-00002-of-00002.safetensors",
+    "lm_head.layer_norm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.embeddings.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.embeddings.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.embeddings.position_embeddings.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.embeddings.token_type_embeddings.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.embeddings.word_embeddings.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.0.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.1.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.10.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.11.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.12.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.13.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.14.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.15.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.16.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.17.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.18.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.19.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.2.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.20.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.21.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.22.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.23.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.24.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.25.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.26.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.key.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.key.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.query.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.query.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.value.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.attention.self.value.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.intermediate.LayerNorm.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.intermediate.LayerNorm.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.intermediate.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.intermediate.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.output.dense.bias": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.27.output.dense.weight": "model-00002-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.3.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.4.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.5.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.6.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.7.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.8.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.output.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.key.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.key.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.query.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.query.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.value.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.attention.self.value.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.intermediate.LayerNorm.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.intermediate.LayerNorm.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.intermediate.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.intermediate.dense.weight": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.output.dense.bias": "model-00001-of-00002.safetensors",
+    "roberta_prelayernorm.encoder.layer.9.output.dense.weight": "model-00001-of-00002.safetensors"
+  }
+}

requirements.txt

@@ -0,0 +1,4 @@
+transformers==4.57.1
+torch==2.9.0
+scikit-learn==1.7.2
+smirk==0.2.0.dev0

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[BOS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[EOS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,267 @@
+{
+  "version": "1.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 0,
+      "content": "[UNK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 159,
+      "content": "[BOS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 160,
+      "content": "[EOS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 161,
+      "content": "[SEP]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 162,
+      "content": "[PAD]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 163,
+      "content": "[CLS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 164,
+      "content": "[MASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": {
+    "type": "Sequence",
+    "normalizers": [
+      {
+        "type": "Replace",
+        "pattern": {
+          "String": "++"
+        },
+        "content": "+2"
+      },
+      {
+        "type": "Replace",
+        "pattern": {
+          "String": "--"
+        },
+        "content": "-2"
+      },
+      {
+        "type": "Strip",
+        "strip_left": true,
+        "strip_right": true
+      }
+    ]
+  },
+  "pre_tokenizer": {
+    "outer": "Br?|Cl?|F|I|N|O|P|S|b|c|n|o|p|s|\\*|[\\.\\-=\\#\\$:/\\\\]|\\d|%|\\(|\\)|\\[.*?]",
+    "inner": "(\\d+)?(A[c|g|l|m|r|s|t|u]|B[a|e|h|i|k|r]?|C[a|d|e|f|l|m|n|o|r|s|u]?|D[b|s|y]|E[r|s|u]|F[e|l|m|r]?|G[a|d|e]|H[e|f|g|o|s]?|I[n|r]?|Kr?|L[a|i|r|u|v]|M[c|d|g|n|o|t]|N[a|b|d|e|h|i|o|p]?|O[g|s]?|P[a|b|d|m|o|r|t|u]?|R[a|b|e|f|g|h|n|u]|S[b|c|e|g|i|m|n|r]?|T[a|b|c|e|h|i|l|m|s]|U|V|W|Xe|Yb?|Z[n|r]|as|b|c|n|o|p|se?|\\*)(?:(@(?:@|AL|OH|SP|T[B|H])?)(\\d{1,2})?)?(?:(H)(\\d)?)?(?:([+-]{1,2})(\\d{0,2}))?(?:(:)(\\d+))?"
+  },
+  "post_processor": null,
+  "decoder": {
+    "type": "Fuse"
+  },
+  "model": {
+    "type": "WordLevel",
+    "vocab": {
+      "[UNK]": 0,
+      "#": 1,
+      "$": 2,
+      "%": 3,
+      "(": 4,
+      ")": 5,
+      "*": 6,
+      "+": 7,
+      "-": 8,
+      ".": 9,
+      "/": 10,
+      "0": 11,
+      "1": 12,
+      "2": 13,
+      "3": 14,
+      "4": 15,
+      "5": 16,
+      "6": 17,
+      "7": 18,
+      "8": 19,
+      "9": 20,
+      ":": 21,
+      "=": 22,
+      "@": 23,
+      "@@": 24,
+      "@AL": 25,
+      "@OH": 26,
+      "@SP": 27,
+      "@TB": 28,
+      "@TH": 29,
+      "Ac": 30,
+      "Ag": 31,
+      "Al": 32,
+      "Am": 33,
+      "Ar": 34,
+      "As": 35,
+      "At": 36,
+      "Au": 37,
+      "B": 38,
+      "Ba": 39,
+      "Be": 40,
+      "Bh": 41,
+      "Bi": 42,
+      "Bk": 43,
+      "Br": 44,
+      "C": 45,
+      "Ca": 46,
+      "Cd": 47,
+      "Ce": 48,
+      "Cf": 49,
+      "Cl": 50,
+      "Cm": 51,
+      "Cn": 52,
+      "Co": 53,
+      "Cr": 54,
+      "Cs": 55,
+      "Cu": 56,
+      "Db": 57,
+      "Ds": 58,
+      "Dy": 59,
+      "Er": 60,
+      "Es": 61,
+      "Eu": 62,
+      "F": 63,
+      "Fe": 64,
+      "Fl": 65,
+      "Fm": 66,
+      "Fr": 67,
+      "Ga": 68,
+      "Gd": 69,
+      "Ge": 70,
+      "H": 71,
+      "He": 72,
+      "Hf": 73,
+      "Hg": 74,
+      "Ho": 75,
+      "Hs": 76,
+      "I": 77,
+      "In": 78,
+      "Ir": 79,
+      "K": 80,
+      "Kr": 81,
+      "La": 82,
+      "Li": 83,
+      "Lr": 84,
+      "Lu": 85,
+      "Lv": 86,
+      "Mc": 87,
+      "Md": 88,
+      "Mg": 89,
+      "Mn": 90,
+      "Mo": 91,
+      "Mt": 92,
+      "N": 93,
+      "Na": 94,
+      "Nb": 95,
+      "Nd": 96,
+      "Ne": 97,
+      "Nh": 98,
+      "Ni": 99,
+      "No": 100,
+      "Np": 101,
+      "O": 102,
+      "Og": 103,
+      "Os": 104,
+      "P": 105,
+      "Pa": 106,
+      "Pb": 107,
+      "Pd": 108,
+      "Pm": 109,
+      "Po": 110,
+      "Pr": 111,
+      "Pt": 112,
+      "Pu": 113,
+      "Ra": 114,
+      "Rb": 115,
+      "Re": 116,
+      "Rf": 117,
+      "Rg": 118,
+      "Rh": 119,
+      "Rn": 120,
+      "Ru": 121,
+      "S": 122,
+      "Sb": 123,
+      "Sc": 124,
+      "Se": 125,
+      "Sg": 126,
+      "Si": 127,
+      "Sm": 128,
+      "Sn": 129,
+      "Sr": 130,
+      "Ta": 131,
+      "Tb": 132,
+      "Tc": 133,
+      "Te": 134,
+      "Th": 135,
+      "Ti": 136,
+      "Tl": 137,
+      "Tm": 138,
+      "Ts": 139,
+      "U": 140,
+      "V": 141,
+      "W": 142,
+      "Xe": 143,
+      "Y": 144,
+      "Yb": 145,
+      "Zn": 146,
+      "Zr": 147,
+      "[": 148,
+      "\\": 149,
+      "]": 150,
+      "as": 151,
+      "b": 152,
+      "c": 153,
+      "n": 154,
+      "o": 155,
+      "p": 156,
+      "s": 157,
+      "se": 158
+    },
+    "unk_token": "[UNK]"
+  }
+}

tokenizer_config.json

@@ -0,0 +1,72 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "[BOS]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "[EOS]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "[BOS]",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "eos_token": "[EOS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "SmirkTokenizerFast",
+  "unk_token": "[UNK]",
+  "vocab_file": "/nfs/turbo/coe-venkvis/abhutani/electrolyte-fm/.venv/lib/python3.10/site-packages/smirk/vocab_smiles.json"
+}