Publish current best Formosan ASR checkpoint for Saaroa

.gitattributes

@@ -1,35 +1 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,131 @@
+---
+language:
+- "sxr"
+library_name: transformers
+pipeline_tag: automatic-speech-recognition
+tags:
+- formosanbank
+- formosan
+- endangered-languages
+- speech
+- asr
+- xlsr53
+- tacl
+base_model: facebook/wav2vec2-large-xlsr-53
+base_model_relation: finetune
+license: cc-by-4.0
+metrics:
+- wer
+- cer
+---
+
+# Formosan ASR — Saaroa
+
+This is FormosanBank's current general-purpose automatic speech recognition
+checkpoint for **Saaroa**. It was selected by the lowest normalized
+micro word error rate among the completed language-level TACL systems evaluated
+on the same frozen all-corpora test partition.
+
+## Model
+
+- Repository: `FormosanBank/formosan-asr-saaroa`
+- Architecture family: **XLS-R 53**
+- Training path: **train-only pan-Formosan continued pretraining, pan-Formosan supervised fine-tuning, and language adaptation**
+- Base model: `facebook/wav2vec2-large-xlsr-53`
+- Parameters are stored in `model.safetensors`
+
+### Training lineage
+
+1. Base checkpoint: `facebook/wav2vec2-large-xlsr-53`.
+2. Continued self-supervised pretraining on the frozen pan-Formosan training partition only; development and test audio were excluded.
+3. Supervised pan-Formosan ASR training, then adaptation to the target language's pooled training corpora.
+
+The target-language supervised stage used `ILRDF`, `ePark1`, `ePark2`, `ePark3`. The experiment used
+`dataset_v1`, frozen `split_v2`, normalization `formosan_safe_v1`, and seed
+`13`. Full machine-readable provenance is included in
+`training_config.json`.
+
+## Evaluation
+
+Evaluation uses the frozen, leakage-controlled `split_v2` test manifests and
+normalized text. Metrics are computed by summing edit counts across all
+available Saaroa test corpora, so larger corpora contribute in
+proportion to their reference tokens.
+
+- Normalized micro WER: **32.19%**
+- Normalized micro CER: **4.22%**
+- Test utterances: **1,014**
+- Test corpora: **4**
+- Automated readiness: **review**
+
+The automated readiness label is `review`, with zero hard failures. The retained warning is recorded in `readiness_summary.json` and should be considered when reproducing or promoting the checkpoint.
+
+| Corpus | Utterances | WER | CER |
+|---|---:|---:|---:|
+| ILRDF | 414 | 42.19% | 5.98% |
+| ePark1 | 161 | 26.00% | 2.53% |
+| ePark2 | 94 | 33.80% | 8.10% |
+| ePark3 | 345 | 24.46% | 1.92% |
+
+Detailed edit counts are provided in `evaluation_results.json` and
+`metrics_by_corpus.tsv`.
+
+These figures are specific to the frozen TACL evaluation protocol. They should
+not be compared directly with the April 2026 stage-one releases, which used a
+different experiment snapshot.
+
+## Usage
+
+```python
+from transformers import pipeline
+
+repo_id = "FormosanBank/formosan-asr-saaroa"
+transcribe = pipeline("automatic-speech-recognition", model=repo_id)
+result = transcribe("path/to/16khz_audio.wav")
+print(result["text"])
+```
+
+For long recordings, segment the audio before inference. The training recipe
+used utterances between 2 and 20 seconds.
+
+## Intended use
+
+This checkpoint supports research, education, language documentation, and
+revitalization work involving Saaroa. Human review is recommended
+before using transcriptions in archives, publications, teaching materials, or
+other consequential settings.
+
+## Limitations
+
+Performance varies substantially across corpora, speakers, recording
+conditions, dialects, speaking styles, and orthographic conventions. The
+aggregate score can hide weak performance on a particular corpus. The model
+may omit, substitute, or hallucinate words and should not be treated as an
+authoritative transcription source.
+
+## Data provenance
+
+Training and evaluation artifacts come from the frozen Hunter Formosan TACL
+pipeline. Source corpus IDs can include Bible, ILRDF, NTU, ePark1, ePark2,
+ePark3, Xuan, YeddaPalemeqBlog, Youtube, and YutasWilang, depending on language
+availability. Audio is not redistributed in this repository. Underlying
+recordings retain their original rights and access conditions.
+
+## License and attribution
+
+This model release and FormosanBank annotations/metadata are provided under
+**CC BY 4.0**. The upstream `facebook/wav2vec2-large-xlsr-53` checkpoint is distributed under
+Apache 2.0. Users remain responsible for respecting the terms associated with
+underlying source recordings.
+
+Please cite FormosanBank:
+
+```bibtex
+@misc{mohamed2024formosanbank,
+  author = {Mohamed, W. and Le Ferrand, É. and Sung, L.-M. and Prud'hommeaux, E. and Hartshorne, J. K.},
+  title = {FormosanBank},
+  year = {2024},
+  note = {Electronic Resource},
+  url = {https://ai4commsci.gitbook.io/formosanbank}
+}
+```

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "</s>": 84,
+  "<s>": 83
+}

config.json

@@ -0,0 +1,117 @@
+{
+  "_name_or_path": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_direct_train-all_eval-all-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13/best_model",
+  "activation_dropout": 0.0,
+  "adapter_attn_dim": null,
+  "adapter_kernel_size": 3,
+  "adapter_stride": 2,
+  "add_adapter": false,
+  "apply_spec_augment": true,
+  "architectures": [
+    "Wav2Vec2ForCTC"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "classifier_proj_size": 256,
+  "codevector_dim": 768,
+  "contrastive_logits_temperature": 0.1,
+  "conv_bias": true,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "mean",
+  "ctc_zero_infinity": true,
+  "diversity_loss_weight": 0.1,
+  "do_stable_layer_norm": true,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_dropout": 0.0,
+  "feat_extract_norm": "layer",
+  "feat_proj_dropout": 0.0,
+  "feat_quantizer_dropout": 0.0,
+  "final_dropout": 0.0,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.0,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.0,
+  "mask_channel_length": 10,
+  "mask_channel_min_space": 1,
+  "mask_channel_other": 0.0,
+  "mask_channel_prob": 0.0,
+  "mask_channel_selection": "static",
+  "mask_feature_length": 10,
+  "mask_feature_min_masks": 0,
+  "mask_feature_prob": 0.0,
+  "mask_time_length": 10,
+  "mask_time_min_masks": 2,
+  "mask_time_min_space": 1,
+  "mask_time_other": 0.0,
+  "mask_time_prob": 0.0,
+  "mask_time_selection": "static",
+  "model_type": "wav2vec2",
+  "num_adapter_layers": 3,
+  "num_attention_heads": 16,
+  "num_codevector_groups": 2,
+  "num_codevectors_per_group": 320,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 24,
+  "num_negatives": 100,
+  "output_hidden_size": 1024,
+  "pad_token_id": 82,
+  "proj_codevector_dim": 768,
+  "tdnn_dilation": [
+    1,
+    2,
+    3,
+    1,
+    1
+  ],
+  "tdnn_dim": [
+    512,
+    512,
+    512,
+    512,
+    1500
+  ],
+  "tdnn_kernel": [
+    5,
+    3,
+    3,
+    1,
+    1
+  ],
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.3",
+  "use_weighted_layer_sum": false,
+  "vocab_size": 85,
+  "xvector_output_dim": 512
+}

dev_selection_history.tsv

@@ -0,0 +1,7 @@
+step	train_loss	dev_loss	selection_metric	selection_scope	selection_cer	selection_wer	macro_cell_cer	macro_cell_wer	macro_language_cer	macro_language_wer	micro_cer	micro_wer	dev_cells	dev_languages	dev_rows
+1	0.1070520281791687	0.2688464487162162	decoded_dev_normalized_cer	macro_cell	0.047556	0.447477	0.047556	0.447477	0.051366	0.447071	0.051366	0.447071	4	1	1014
+6057	0.04012543708086014	0.29733945731454947	decoded_dev_normalized_cer	macro_cell	0.038665	0.351145	0.038665	0.351145	0.044238	0.36286	0.044238	0.36286	4	1	1014
+12114	0.024610912427306175	0.32354084358323276	decoded_dev_normalized_cer	macro_cell	0.040227	0.347639	0.040227	0.347639	0.047028	0.369414	0.047028	0.369414	4	1	1014
+18171	0.003530024318024516	0.3870350926469161	decoded_dev_normalized_cer	macro_cell	0.039817	0.361489	0.039817	0.361489	0.045735	0.367428	0.045735	0.367428	4	1	1014
+24228	0.048893608152866364	0.4081318523060149	decoded_dev_normalized_cer	macro_cell	0.039288	0.345583	0.039288	0.345583	0.045633	0.358292	0.045633	0.358292	4	1	1014
+30282	0.007620668970048428	0.4087944376011064	decoded_dev_normalized_cer	macro_cell	0.036756	0.328515	0.036756	0.328515	0.042057	0.341609	0.042057	0.341609	4	1	1014

evaluation_results.json

@@ -0,0 +1,150 @@
+{
+  "aggregate": {
+    "cer_normalized_micro": 0.042242288595611126,
+    "character_errors": 1642,
+    "corpora": 4,
+    "reference_characters": 38871,
+    "reference_words": 4909,
+    "utterances": 1014,
+    "wer_normalized_micro": 0.32185781218170706,
+    "word_errors": 1580
+  },
+  "by_corpus": [
+    {
+      "cer_normalized": "0.059753",
+      "cer_raw": "0.059753",
+      "char_deletions": "370",
+      "char_deletions_raw": "370",
+      "char_insertions": "312",
+      "char_insertions_raw": "312",
+      "char_substitutions": "290",
+      "char_substitutions_raw": "290",
+      "eval_corpus": "ILRDF",
+      "eval_language": "saaroa",
+      "eval_utterances": "414",
+      "reference_characters": "16267",
+      "reference_characters_raw": "16267",
+      "reference_words": "1920",
+      "reference_words_raw": "1920",
+      "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+      "wer_normalized": "0.421875",
+      "wer_raw": "0.421875",
+      "word_deletions": "112",
+      "word_deletions_raw": "112",
+      "word_insertions": "95",
+      "word_insertions_raw": "95",
+      "word_substitutions": "603",
+      "word_substitutions_raw": "603"
+    },
+    {
+      "cer_normalized": "0.02533",
+      "cer_raw": "0.02533",
+      "char_deletions": "27",
+      "char_deletions_raw": "27",
+      "char_insertions": "38",
+      "char_insertions_raw": "38",
+      "char_substitutions": "6",
+      "char_substitutions_raw": "6",
+      "eval_corpus": "ePark1",
+      "eval_language": "saaroa",
+      "eval_utterances": "161",
+      "reference_characters": "2803",
+      "reference_characters_raw": "2803",
+      "reference_words": "350",
+      "reference_words_raw": "350",
+      "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+      "wer_normalized": "0.26",
+      "wer_raw": "0.26",
+      "word_deletions": "9",
+      "word_deletions_raw": "9",
+      "word_insertions": "13",
+      "word_insertions_raw": "13",
+      "word_substitutions": "69",
+      "word_substitutions_raw": "69"
+    },
+    {
+      "cer_normalized": "0.080982",
+      "cer_raw": "0.080982",
+      "char_deletions": "99",
+      "char_deletions_raw": "99",
+      "char_insertions": "100",
+      "char_insertions_raw": "100",
+      "char_substitutions": "88",
+      "char_substitutions_raw": "88",
+      "eval_corpus": "ePark2",
+      "eval_language": "saaroa",
+      "eval_utterances": "94",
+      "reference_characters": "3544",
+      "reference_characters_raw": "3544",
+      "reference_words": "358",
+      "reference_words_raw": "358",
+      "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+      "wer_normalized": "0.337989",
+      "wer_raw": "0.337989",
+      "word_deletions": "15",
+      "word_deletions_raw": "15",
+      "word_insertions": "32",
+      "word_insertions_raw": "32",
+      "word_substitutions": "74",
+      "word_substitutions_raw": "74"
+    },
+    {
+      "cer_normalized": "0.019192",
+      "cer_raw": "0.019192",
+      "char_deletions": "124",
+      "char_deletions_raw": "124",
+      "char_insertions": "126",
+      "char_insertions_raw": "126",
+      "char_substitutions": "62",
+      "char_substitutions_raw": "62",
+      "eval_corpus": "ePark3",
+      "eval_language": "saaroa",
+      "eval_utterances": "345",
+      "reference_characters": "16257",
+      "reference_characters_raw": "16257",
+      "reference_words": "2281",
+      "reference_words_raw": "2281",
+      "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+      "wer_normalized": "0.24463",
+      "wer_raw": "0.24463",
+      "word_deletions": "76",
+      "word_deletions_raw": "76",
+      "word_insertions": "104",
+      "word_insertions_raw": "104",
+      "word_substitutions": "378",
+      "word_substitutions_raw": "378"
+    }
+  ],
+  "condition": "cpt_pan_formosan_to_language",
+  "dataset_version": "dataset_v1",
+  "iso_639_3": "sxr",
+  "language": "Saaroa",
+  "language_id": "saaroa",
+  "model_family": "xlsr53",
+  "normalization_version": "formosan_safe_v1",
+  "readiness": {
+    "actual_train_rows": 8075,
+    "best_selection_cer": 0.03675642331286832,
+    "best_selection_wer": 0.3285151319751156,
+    "best_step": 30282,
+    "condition": "cpt_pan_formosan_to_language",
+    "dominant_hypothesis_rate": 0.0029585798816568047,
+    "estimated_effective_epochs": 30.000743034055727,
+    "failures": 0,
+    "generated_at": "2026-06-30T08:46:18+00:00",
+    "intended_train_rows": 8075,
+    "manifest_dir": "/projects/prudlab/tacl_formosan_asr/experiments/manifests/gate6_xlsr_cpt_downstream_matrix/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "nonempty_hypothesis_rate": 1.0,
+    "prediction_rows": 1014,
+    "result_dir": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "result_label": "final",
+    "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "status": "review",
+    "steps": 30282,
+    "test_rows": 1014,
+    "train_batch_size": 8,
+    "warnings": 1
+  },
+  "seed": 13,
+  "split_version": "split_v2"
+}

metrics_by_corpus.tsv

@@ -0,0 +1,5 @@
+run_id	eval_language	eval_corpus	eval_utterances	wer_raw	cer_raw	wer_normalized	cer_normalized	word_substitutions	word_deletions	word_insertions	reference_words	char_substitutions	char_deletions	char_insertions	reference_characters	word_substitutions_raw	word_deletions_raw	word_insertions_raw	reference_words_raw	char_substitutions_raw	char_deletions_raw	char_insertions_raw	reference_characters_raw
+xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13	saaroa	ILRDF	414	0.421875	0.059753	0.421875	0.059753	603	112	95	1920	290	370	312	16267	603	112	95	1920	290	370	312	16267
+xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13	saaroa	ePark1	161	0.26	0.02533	0.26	0.02533	69	9	13	350	6	27	38	2803	69	9	13	350	6	27	38	2803
+xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13	saaroa	ePark2	94	0.337989	0.080982	0.337989	0.080982	74	15	32	358	88	99	100	3544	74	15	32	358	88	99	100	3544
+xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13	saaroa	ePark3	345	0.24463	0.019192	0.24463	0.019192	378	76	104	2281	62	124	126	16257	378	76	104	2281	62	124	126	16257

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5385ed1fcd2632ff041a5d8ee4cc3338340b6aa64db0cf78750c2dd3938e0463
+size 1262151788

preprocessor_config.json

@@ -0,0 +1,10 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "processor_class": "Wav2Vec2Processor",
+  "return_attention_mask": true,
+  "sampling_rate": 16000
+}

readiness_summary.json

@@ -0,0 +1,23 @@
+{
+  "actual_train_rows": 8075,
+  "best_selection_cer": 0.03675642331286832,
+  "best_selection_wer": 0.3285151319751156,
+  "best_step": 30282,
+  "condition": "cpt_pan_formosan_to_language",
+  "dominant_hypothesis_rate": 0.0029585798816568047,
+  "estimated_effective_epochs": 30.000743034055727,
+  "failures": 0,
+  "generated_at": "2026-06-30T08:46:18+00:00",
+  "intended_train_rows": 8075,
+  "manifest_dir": "/projects/prudlab/tacl_formosan_asr/experiments/manifests/gate6_xlsr_cpt_downstream_matrix/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "nonempty_hypothesis_rate": 1.0,
+  "prediction_rows": 1014,
+  "result_dir": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "result_label": "final",
+  "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "status": "review",
+  "steps": 30282,
+  "test_rows": 1014,
+  "train_batch_size": 8,
+  "warnings": 1
+}

release_manifest.json

@@ -0,0 +1,97 @@
+{
+  "aggregate_metrics": {
+    "cer_normalized_micro": 0.042242288595611126,
+    "character_errors": 1642,
+    "corpora": 4,
+    "reference_characters": 38871,
+    "reference_words": 4909,
+    "utterances": 1014,
+    "wer_normalized_micro": 0.32185781218170706,
+    "word_errors": 1580
+  },
+  "base_model": "facebook/wav2vec2-large-xlsr-53",
+  "condition": "cpt_pan_formosan_to_language",
+  "files": [
+    {
+      "bytes": 50,
+      "name": ".gitattributes",
+      "sha256": "ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550"
+    },
+    {
+      "bytes": 4565,
+      "name": "README.md",
+      "sha256": "9cce0eb27fd753c1978fc8f0070665f39e35a1d956bf5ebff980266f25f83ec7"
+    },
+    {
+      "bytes": 30,
+      "name": "added_tokens.json",
+      "sha256": "e3ac5388f991629a5b855d55cfaa605ad3c2393f9e00949ce129e1db347ec831"
+    },
+    {
+      "bytes": 2493,
+      "name": "config.json",
+      "sha256": "14d520955fe9896a0ede65aee43db0003c1d4247b1b51f9d983e4289fa60a8d5"
+    },
+    {
+      "bytes": 1196,
+      "name": "dev_selection_history.tsv",
+      "sha256": "4b7034efdc302a04269d0c6d062952e06b3c57186be557d53a95e77756479f8d"
+    },
+    {
+      "bytes": 5506,
+      "name": "evaluation_results.json",
+      "sha256": "a56ff92dc3d4864f19adf43e549b97ffbab66ed062ae8529c09c491b39cac413"
+    },
+    {
+      "bytes": 1316,
+      "name": "metrics_by_corpus.tsv",
+      "sha256": "3e48def92a66b6b556c46cfa391f6bf4ccbd5f141a4e1949795281420946e89c"
+    },
+    {
+      "bytes": 1262151788,
+      "name": "model.safetensors",
+      "sha256": "5385ed1fcd2632ff041a5d8ee4cc3338340b6aa64db0cf78750c2dd3938e0463"
+    },
+    {
+      "bytes": 256,
+      "name": "preprocessor_config.json",
+      "sha256": "ffc72f475cd79680d5d659fbaefc691cdbcee5c9703a15faf708a5e91452a8d8"
+    },
+    {
+      "bytes": 1152,
+      "name": "readiness_summary.json",
+      "sha256": "6b9083479df00987fb8ba93fa70e6dcf2f963c8fe043d90a6dc19ff7d56209fd"
+    },
+    {
+      "bytes": 96,
+      "name": "special_tokens_map.json",
+      "sha256": "9046da57c270c8e74d0f38832b4adce269c9d914ef21d2a0925e7772152dd793"
+    },
+    {
+      "bytes": 1096,
+      "name": "tokenizer_config.json",
+      "sha256": "04bfce34f2f4e88dc411ec55c7303fa9eeb7c05e98f70ab8357348323e66866e"
+    },
+    {
+      "bytes": 6760,
+      "name": "training_config.json",
+      "sha256": "fe9381e8ca3fad83ef93401d3fe7179827881ffa39aae7893ea119e228904bc9"
+    },
+    {
+      "bytes": 1464,
+      "name": "training_summary.json",
+      "sha256": "9fe15a9243686979907c4974caeacbceb1ca930521bc4439641d739e1e46fcf7"
+    },
+    {
+      "bytes": 988,
+      "name": "vocab.json",
+      "sha256": "d16944b38ec913130c092037e8b3bd30b864bead6ae2aff78172ec220e658a34"
+    }
+  ],
+  "iso_639_3": "sxr",
+  "language": "Saaroa",
+  "model_family": "xlsr53",
+  "readiness_status": "review",
+  "repo_id": "FormosanBank/formosan-asr-saaroa",
+  "source_run": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13"
+}

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,48 @@
+{
+  "added_tokens_decoder": {
+    "81": {
+      "content": "<unk>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<pad>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "84": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "processor_class": "Wav2Vec2Processor",
+  "replace_word_delimiter_char": " ",
+  "target_lang": null,
+  "tokenizer_class": "Wav2Vec2CTCTokenizer",
+  "unk_token": "<unk>",
+  "word_delimiter_token": "|"
+}

training_config.json

@@ -0,0 +1,192 @@
+{
+  "code_git_commit": "1304b5f9c571a1adb6e5b91accb22c7b1e266dfc",
+  "code_git_dirty": "unknown",
+  "condition": "cpt_pan_formosan_to_language",
+  "cuda_visible_devices": "0",
+  "dataset_version": "dataset_v1",
+  "dev_corpus": "",
+  "dev_hours": 1.793138,
+  "dev_language": "saaroa",
+  "dev_rows": 1014,
+  "dev_scope": "language_pooled",
+  "device": "cuda",
+  "effective_eval_every_steps": 6057,
+  "effective_max_steps": 30282,
+  "eval_batch_size": 8,
+  "eval_corpus": "",
+  "eval_every_steps": 6057,
+  "eval_language": "saaroa",
+  "eval_scope": "language_pooled",
+  "generated_at": "2026-06-30T05:08:54+00:00",
+  "hostname": "g012",
+  "max_steps": 1000,
+  "model_family": "xlsr53",
+  "model_id": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_direct_train-all_eval-all-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13/best_model",
+  "model_revision": "",
+  "normalization_version": "formosan_safe_v1",
+  "package_freeze": [
+    "absl-py==2.4.0",
+    "accelerate==1.1.1",
+    "aiohappyeyeballs==2.6.2",
+    "aiohttp==3.14.1",
+    "aiosignal==1.4.0",
+    "async-timeout==5.0.1",
+    "attrs==26.1.0",
+    "audioread==3.1.0",
+    "certifi==2026.6.17",
+    "cffi==2.0.0",
+    "charset-normalizer==3.4.7",
+    "click==8.4.1",
+    "datasets==3.1.0",
+    "decorator==5.3.1",
+    "dill==0.3.8",
+    "evaluate==0.4.3",
+    "filelock==3.29.0",
+    "frozenlist==1.8.0",
+    "fsspec==2024.9.0",
+    "grpcio==1.81.1",
+    "hf-xet==1.5.1",
+    "huggingface_hub==0.36.2",
+    "idna==3.18",
+    "Jinja2==3.1.6",
+    "jiwer==3.0.5",
+    "joblib==1.5.3",
+    "lazy-loader==0.5",
+    "librosa==0.10.2.post1",
+    "llvmlite==0.47.0",
+    "Markdown==3.10.2",
+    "MarkupSafe==3.0.3",
+    "mpmath==1.3.0",
+    "msgpack==1.2.1",
+    "multidict==6.7.1",
+    "multiprocess==0.70.16",
+    "networkx==3.4.2",
+    "numba==0.65.1",
+    "numpy==2.2.6",
+    "nvidia-cublas-cu12==12.4.5.8",
+    "nvidia-cuda-cupti-cu12==12.4.127",
+    "nvidia-cuda-nvrtc-cu12==12.4.127",
+    "nvidia-cuda-runtime-cu12==12.4.127",
+    "nvidia-cudnn-cu12==9.1.0.70",
+    "nvidia-cufft-cu12==11.2.1.3",
+    "nvidia-curand-cu12==10.3.5.147",
+    "nvidia-cusolver-cu12==11.6.1.9",
+    "nvidia-cusparse-cu12==12.3.1.170",
+    "nvidia-nccl-cu12==2.21.5",
+    "nvidia-nvjitlink-cu12==12.4.127",
+    "nvidia-nvtx-cu12==12.4.127",
+    "packaging @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_packaging_1777103621/work",
+    "pandas==2.3.3",
+    "platformdirs==4.10.0",
+    "pooch==1.9.0",
+    "propcache==0.5.2",
+    "protobuf==7.35.1",
+    "psutil==7.2.2",
+    "pyarrow==24.0.0",
+    "pycparser==3.0",
+    "python-dateutil==2.9.0.post0",
+    "pytz==2026.2",
+    "PyYAML==6.0.3",
+    "RapidFuzz==3.14.5",
+    "regex==2026.5.9",
+    "requests==2.34.2",
+    "safetensors==0.8.0",
+    "scikit-learn==1.7.2",
+    "scipy==1.15.3",
+    "sentencepiece==0.2.0",
+    "six==1.17.0",
+    "soundfile==0.12.1",
+    "soxr==1.1.0",
+    "sympy==1.13.1",
+    "tensorboard==2.18.0",
+    "tensorboard-data-server==0.7.2",
+    "threadpoolctl==3.6.0",
+    "tokenizers==0.20.3",
+    "torch==2.5.1+cu124",
+    "torchaudio==2.5.1+cu124",
+    "tqdm==4.68.3",
+    "transformers==4.46.3",
+    "triton==3.1.0",
+    "typing_extensions==4.15.0",
+    "tzdata==2026.2",
+    "urllib3==2.7.0",
+    "Werkzeug==3.1.8",
+    "xxhash==3.7.0",
+    "yarl==1.24.2"
+  ],
+  "package_freeze_count": 88,
+  "pretraining_checkpoint_path": "/scratch/scheppat/projects/tacl_formosan_asr/runs_cpt/xlsr_cpt_pan_train_only_v1/final_model",
+  "pretraining_condition": "cpt_train_only",
+  "pretraining_manifest": "/projects/prudlab/tacl_formosan_asr/experiments/cpt_manifests/dataset_v1_split_v2_train_only/pan_formosan/train.tsv",
+  "pretraining_model_revision": "c3f9d884181a224a6ac87bf8885c84d1cff3384f",
+  "pretraining_run_id": "xlsr_cpt_pan_train_only_v1",
+  "pretraining_source_policy": "train_only_no_dev_no_test",
+  "python_argv": [
+    "/projects/prudlab/tacl_formosan_asr/code/tacl_pipeline/src/tacl_formosan_asr/train_ctc_asr.py",
+    "--source-root",
+    "/projects/prudlab/hunter_formosan_audio",
+    "--run-dir",
+    "/projects/prudlab/tacl_formosan_asr/experiments/manifests/gate6_xlsr_cpt_downstream_matrix/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "--vocab-json",
+    "/projects/prudlab/tacl_formosan_asr/vocabularies/ctc_dataset_v1_split_v2_formosan_safe_v1/pan_formosan/vocab.json",
+    "--out-dir",
+    "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "--run-id",
+    "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+    "--model-id",
+    "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_direct_train-all_eval-all-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13/best_model",
+    "--model-revision",
+    "",
+    "--seed",
+    "13",
+    "--max-steps",
+    "1000",
+    "--train-epochs",
+    "30",
+    "--train-batch-size",
+    "8",
+    "--eval-batch-size",
+    "8",
+    "--learning-rate",
+    "1e-4",
+    "--eval-every-steps",
+    "0",
+    "--max-eval-batches",
+    "0",
+    "--max-train-rows",
+    "0",
+    "--max-dev-rows",
+    "0",
+    "--max-test-rows",
+    "0",
+    "--save-best-model"
+  ],
+  "python_executable": "/scratch/scheppat/projects/tacl_formosan_asr/conda/tacl_asr_py310/bin/python",
+  "python_version": "3.10.20 | packaged by conda-forge | (main, Jun 11 2026, 03:31:56) [GCC 14.3.0]",
+  "requested_eval_every_steps": 0,
+  "run_dir": "/projects/prudlab/tacl_formosan_asr/experiments/manifests/gate6_xlsr_cpt_downstream_matrix/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "seed": 13,
+  "slurm_job_id": "2647043",
+  "slurm_job_name": "tacl_ctc_registry_array",
+  "split_version": "split_v2",
+  "status": "planned",
+  "tacl_code_version_file": "1304b5f9c571a1adb6e5b91accb22c7b1e266dfc",
+  "test_hours": 1.793222,
+  "test_rows": 1014,
+  "train_batch_size": 8,
+  "train_corpus": "",
+  "train_corpus_counts": {
+    "ILRDF": 3305,
+    "ePark1": 1281,
+    "ePark2": 738,
+    "ePark3": 2751
+  },
+  "train_epochs": 30.0,
+  "train_hours": 14.354056,
+  "train_language": "saaroa",
+  "train_rows": 8075,
+  "train_scope": "language_pooled",
+  "vocab_json": "/projects/prudlab/tacl_formosan_asr/vocabularies/ctc_dataset_v1_split_v2_formosan_safe_v1/pan_formosan/vocab.json",
+  "vocab_scope": "pan_formosan"
+}

training_summary.json

@@ -0,0 +1,31 @@
+{
+  "best_dev_loss": 0.4087944376011064,
+  "best_selection_cer": 0.03675642331286832,
+  "best_selection_metrics": {
+    "dev_cells": 4,
+    "dev_languages": 1,
+    "dev_rows": 1014,
+    "macro_cell_cer": 0.03675642331286832,
+    "macro_cell_wer": 0.3285151319751156,
+    "macro_language_cer": 0.04205666742764376,
+    "macro_language_wer": 0.3416087388282026,
+    "micro_cer": 0.04205666742764376,
+    "micro_wer": 0.3416087388282026,
+    "selection_cer": 0.03675642331286832,
+    "selection_metric": "decoded_dev_normalized_cer",
+    "selection_scope": "macro_cell",
+    "selection_wer": 0.3285151319751156
+  },
+  "best_selection_wer": 0.3285151319751156,
+  "best_step": 30282,
+  "checkpoint_path": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13/best_model",
+  "end_train_loss": 0.007620668970048428,
+  "generated_at": "2026-06-30T08:46:15+00:00",
+  "max_steps": 1000,
+  "out_dir": "/scratch/scheppat/projects/tacl_formosan_asr/runs_xlsr_cpt_downstream_v1/xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "prediction_rows": 1014,
+  "run_id": "xlsr53_cpt_pan_formosan_to_language_train-saaroa_eval-saaroa-all-corpora_dataset_v1_split_v2_formosan_safe_v1_seed13",
+  "start_train_loss": 0.1070520281791687,
+  "steps": 30282,
+  "train_epochs": 30.0
+}

vocab.json

@@ -0,0 +1,85 @@
+{
+  "'": 0,
+  "-": 1,
+  ":": 2,
+  "<pad>": 82,
+  "<unk>": 81,
+  "R": 3,
+  "^": 4,
+  "_": 5,
+  "a": 6,
+  "b": 7,
+  "c": 8,
+  "d": 9,
+  "e": 10,
+  "f": 11,
+  "g": 12,
+  "h": 13,
+  "i": 14,
+  "j": 15,
+  "k": 16,
+  "l": 17,
+  "m": 18,
+  "n": 19,
+  "o": 20,
+  "p": 21,
+  "q": 22,
+  "r": 23,
+  "s": 24,
+  "t": 25,
+  "u": 26,
+  "v": 27,
+  "w": 28,
+  "x": 29,
+  "y": 30,
+  "z": 31,
+  "|": 80,
+  "è": 32,
+  "é": 33,
+  "ê": 34,
+  "í": 35,
+  "ó": 36,
+  "ü": 37,
+  "ā": 38,
+  "ē": 39,
+  "ǔ": 40,
+  "ɨ": 41,
+  "ʉ": 42,
+  "а": 43,
+  "и": 44,
+  "к": 45,
+  "м": 46,
+  "н": 47,
+  "о": 48,
+  "р": 49,
+  "с": 50,
+  "у": 51,
+  "ы": 52,
+  "я": 53,
+  "ṟ": 54,
+  "あ": 55,
+  "う": 56,
+  "お": 57,
+  "か": 58,
+  "き": 59,
+  "じ": 60,
+  "す": 61,
+  "そ": 62,
+  "だ": 63,
+  "ち": 64,
+  "っ": 65,
+  "で": 66,
+  "と": 67,
+  "の": 68,
+  "め": 69,
+  "ゅ": 70,
+  "ょ": 71,
+  "ら": 72,
+  "る": 73,
+  "れ": 74,
+  "ㄅ": 75,
+  "ㄆ": 76,
+  "ㄈ": 77,
+  "ㄛ": 78,
+  "ㆍ": 79
+}