h-gajdov/full_moonshine_tiny_all_data_wup5_ep12_lr5e-5_new_transcripts

full_moonshine_tiny_all_data_wup5_ep12_lr1e-4_new_transcripts

Full fine-tune of UsefulSensors/moonshine-tiny (27.1M parameters) for Macedonian speech-to-text, trained on the same all-data mix as full_moonshine_tiny_all_data_wup1_ep3_lr1e-4 (vezilka-asri plus seven curated sets -- videa_so_transkript_od_yt, alfa_audios, sitel_audios, doniraj (accepted donations), mozzila_common_voice, fleurs_mk and jargon), retrained after vezilka-asri's transcripts were corrected/retranscribed -- same data selection as that run, but the audio-text pairs themselves changed for the ~99% of the mix that vezilka-asri makes up.

Moonshine is a raw-waveform encoder-decoder: it takes input_values rather than Whisper's fixed 30 s log-mel window, so its cost scales with actual audio length instead of being flat per clip. That is what makes it viable on-device, and it is why its real-time factor barely moves with utterance length where Whisper's does.

Evaluation

Held-out test set rachno_provereno_od_yt (44 clips, 0.32 h), cuda, batch 16, 1 beam.

metric	value
WER	11.91
CER	4.85
SER	100.00
Parameters	27.1 M
Latency	118 ms/utterance
Throughput	223x real-time
RTF	0.00448

SER is exact-match over whole utterances. These test clips average 62 words, so at this WER an exactly-correct utterance is vanishingly unlikely -- read WER/CER.

Best WER seen during training (on the same held-out set): 12.02 at step 535000 (epoch 5.65 of the 12 configured -- load_best_model_at_end kept this checkpoint as the final save rather than a later, worse one).

Note: the peak learning rate actually used was 5e-5, not the lr1e-4 its output-dir/repo name carries over from an earlier revision of the config it was launched from -- the Training table below reflects the value actually used.

Compared with the doniraj/prvatv Whisper sweep

This model is far behind every Whisper LoRA/full-finetune adapter in that sweep (WER ~4-5% there vs 11.91% here) -- expected, since Moonshine-tiny is a 27M-parameter model built for on-device use, being compared against a fine-tuned 809M-parameter Whisper Large v3 Turbo.

Training

setting	value
learning rate	5e-05
warmup ratio	0.05
epochs (configured / reached)	12 / 5.65
batch size	32 x 1 accum
scheduler	cosine
weight decay	0.01
precision	bf16
steps at best checkpoint	535000

Usage

from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import librosa, torch

model_id = "h-gajdov/full_moonshine_tiny_all_data_wup5_ep12_lr1e-4_new_transcripts"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id).eval()

wav, _ = librosa.load("clip.wav", sr=16000, mono=True)
feats = processor.feature_extractor([wav], sampling_rate=16000,
                                    return_tensors="pt", padding=True)
with torch.no_grad():
    ids = model.generate(**feats, max_length=192)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

max_length is capped at 192 because the decoder has 194 positions (max_position_embeddings). Moonshine has no forced language/task tokens, so do not pass language= or task= to generate() -- it will raise.