---
language:
- en
- ne
- mai
library_name: peft
pipeline_tag: automatic-speech-recognition
base_model: unsloth/gemma-4-e4b-it-unsloth-bnb-4bit
license: other
tags:
- gemma
- gemma-4
- peft
- lora
- unsloth
- audio
- speech-recognition
- automatic-speech-recognition
- speech-translation
- nepali
- maithili
- english
- devanagari
- multilingual
---

# VERSE V2 — Nepali, Maithili and English Speech Model

VERSE V2 is a multilingual LoRA/PEFT adapter fine-tuned from Gemma 4 E4B for speech transcription and speech-to-text translation across English, Nepali, and Maithili.

This repository contains the final V2 adapter from training step `12761`.

## Model details

- **Developed by:** Aashish Thakuri and Team Northlight
- **Model type:** Multimodal speech-to-text LoRA/PEFT adapter
- **Base model:** `unsloth/gemma-4-e4b-it-unsloth-bnb-4bit`
- **Languages:** English, Nepali, and Maithili
- **Fine-tuning method:** LoRA / PEFT
- **LoRA rank:** 8
- **LoRA alpha:** 16
- **LoRA dropout:** 0
- **Trainable parameters:** 18,350,080
- **Approximate trainable share:** 0.23%
- **Final checkpoint:** `checkpoint-12761`
- **Funding:** Not publicly disclosed
- **License:** This adapter is distributed subject to the applicable Gemma terms and the licenses of the source dataset and dependencies.

## Supported tasks

### Transcription

- English audio → English text
- Nepali audio → Nepali text
- Maithili audio → Maithili text

### Speech-to-text translation

- English audio → Nepali text
- English audio → Maithili text
- Nepali audio → English text
- Nepali audio → Maithili text
- Maithili audio → English text
- Maithili audio → Nepali text

This gives nine total language directions:

```text
en → en, ne, mai
ne → ne, en, mai
mai → mai, en, ne
```

## Training data

The source audio dataset was:

```text
Firoj112/chatterbox-multilingual-data
```

### Clean training source clips

| Language | Clips |
|---|---:|
| English | 21,846 |
| Nepali | 25,184 |
| Maithili | 19,784 |
| **Total** | **66,814** |

Each source clip produced one same-language transcription task and two speech-translation tasks.

### Final V2 multitask data

| Task type | Examples |
|---|---:|
| Transcription | 66,814 |
| Speech translation | 133,628 |
| **Total** | **200,442** |

Validation data:

| Validation item | Count |
|---|---:|
| Source clips | 3,712 |
| Multitask examples | 11,136 |

## Why IndicTrans2 was used

The original dataset contained audio and same-language transcripts, but it did not contain every required cross-language translation target.

IndicTrans2 was used only to generate translated text targets for training:

- `AI4Bharat/indictrans2-en-indic-200M`
- `AI4Bharat/indictrans2-indic-en-200M`
- `AI4Bharat/indictrans2-indic-indic-320M`

It was used for English ↔ Nepali, English ↔ Maithili, and Nepali ↔ Maithili target generation. IndicTrans2 did not generate the source audio and is not used as the final inference model.

## Training setup

- **Cloud platform:** RunPod
- **Project path:** `/workspace/verse-v2`
- **Training method:** LoRA / PEFT
- **Epochs:** 1
- **Per-device batch size:** 16
- **Final optimizer step:** 12,761
- **Final checkpoint path:** `/workspace/verse-v2/gemma4-e4b-verse-v2-multitask/checkpoint-12761`

## Quick validation results

The current results are a small validation estimate using five clips per language.

| Language | Samples | WER | CER |
|---|---:|---:|---:|
| English | 5 | 6.72% | 3.05% |
| Nepali | 5 | 8.20% | 1.57% |
| Maithili | 5 | 30.36% | 6.45% |
| **Combined** | **15** | **12.35%** | **3.44%** |

Lower WER and CER are better. These values are preliminary and should not be treated as a final benchmark or state-of-the-art claim.

## Loading the adapter

```python
from peft import PeftModel
from unsloth import FastModel

BASE_MODEL = "unsloth/gemma-4-e4b-it-unsloth-bnb-4bit"
ADAPTER_REPO = "Aashishhhhhhhh/verse-v2-nepali-maithili"

base_model, processor = FastModel.from_pretrained(
    model_name=BASE_MODEL,
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_REPO,
    is_trainable=False,
)

model.eval()
```

## Intended uses

VERSE V2 is intended for research and development involving live captions, subtitle generation, podcast transcription, multilingual speech accessibility, Nepali and Maithili speech recognition, and speech-to-text translation.

## Limitations

- Maithili transcription currently has higher WER than English and Nepali.
- The reported validation sample contains only 15 clips.
- Translation targets partly depend on IndicTrans2-generated labels.
- Noise, overlapping speakers, music, long audio, rare dialects, and code-switching may reduce accuracy.
- The adapter requires the compatible Gemma 4 E4B base model.
- Independent evaluation is recommended before production or high-stakes use.

## Ethical considerations

This model may produce incorrect transcriptions or translations. Outputs should be reviewed before use in legal, medical, financial, safety-critical, or other high-stakes settings. Users are responsible for respecting privacy, consent, copyright, dataset terms, and applicable laws when processing audio.

## Model sources

- **Base model:** `unsloth/gemma-4-e4b-it-unsloth-bnb-4bit`
- **Dataset:** `Firoj112/chatterbox-multilingual-data`
- **Project code:** `https://github.com/AashishThakuri/-Gemma_Margadarshan_hackathon_Team_Northlight`

## Attribution

VERSE V2 was built using Gemma 4 by Google DeepMind, Unsloth, Hugging Face Transformers and PEFT, IndicTrans2 by AI4Bharat, RunPod cloud infrastructure, and `Firoj112/chatterbox-multilingual-data`.

## Citation

A formal paper citation is not available yet. Until one is published, cite the model repository:

```text
Aashish Thakuri and Team Northlight.
VERSE V2: Nepali, Maithili and English Speech Model.
Hugging Face, 2026.
https://huggingface.co/Aashishhhhhhhh/verse-v2-nepali-maithili
```

## Disclaimer

This repository contains a LoRA/PEFT adapter, not the full merged Gemma base model. To use VERSE V2, load the compatible Gemma 4 E4B base model and then attach this adapter.