metadata
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
base_model_relation: adapter
library_name: peft
license: apache-2.0
tags:
- lora
- peft
- sft
- finance
- indian-finance
- mixtral
pipeline_tag: text-generation
adaption_indian_finance_dataset
A LoRA adapter fine-tuned on top of mistralai/Mixtral-8x7B-Instruct-v0.1 for the Indian personal-finance / financial-inclusion domain.
It spans a wide range of topics: banking services, digital payments and UPI, savings and investment planning, mutual funds, stocks, fixed-income products, insurance, retirement planning, taxation and government benefit schemes, credit cards, personal and business loans, credit scores, fraud and scam awareness, cybersecurity in financial transactions, regulations, RBI and SEBI guidelines, consumer rights, and India's evolving digital infrastructure.
Model Details
- Base model: mistralai/Mixtral-8x7B-Instruct-v0.1
- Relation to base: LoRA adapter (PEFT)
- Training method: Supervised fine-tuning (SFT)
- Training type: LoRA
- Data format: chat
- Domain: Indian finance / financial inclusion
- DataSet: https://huggingface.co/datasets/Vishykm/adaption-financial-inclusion-dataset-for-india
Training metrics
| Metric | base | adapted |
|---|---|---|
| Win rate (your dataset) | 36 | 64 |
| Win rate (Personal Finance category) | 21 | 80 |
LoRA Configuration
| Parameter | Value |
|---|---|
| lora_r | 64 |
| lora_alpha | 128 |
| lora_dropout | 0 |
| target modules | q_proj, k_proj, v_proj, o_proj |
| trainable modules | all-linear |
| task type | CAUSAL_LM |
Training Hyperparameters
| Parameter | Value |
|---|---|
| n_epochs | 5 |
| batch_size | max |
| learning_rate | 0.0002 |
| lr_scheduler_type | cosine |
| scheduler_num_cycles | 0.5 |
| min_lr_ratio | 0.1 |
| warmup_ratio | 0.03 |
| weight_decay | 0.01 |
| max_grad_norm | 1 |
| train_on_inputs | false |
How to Get Started
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "mistralai/Mixtral-8x7B-Instruct-v0.1"
adapter = "Vishykm/adaption_indian_finance_dataset"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
Job Metadata
- finetune_job_id: d0d3b083-e1f6-4d87-8fe7-bb1934855615
- training_experiment_id: cc84875e-5b62-4a18-b2e1-d0ba9f19922c
- trained_model_name: adaption_indian_finance_dataset
Framework versions
- PEFT 0.15.1