File size: 2,920 Bytes
f59bc2d f852851 789d5b2 f852851 f59bc2d f852851 f59bc2d f852851 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
---
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
base_model_relation: adapter
library_name: peft
license: apache-2.0
tags:
- lora
- peft
- sft
- finance
- indian-finance
- mixtral
pipeline_tag: text-generation
---
# adaption_indian_finance_dataset
A LoRA adapter fine-tuned on top of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) for the Indian personal-finance / financial-inclusion domain.
It spans a wide range of topics: banking services, digital payments and UPI, savings and investment planning, mutual funds, stocks, fixed-income products, insurance, retirement planning, taxation and government benefit schemes, credit cards, personal and business loans, credit scores, fraud and scam awareness, cybersecurity in financial transactions, regulations, RBI and SEBI guidelines, consumer rights, and India's evolving digital infrastructure.
## Model Details
- **Base model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
- **Relation to base:** LoRA adapter (PEFT)
- **Training method:** Supervised fine-tuning (SFT)
- **Training type:** LoRA
- **Data format:** chat
- **Domain:** Indian finance / financial inclusion
- **DataSet:** https://huggingface.co/datasets/Vishykm/adaption-financial-inclusion-dataset-for-india
### Training metrics
| Metric | base | adapted |
|---|---|---|
| Win rate (your dataset) | 36 | 64 |
| Win rate (Personal Finance category) | 21 | 80 |
## LoRA Configuration
| Parameter | Value |
|---|---|
| lora_r | 64 |
| lora_alpha | 128 |
| lora_dropout | 0 |
| target modules | q_proj, k_proj, v_proj, o_proj |
| trainable modules | all-linear |
| task type | CAUSAL_LM |
## Training Hyperparameters
| Parameter | Value |
|---|---|
| n_epochs | 5 |
| batch_size | max |
| learning_rate | 0.0002 |
| lr_scheduler_type | cosine |
| scheduler_num_cycles | 0.5 |
| min_lr_ratio | 0.1 |
| warmup_ratio | 0.03 |
| weight_decay | 0.01 |
| max_grad_norm | 1 |
| train_on_inputs | false |
## How to Get Started
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "mistralai/Mixtral-8x7B-Instruct-v0.1"
adapter = "Vishykm/adaption_indian_finance_dataset"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
```
## Job Metadata
- **finetune_job_id:** d0d3b083-e1f6-4d87-8fe7-bb1934855615
- **training_experiment_id:** cc84875e-5b62-4a18-b2e1-d0ba9f19922c
- **trained_model_name:** adaption_indian_finance_dataset
### Framework versions
- PEFT 0.15.1
</content>
</invoke>
|