Upload fine-tuned DistilBERT sentiment classifier (Run-2)

README.md

@@ -0,0 +1,134 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- text-classification
+- sentiment-analysis
+- distilbert
+- pytorch
+- transformers
+- cardiffnlp
+- twitter
+datasets:
+- cardiffnlp/tweet_sentiment_multilingual
+metrics:
+- accuracy
+- f1
+pipeline_tag: text-classification
+model-index:
+- name: ZyroGod/distilbert-sentiment-classifier
+  results:
+  - task:
+      type: text-classification
+      name: Sentiment Analysis
+    dataset:
+      name: cardiffnlp/tweet_sentiment_multilingual (English)
+      type: cardiffnlp/tweet_sentiment_multilingual
+      args: english
+    metrics:
+    - name: Test Accuracy
+      type: accuracy
+      value: 0.6690
+    - name: Test Macro F1
+      type: f1
+      value: 0.6651
+    - name: Test Weighted F1
+      type: f1
+      value: 0.6651
+---
+
+# DistilBERT Sentiment Classifier (Run-2 Champion)
+
+Fine-tuned [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) for 3-class sentiment classification (`negative`, `neutral`, `positive`) on the English subset of the [cardiffnlp/tweet_sentiment_multilingual](https://huggingface.co/datasets/cardiffnlp/tweet_sentiment_multilingual) dataset.
+
+Trained with hardware optimizations designed for a **4GB VRAM NVIDIA GeForce RTX 3050 Laptop GPU**, including FP16 Tensor Core mixed precision and CUDA-fused AdamW (`adamw_torch_fused`).
+
+---
+
+## Benchmark Results (Test Set: 870 Unseen Samples)
+
+| Metric | Score | Notes |
+| :--- | :--- | :--- |
+| **Accuracy** | **66.90%** | +1.04% over 3-epoch baseline |
+| **Macro F1** | **66.51%** | Balanced across all 3 classes |
+| **Weighted F1** | **66.51%** | Metric used for checkpoint selection |
+| **Negative F1 / Recall** | **72.98% / 82.41%** | Robust detection of complaints & negative sentiment |
+| **Positive F1 / Precision** | **73.70% / 79.60%** | High precision when flagging positive sentiment |
+| **Neutral F1 / Recall** | **52.84% / 49.66%** | Weakest class; neutral is often confused with its neighbors |
+| **Severe Polarity Inversion** | **3.56%** | Low confusion between opposing sentiments (31/870 samples) |
+
+
+### Confusion Matrix
+![Confusion Matrix](confusion_matrix.png)
+
+---
+
+## Training Configuration & Hyperparameters
+
+- **Base Architecture:** `distilbert-base-uncased` (66.95M parameters)
+- **Epochs:** 10 (Champion run with best model selection)
+- **Learning Rate:** 2.0e-5 (Linear warmup & decay)
+- **Per-Device Batch Size:** 16
+- **Gradient Accumulation Steps:** 2 (Effective Batch Size = 32)
+- **Precision:** Mixed Precision FP16 (AMP)
+- **Optimizer:** `adamw_torch_fused` (Weight Decay = 0.01)
+- **Hardware Footprint:** Peak reserved VRAM of 1,456.0 MB (~1.42 GB / 36.4% utilization)
+
+---
+
+## Quickstart & Usage
+
+### Option 1: Hugging Face Pipeline (Simplest)
+
+```python
+from transformers import pipeline
+
+# Load pipeline directly from Hugging Face Hub
+classifier = pipeline("sentiment-analysis", model="ZyroGod/distilbert-sentiment-classifier")
+
+# Run inference
+sample_text = "I absolutely love this product! Best purchase ever!"
+prediction = classifier(sample_text)
+print(prediction)
+# Output: [{'label': 'positive', 'score': 0.7588}]
+```
+
+### Option 2: PyTorch AutoModel & AutoTokenizer
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+model_name = "ZyroGod/distilbert-sentiment-classifier"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+model.eval()
+
+text = "Amazing customer service and quick resolution!"
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
+
+with torch.no_grad():
+    outputs = model(**inputs)
+    probs = torch.softmax(outputs.logits, dim=-1)[0]
+
+id2label = model.config.id2label
+for idx, prob in enumerate(probs):
+    print(f"{id2label[idx]:<10s}: {prob.item():.2%}")
+```
+
+---
+
+## Label Mapping
+
+| Label ID | Sentiment Class |
+| :--- | :--- |
+| `0` | `negative` |
+| `1` | `neutral` |
+| `2` | `positive` |
+
+---
+
+## License
+
+This model is open-sourced under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).

config.json

@@ -0,0 +1,37 @@
+{
+  "activation": "gelu",
+  "architectures": [
+    "DistilBertForSequenceClassification"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": null,
+  "dim": 768,
+  "dropout": 0.1,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_dim": 3072,
+  "id2label": {
+    "0": "negative",
+    "1": "neutral",
+    "2": "positive"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "negative": 0,
+    "neutral": 1,
+    "positive": 2
+  },
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "pad_token_id": 0,
+  "problem_type": "single_label_classification",
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": false,
+  "tie_weights_": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e47cecf72866c174d5f1104b6b85a0e1c5f0e24523ab787d3d97b13c3521e113
+size 267835644

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 128,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}