Vandita/Bert-finetuned-Sarc

EmoCentricSarcBERT

EmoCentricSarcBERT is a fine-tuned version of bert-base-cased on the SarcOji dataset. It achieves the following results on the Validation set (20% of stratified split of SarcOji dataset):

• Loss: 2.1366
• Accuracy: 0.7126
• Precision: 0.4491
• Recall: 0.6635
• F1: 0.5356
• Mcc: 0.3521
• Roc Auc: 0.7425

Model description

The model uses text and emojis within text to classify sarcasm. The model first featured in:

@article{grover2026emoji,
  title     = {An emoji centric approach to sarcasm detection in online discourse},
  author    = {Grover, V and Banati, H},
  journal   = {Scientific Reports},
  volume    = {16},
  number    = {1},
  pages     = {3891},
  year      = {2026},
  publisher = {Nature Publishing Group UK London}
}

Intended uses & limitations

While this model is effective for sarcasm classification on PlainText data since it has been fine-tuned on SarcOji (a sarcasm dataset). Being emoji-centric it relies on emojis for an enhanced sarcasm classification. At present it can tokenize 1444 emojis.

Training and evaluation data

Training and Validation set: SarcOji

@inproceedings{grover2022understanding,
  title={Understanding the sarcastic nature of emojis with SarcOji},
  author={Grover, Vandita and Banati, Hema},
  booktitle={Proceedings of the Fifth International Workshop on Emoji Understanding and Applications in Social Media},
  pages={29--39},
  year={2022}
}

SarcOji dataset is available on: https://github.com/VanditaGroverKapila/SarcOji

Test Sets SarcOjiTest1 and SarcOjiTest2

@article{grover2024attention,
  title={An attention approach to emoji focused sarcasm detection},
  author={Grover, Vandita and Banati, Hema},
  journal={Heliyon},
  volume={10},
  number={17},
  year={2024},
  publisher={Elsevier}
}

The tests are available on : https://github.com/VanditaGroverKapila/SarcOjiTestSets

Training procedure

For training procedure refer to the thesis:

@phdthesis{grover2026emojis,
  author    = {Grover, V.},
  title     = {Emojis as Affective Signals for Sarcasm Detection: An Empirical Analysis Informing EmoCentricSarcBERT and Its Application in the RADMAD Framework for Toxicity Mitigation in Online Discourse},
  publisher  = {Zenodo},
  year      = {2026},
  type      = {Thesis},
  doi       = {10.5281/zenodo.21297666},
  
}

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• distributed_type: tpu
• optimizer: Use OptimizerNames.ADAMW_TORCH_XLA with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	Mcc	Roc Auc
0.3984	1.0	735	0.3041	0.8507	0.8871	0.7070	0.7869	0.6846	0.9440
0.2789	2.0	1470	0.2903	0.8719	0.8810	0.7760	0.8252	0.7283	0.9522
0.1630	3.0	2205	0.3008	0.8860	0.8689	0.8332	0.8506	0.7589	0.9560
0.1275	4.0	2940	0.4138	0.8821	0.8730	0.8162	0.8436	0.7502	0.9538
0.0657	5.0	3675	0.5139	0.8807	0.8897	0.7921	0.8381	0.7473	0.9548
0.0555	6.0	4410	0.6063	0.8822	0.8864	0.8004	0.8412	0.7505	0.9525
0.0338	7.0	5145	0.6884	0.8878	0.8585	0.8528	0.8556	0.7640	0.9538
0.0256	8.0	5880	0.7241	0.8877	0.8581	0.8528	0.8555	0.7636	0.9551
0.0199	9.0	6615	0.7579	0.8892	0.8510	0.8677	0.8592	0.7680	0.9548
0.0148	10.0	7350	0.7697	0.8851	0.8543	0.8502	0.8523	0.7583	0.9546

Framework versions

• Transformers 5.0.0
• Pytorch 2.9.0+cpu
• Datasets 4.5.0
• Tokenizers 0.22.2