SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Explain ZMOT, also known as Zero Moment of Truth. And then tell me the difference between that & the 3-step model marketing, whatever was prevalent back in the day',
    'The traditional 3-step model of marketing, introduced by P&G, consists of the consumer discovering a product, followed by the first moment of truth (FMOT) where the consumer evaluates the product, and finally the second moment of truth (SMOT) where the consumer purchases and uses the product. ZMOT, or Zero Moment of Truth, refers to the period between the initial discovery and FMOT, where the consumer researches and learns more about the product',
    "The 3-step model of marketing, which was widely used until the introduction of ZMOT, involves the consumer becoming aware of a product's features, then experiencing the first moment of truth (FMOT) at the point of product demonstration, and finally the second moment of truth (SMOT) when they decide to recommend the product to friends. ZMOT, on the other hand, occurs between the initial awareness and FMOT, and involves the consumer seeking out more information about the product's accessories.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 306,636 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 6 tokens mean: 22.27 tokens max: 100 tokens	min: 3 tokens mean: 64.64 tokens max: 256 tokens	min: 4 tokens mean: 36.54 tokens max: 189 tokens

Samples:

anchor	positive	negative
`How is cryogenics used in the study and treatment of viruses in virology?`	`Certain vaccines must be only stored at cryogenic temperature (ultra low temperatures of order -90 to -60 degree celcius / -130 F to -76 F)`	`Certain vaccines must be stored at refrigerated temperatures, ranging from 2 to 8 degrees Celsius, to maintain their potency.`
`I would like to know the domestication status of these animals: cats, dogs, sheep, whales, bats, and bald eagles.`	`Among the listed animals, cats, dogs, and sheep have undergone domestication, while bald eagles, whales, and bats have not.`	`Among the listed animals, cats, dogs, and bats have undergone domestication, while bald eagles, whales, and sheep have not.`
`I need to know who was winner of mens 2022 fifa world cup and where did it happen`	`The men's 2022 FIFA world cup was located in Qatar. Argentina won the world cup, beating France on penalties after drawing 3-3 during the game.`	`France won the men's 2022 FIFA World Cup, which was held in Qatar.`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Evaluation Dataset

Unnamed Dataset

Size: 16,103 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 5 tokens mean: 20.67 tokens max: 107 tokens	min: 3 tokens mean: 64.41 tokens max: 256 tokens	min: 5 tokens mean: 34.83 tokens max: 168 tokens

Samples:

anchor	positive	negative
`I dont know anything about ironman can you tell me how many miles is the triathlon?`	`The total distance of an Ironman triathlon is 140.6 miles, consisting of a 2.4-mile swim, a 112-mile bike ride, and a 26.2-mile run.`	`An Ironman triathlon covers a total distance of around 160 miles, broken down into a 3.8-mile swim, a 120-mile bike segment, and a 36-mile run.`
`what is this meetup group business all about`	`Meetup events are real-life gatherings where members and organizers get together to connect, discuss, and practice activities related to their shared interests.`	`Meetup events are online gatherings where members and organizers connect, discuss, and practice activities related to their shared interests.`
`what exactly is gettysburg college? i need to know about it`	`Gettysburg College is a privately funded liberal arts institution in Gettysburg, Pennsylvania, known for its historical significance in the Battle of Gettysburg during the American Civil War.`	`Gettysburg College is a privately funded university in Gettysburg, Pennsylvania, known for its historical significance in the Battle of Gettysburg during World War I.`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Framework Versions

Python: 3.12.3
Sentence Transformers: 4.1.0
Transformers: 4.52.3
PyTorch: 2.6.0+cu124
Accelerate: 1.7.0
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}