zeromodels/distiluse-base-multilingual-cased-v2

Run sentence embeddings with Keras 3

zeromodels/distiluse-base-multilingual-cased-v2

Pure-Keras 3 conversion of sentence-transformers/distiluse-base-multilingual-cased-v2 for ZeroModels. It encodes text into 512-dimensional vectors using a distilbert backbone. Its pooling is mean, its embeddings are not normalized, and the intended scoring function is cosine similarity. The conversion preserves the source model's embedding recipe; it does not introduce a new training run.

For details about training and intended use, see the original model card.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # set before importing Keras / ZeroModels

from keras import ops
from zeromodels.sentence_transformers import (
    ZMSentenceTransformer,
    ZMSentenceTransformerTokenizer,
)

repo = "zeromodels/distiluse-base-multilingual-cased-v2"
model = ZMSentenceTransformer.from_weights(repo)
tokenizer = ZMSentenceTransformerTokenizer.from_weights(repo)

texts = ["A person is reading a book.", "Someone reads a novel."]
embeddings = model.encode(texts, tokenizer=tokenizer)
scores = ops.convert_to_numpy(model.similarity(embeddings, embeddings))
print(embeddings.shape)  # (2, 512)
print(scores)

model.similarity() uses cosine similarity, matching the source model. If you need unit-length vectors for a checkpoint that is not normalized, pass normalize=True to model.encode(...); this changes the original embedding recipe and may change retrieval scores.

Model details

Property	Value
Backbone	distilbert
Pooling	mean
Dense projection	512-dimensional Dense projection (tanh)
Embedding dimension	512
L2 normalization	no
Scoring	cosine similarity
Maximum input length	128 tokens
Original checkpoint	sentence-transformers/distiluse-base-multilingual-cased-v2
Original revision	bfe45d0732ca50787611c0fe107ba278c7f3f889

All 13 converted variants use from_weights("zeromodels/<variant>"):

Variant	Encoder and recipe
all-mpnet-base-v2	MPNet · mean + normalize
multi-qa-mpnet-base-dot-v1	MPNet · CLS + dot product
all-distilroberta-v1	RoBERTa · mean + normalize
all-MiniLM-L12-v2	BERT · mean + normalize
multi-qa-distilbert-cos-v1	DistilBERT · mean + normalize
all-MiniLM-L6-v2	BERT · mean + normalize
multi-qa-MiniLM-L6-cos-v1	BERT · mean + normalize
paraphrase-multilingual-mpnet-base-v2	XLM-R · mean
paraphrase-albert-small-v2	ALBERT · mean
paraphrase-multilingual-MiniLM-L12-v2	BERT · mean
paraphrase-MiniLM-L3-v2	BERT · mean
distiluse-base-multilingual-cased-v1	DistilBERT · mean + Dense
distiluse-base-multilingual-cased-v2	DistilBERT · mean + Dense

Tips

• Set KERAS_BACKEND before importing Keras or ZeroModels. Embedding parity was checked on the Torch backend; full TensorFlow/JAX parity is not claimed.
• Use the tokenizer from the same repo as the model. It preserves the source truncation length of 128 tokens for this checkpoint.
• For exact source loading without these converted Keras weights, use ZMSentenceTransformer.from_weights("hf:sentence-transformers/distiluse-base-multilingual-cased-v2").
• The repository contains model.weights.h5, zm_config.json, and tokenizer.json. The pinned source commit is listed above.

Verification and attribution

The conversion was compared with Sentence Transformers at the pinned source revision. Maximum embedding absolute difference: 0.000112148; minimum embedding cosine: 0.99999982; maximum similarity-score absolute difference: 5.72074e-05.

Special thanks to the Sentence Transformers team for the original checkpoint. The upstream card declares apache-2.0.

ZeroModels documentation · Source model · This checkpoint