Hugging Face's logo

AXERA-TECH
/
jina-embeddings-v5-omni-nano

Feature Extraction
Transformers
multilingual
axera
ax650
embedding
multimodal
vision
audio
lora
Model card Files
xet
 Community

jina-embeddings-v5-omni-nano on AXERA NPU

Ready-to-run deployment package for jinaai/jina-embeddings-v5-omni-nano on AX650 / NPU3.

  • Runtime: bundled axllm service with an OpenAI-compatible /v1/embeddings API
  • Inputs: text and images through the packaged API and demo; fixed-profile image and audio encoder models are included
  • Output: one L2-normalized 768-dimensional embedding per request
  • Tasks: retrieval, clustering, classification, and text matching, selected at runtime without loading four complete text backbones
  • Included assets: compiled .axmodel files, tokenizer data, packed task adapters, sample images, validation results, and a browser demo

Supported Platform

  • AX650 / NPU3
  • Single NPU

Performance

All measurements below were taken on one AX650 NPU with the files in this package. Standalone .axmodel latency uses 3 warm-up runs followed by 20 measured runs. API latency includes tokenization, adapter selection, model execution, and local HTTP overhead.

End-to-End Text Embedding and Task Switching

Scenario Average time
Same-task short-text embedding request 53.625 ms
Alternating retrieval / clustering requests 97.496 ms
Internal switch between loaded adapters 41.100 ms

The model encodes each input in one forward pass. It has a prefill stage but no autoregressive decode stage.

Standalone Module Latency

Module Fixed input profile Average time
Text backbone, sum of 12 layer models 128 tokens, shape group 1 28.177 ms
Embedding post-processing [1, 768] output 5.601 ms
Vision encoder 256×256 image 15.838 ms
Vision merger, retrieval 64 visual tokens 0.671 ms
Vision merger, clustering 64 visual tokens 0.671 ms
Audio encoder 8-second profile 211.082 ms
Audio projector, retrieval task-specific 0.239 ms
Audio projector, clustering task-specific 0.242 ms

Standalone results were measured with ax_run_model --warmup=3 --repeat=20; the text layer models additionally use --group=1. The recorded averages are available in validation/ax_run_model_average.json. Task-switch samples are available in validation/task_switch_client_latency.json and validation/task_switch_internal_latency.json.

Runtime Memory

Measured runtime state CMM used OS memory used
axllm initialized and retrieval/clustering smoke test completed 471.148 MiB 85.392 MiB PSS

CMM used is the increase in allocated CMM between the stopped-service baseline and the measured running state on the same board. PSS is the proportional set size reported for the axllm process after the smoke test. These are consumed amounts and do not depend on the board's installed CMM capacity.

The audio encoder is initialized lazily and was not loaded in this text smoke-test state. The measurement record is available in validation/runtime_memory.json.

Package Size

Packaged component Size
Shared text backbone, embedding table, and post model 430.6 MiB
Four packed LoRA task adapters 51.8 MiB total
Vision models 116.5 MiB
Audio models 671.7 MiB
Complete runtime package 1.27 GiB

The four tasks share the same text backbone. Package sizes describe files on storage and are independent of the runtime CMM and OS-memory measurements above.

Functional Validation

Check Result
Retrieval relevant cosine 0.761431
Retrieval irrelevant cosine 0.031666
Clustering within-topic cosine 0.759586
Clustering cross-topic cosine 0.030956
Retrieval round-trip max absolute difference 0.0
Complete web-demo checks 5 / 5 PASS

Machine-readable results are stored under validation/.

Package Layout 

.
├── README.md
├── config.json
├── start_axllm.sh
├── bin/axllm
├── jina_v5_omni_tokenizer.txt
├── tokenizer.json
├── tokenizer_config.json
├── model.embed_tokens.weight.bfloat16.bin
├── jina_embeddings_v5_omni_p128_l0_together.axmodel
├── ...
├── jina_embeddings_v5_omni_p128_l11_together.axmodel
├── jina_embeddings_v5_omni_post.axmodel
├── jina_v5_omni_nano_vision_tower_256x256.axmodel
├── jina_v5_omni_nano_vision_merger_<retrieval|clustering>_256x256.axmodel
├── jina_v5_omni_nano_audio_tower_8s.axmodel
├── jina_v5_omni_nano_audio_projector_<retrieval|clustering>_8s.axmodel
├── lora/{retrieval,clustering,classification,text-matching}/
├── scripts/test_retrieval_clustering.py
├── demo/
├── assets/
├── validation/
└── SHA256SUMS

The root directory is the axllm model directory. Keep the relative paths unchanged because config.json resolves the text, media, and adapter files from this layout.

Download the Package 

mkdir -p AXERA-TECH/jina-embeddings-v5-omni-nano
cd AXERA-TECH/jina-embeddings-v5-omni-nano
hf download AXERA-TECH/jina-embeddings-v5-omni-nano --local-dir .

Copy the downloaded directory to the AX650 board before continuing.

Run on the Board

From the package root, start the bundled service on port 18201:

bash ./start_axllm.sh 18201

In another terminal, verify the service and model id:

curl http://127.0.0.1:18201/health
curl http://127.0.0.1:18201/v1/models

A healthy service returns:

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

The model list contains AXERA-TECH/jina-embeddings-v5-omni-nano.

Select a Task Adapter

Set task_id in each embedding request:

task_id Intended use Packaged task-specific modules Input role
retrieval Cross-modal search, semantic retrieval, and RAG text, vision, audio query for queries; document for indexed items
clustering Topic discovery, grouping, and deduplication text, vision, audio document
classification Features for zero-shot, few-shot, or downstream classifiers text document
text-matching Semantic similarity, paraphrase, and near-duplicate matching text document

Adapter files are validated during model initialization. A task switch then loads the selected matrices without repeating the full file validation.

OpenAI-Compatible Text Request 

curl http://127.0.0.1:18201/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/jina-embeddings-v5-omni-nano",
    "task_id": "retrieval",
    "input_type": "query",
    "input": "a photo of a cat"
  }'

The embedding is returned in data[0].embedding. It contains 768 floating-point values with an L2 norm of approximately 1.0. For retrieval, use input_type: query for the search query and input_type: document for indexed text, images, or audio.

The packaged smoke test verifies retrieval ranking, clustering separation, and a bit-exact retrieval -> clustering -> retrieval round trip:

python3 scripts/test_retrieval_clustering.py \
  --url http://127.0.0.1:18201/v1/embeddings

Browser Demo

Keep axllm running and start the packaged web application in another terminal:

python3 demo/openai_task_switch_web.py \
  --axllm-url http://127.0.0.1:18201/v1/embeddings \
  --host 0.0.0.0 \
  --port 8080

The web process checks the embedding service before it starts. Open http://<board-ip>:8080 to search the 12 packaged animal images with natural language. The first search creates an in-memory image index; later searches reuse the cached image embeddings and compute only the new text embedding.

Open http://<board-ip>:8080/index.html for the complete capability validation. It exercises all four tasks and then checks a bit-exact switch back to retrieval.

For the query 找出所有猫的图像, all three cat images are ranked first:

cat_0.jpeg
1. cat_0.jpeg
cosine 0.405399		 cat_2.jpg
2. cat_2.jpg
cosine 0.403438		 cat_1.jpeg
3. cat_1.jpeg
cosine 0.344883

The complete log below was captured from the packaged demo on AX650.

Complete AX650 web-demo log 
Jina Embeddings v5 Omni - OpenAI API 多任务切换 Demo
Model: AXERA-TECH/jina-embeddings-v5-omni-nano
Assets: cat=3, dog=3, fox=3, rabbit=3

--- retrieval：文本查询召回动物图片 ---

=== task switch: <startup> -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=   57.00 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  316.37 ms dim=768 norm=1.000000
[request] task=retrieval     input=image:cat_1.jpeg   latency=  369.30 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_2.jpg    latency=  253.63 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_0.jpeg   latency=  143.66 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_1.jpg    latency=  165.81 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_2.jpeg   latency=  586.12 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:fox_0.jpeg   latency=  164.39 ms dim=768 norm=1.000000
[request] task=retrieval     input=image:fox_1.jpeg   latency=  117.97 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:fox_2.jpeg   latency=  129.73 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_0.jpeg latency=  122.36 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_1.jpeg latency=  105.08 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_2.jpeg latency=  105.57 ms dim=768 norm=1.000000
[result] retrieval ranking:
   1. cat_2.jpg         cosine=0.481803
   2. cat_0.jpeg        cosine=0.471761
   3. cat_1.jpeg        cosine=0.437954
   4. dog_2.jpeg        cosine=0.334749
   5. dog_0.jpeg        cosine=0.327777
   6. dog_1.jpg         cosine=0.304540
   7. rabbit_1.jpeg     cosine=0.271320
   8. rabbit_0.jpeg     cosine=0.255625
   9. fox_0.jpeg        cosine=0.238203
  10. fox_2.jpeg        cosine=0.232027
  11. rabbit_2.jpeg     cosine=0.225383
  12. fox_1.jpeg        cosine=0.200564
[check] cat top-1: PASS

--- clustering：同类与跨类图片聚类 ---

=== task switch: retrieval -> clustering ===
[request] task=clustering    input=image:cat_0.jpeg   latency=  553.46 ms dim=768 norm=1.000001
[request] task=clustering    input=image:cat_1.jpeg   latency=  371.34 ms dim=768 norm=1.000001
[request] task=clustering    input=image:cat_2.jpg    latency=  256.51 ms dim=768 norm=1.000000
[request] task=clustering    input=image:dog_0.jpeg   latency=  143.50 ms dim=768 norm=1.000000
[request] task=clustering    input=image:dog_1.jpg    latency=  165.48 ms dim=768 norm=1.000001
[request] task=clustering    input=image:dog_2.jpeg   latency=  588.71 ms dim=768 norm=1.000000
[request] task=clustering    input=image:fox_0.jpeg   latency=  163.32 ms dim=768 norm=1.000001
[request] task=clustering    input=image:fox_1.jpeg   latency=  117.79 ms dim=768 norm=1.000000
[request] task=clustering    input=image:fox_2.jpeg   latency=  129.84 ms dim=768 norm=1.000000
[request] task=clustering    input=image:rabbit_0.jpeg latency=  122.27 ms dim=768 norm=1.000001
[request] task=clustering    input=image:rabbit_1.jpeg latency=  104.95 ms dim=768 norm=1.000001
[request] task=clustering    input=image:rabbit_2.jpeg latency=  105.37 ms dim=768 norm=1.000001
[result] within-class average cosine=0.918765
[result] cross-class  average cosine=0.792831
[check] clustering separation: PASS

--- classification：动物文本分类特征分离 ---

=== task switch: clustering -> classification ===
[request] task=classification input=text:A cat sleeps  latency=  271.33 ms dim=768 norm=1.000001
[request] task=classification input=text:A kitten res  latency=   56.57 ms dim=768 norm=1.000001
[request] task=classification input=text:A dog runs i  latency=   61.85 ms dim=768 norm=1.000001
[request] task=classification input=text:A puppy play  latency=   56.83 ms dim=768 norm=1.000001
[request] task=classification input=text:A red fox wa  latency=   68.65 ms dim=768 norm=1.000001
[request] task=classification input=text:A wild fox h  latency=   68.58 ms dim=768 norm=1.000001
[request] task=classification input=text:A rabbit eat  latency=   62.60 ms dim=768 norm=1.000001
[request] task=classification input=text:A bunny sits  latency=   63.04 ms dim=768 norm=1.000001
[result] cat    same-class cosine=0.814281
[result] dog    same-class cosine=0.820817
[result] fox    same-class cosine=0.828818
[result] rabbit same-class cosine=0.852505
[result] same-class average cosine=0.829105
[result] cross-class average cosine=0.776887
[check] classification separation: PASS

--- text-matching：动物文本匹配 ---

=== task switch: classification -> text-matching ===
[request] task=text-matching input=text:A red fox st  latency=  272.79 ms dim=768 norm=1.000000
[request] task=text-matching input=text:A domestic c  latency=   62.90 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A pet dog ru  latency=   59.69 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A wild red f  latency=   56.68 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A small rabb  latency=   57.44 ms dim=768 norm=1.000001
[result] text-matching ranking:
  1. fox    cosine=0.900007
  2. rabbit cosine=0.600154
  3. dog    cosine=0.518257
  4. cat    cosine=0.441050
[check] fox top-1: PASS

--- round-trip：切回 retrieval ---

=== task switch: text-matching -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=   97.80 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  306.87 ms dim=768 norm=1.000000
[result] retrieval query max_abs_diff=0.0000000000
[result] retrieval image max_abs_diff=0.0000000000
[check] retrieval round-trip: PASS

=== summary ===
retrieval       PASS
clustering      PASS
classification  PASS
text-matching   PASS
round-trip      PASS

Fixed Input Profiles

The final embedding output is always [1, 768] and L2 normalized.

Input Compiled profile
Text 128-token processing chunks; 2047-token runtime limit
Image 256×256
Audio fixed 8-second profile

The vision and audio task-specific mapper files are selected together with the active adapter. Image resolutions or audio lengths outside these profiles require rebuilding the corresponding media modules.

Adapter selection is mutable state within one service instance. Serialize requests that change task_id, or run separate service instances when requests must be isolated by task. The first switch may include filesystem cache overhead; use the warm measurements above for steady-state planning.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

The public conversion workflow requires Pulsar2 7.0 or later. Pulsar2 is needed only to rebuild the deployment artifacts, not to run this downloaded package.

Discussion

Downloads last month
9
Inference Providers NEW
Feature Extraction
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AXERA-TECH/jina-embeddings-v5-omni-nano

Base model

jinaai/jina-embeddings-v5-omni-nano
Adapter
(1)
this model