EmbeddingGemma 2 (Apple Silicon)

Original model: google/embeddinggemma-2 (Google)

On-device multimodal embeddings compressed and packaged by TheStage AI for Apple Silicon (CoreML / Neural Engine). Text, images, audio, video, and mixed inputs (an image with its caption) map into one 768-d, L2-normalized space, so a typed query can find a paragraph, a photo, a sound or a clip. Matryoshka-trained: the first 512, 256 or 128 values are themselves embeddings.


Runtime	TheStage Apple SDK (`EmbeddingPipeline`)
Embedding dim	768 (L2-normalized); Matryoshka 512 / 256 / 128
Inputs	text, image, audio, video, mixed
Default device	NPU (Neural Engine)
SDK	1.7.0+
HF engines	`TheStageAI/embeddinggemma-2` (tag `v1.7`)

Overview

This repo hosts CoreML engine bundles for the TheStage Apple SDK. Point engines_path at the repo id; the SDK downloads, caches, and loads the engines automatically. Task prompts, token limits and Matryoshka sizes all come from the bundle. Open only the inputs you use (modalities:) to shorten the first load.

System Requirements

Property	Value
Hardware	Apple Silicon Mac, or physical iPhone / iPad
macOS	15.0+
iOS	18.0+
Xcode	16.0+
Swift	6.0+
Flutter (optional)	3.24+

Simulator is not supported — run on real Apple Silicon hardware.

TheStage AI Access Token

Create a token at app.thestage.ai and pass it to the SDK:

try await TheStageAI.shared.initialize(api_token: "th_…")

await TheStageFlutterSDK.initialize(api_token: 'th_…');

Token is checked online in initialize (once per app process when reachable). Inference runs fully on-device. Offline initialize fails — reconnect and call initialize again.

TheStage Apple SDK

Docs: TheStage Apple SDK · Embeddings · Repo: TheStage Apple SDK

Installation (SwiftPM)

In Xcode: File → Add Package Dependencies…, paste https://github.com/TheStageAI/AppleSDK.git, and add the TheStageSDK product. Or in Package.swift:

.package(
    url: "https://github.com/TheStageAI/AppleSDK.git",
    exact: Version(1, 7, 0)
)

Installation (Flutter / iOS)

# pubspec.yaml
dependencies:
  thestage_apple_sdk:
    git:
      url: https://github.com/TheStageAI/AppleSDK.git
      path: plugin/thestage_apple_sdk
      ref: 1.7.0

Swift

import TheStageSDK

try await TheStageAI.shared.initialize(api_token: "th_…")

let embedder = try await EmbeddingPipeline(
    engines_path: "TheStageAI/embeddinggemma-2",
    modalities: [.TEXT, .IMAGE]          // nil = every input kind
)

// Search: queries and documents use different task prompts
let query: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY)
let doc: EmbeddingResult = try await embedder.infer(text: doc_text, task: .DOCUMENT)
let score: Float = try EmbeddingPipeline.similarity(query, doc)

// Other inputs — same space
let photo: EmbeddingResult = try await embedder.infer(image: cg_image)
let product: EmbeddingResult = try await embedder.infer(inputs: [.image(cg_image), .text("red sneakers")])
// with modalities [.AUDIO] / [.VIDEO]:
//   try await embedder.infer(audio: samples, sample_rate: 44_100)
//   try await embedder.infer(video: file_url, max_frames: 8)

// Smaller vectors
let small: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY, dimension: 256)

Flutter (iOS)

await TheStageFlutterSDK.start_model(
  model_name: 'embed',
  engines_path: 'TheStageAI/embeddinggemma-2',
);

final rows = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'text': 'how to reset the router', 'task': 'query'},
);
final embedding = (rows[0]['embedding'] as List).cast<double>();

// Images, audio, video: pass a file path
final photo = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'image_path': '/path/to/photo.jpg', 'dimension': 256},
);

Inputs / outputs

Direction	Swift	Flutter key	Notes
in	`infer(text:)`	`text`	Up to 2048 tokens; longer text is cut (`truncated`)
in	`infer(image:)`	`image_path`	Any size; 256–280 tokens per image
in	`infer(audio:sample_rate:)`	`audio_path`	Mono, any rate (resampled); 25 tokens/s, first 30 s (`truncated` past that)
in	`infer(video:max_frames:)`	`video_path`, `max_frames`	1 frame/s, up to 14 frames (default), up to 140 tokens per frame
in	`infer(inputs:)`	several keys	One embedding for mixed parts (Flutter order: text, image, audio, video)
in	`task:`	`task`	`query`, `document`, `STS`, `Classification`, `Clustering`, `CodeRetrieval`, `QuestionAnswering`, `FactChecking`
in	`dimension:`	`dimension`	768 (default), 512, 256, 128
out	`embedding`	`embedding`	Unit-length vector
out	`dimension`, `embedding_space_id`	same	Compare only results with equal `embedding_space_id`
out	`token_count`, `truncated`	same	Input length read; whether anything was cut

Matryoshka sizes

Reference quality of the base model (Google model card, bf16):

Size	MTEB eng v2	MTEB Code	MIEB lite	MAEB	MMEB v2
768	68.46	78.68	65.67	49.87	59.01
512	68.41	77.24	65.34	49.76	58.38
256	67.78	76.18	64.16	49.13	56.24
128	65.68	71.41	59.97	47.28	45.65

256 is near-lossless for text; 128 costs noticeably more on images, audio and video. For large indexes, store 768-d vectors, shortlist on the first 256 values renormalized, and re-rank the shortlist on 768.

On-device latency

Release build, Neural Engine, warmed, median of 5 (guidance, not an SLA):

Input	Tokens	Apple M-series Mac	iPhone 13
Text query	16	8.1 ms	12.6 ms
Text document	368	47.6 ms	69.7 ms
Image 768x768	260	439.5 ms	538.0 ms
Audio 3 s	79	137.7 ms	227.7 ms
Audio 30 s	754	236.6 ms	382.7 ms
Video, 4 frames	494	607.1 ms	757.8 ms
Image + caption	271	439.8 ms	538.4 ms

The first load on a device prepares every engine (about two minutes on a Mac, under four on an iPhone 13, with all four inputs); later loads take seconds (7.9 s on an iPhone 13). Memory stays under 200 MB once loaded on an iPhone 13. Always measure release builds on device.

Acknowledgments

This work builds on EmbeddingGemma 2 by Google: google/embeddinggemma-2.

The base model google/embeddinggemma-2 is released under Apache-2.0. This Apple Silicon package is produced by TheStage AI; it is not an official Google release.

Model tree for TheStageAI/embeddinggemma-2

Base model

google/embeddinggemma-2

Quantized

(122)

this model

Collection including TheStageAI/embeddinggemma-2

iOS_Android_SDK

Collection

Bunch of models for deployment on iOS and macOS devices • 41 items • Updated about 17 hours ago

TheStageAI
/

embeddinggemma-2