Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.42, ONNX Runtime 1.25.0	Download
ONNX	w8a8	Universal	QAIRT 2.42, ONNX Runtime 1.25.0	Download
QNN_DLC	w8a8	Universal	QAIRT 2.45	Download
TFLITE	w8a8	Universal	QAIRT 2.45	Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

• Custom weights (e.g., fine-tuned checkpoints)
• Custom input shapes
• Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

• Model checkpoint: sam2.1_hiera_t
• Input resolution: 720p (720x1280)
• Number of parameters (encoder): 33.5M
• Model size (encoder) (float): 128 MB
• Number of parameters (decoder): 6.22M
• Model size (decoder) (float): 23.7 MB

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
decoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	3.17 ms	16 - 235 MB	NPU
decoder	ONNX	float	Snapdragon® X2 Elite	3.275 ms	196 - 196 MB	NPU
decoder	ONNX	float	Snapdragon® X Elite	6.616 ms	164 - 164 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	4.713 ms	17 - 340 MB	NPU
decoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	6.727 ms	16 - 45 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	3.493 ms	3 - 240 MB	NPU
decoder	ONNX	float	Qualcomm® QCS9075	8.35 ms	16 - 61 MB	NPU
decoder	ONNX	float	Qualcomm® QCS8750	3.493 ms	3 - 240 MB	NPU
decoder	ONNX	float	Qualcomm® QCS7181	6.616 ms	164 - 164 MB	NPU
decoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	5.281 ms	4 - 276 MB	NPU
decoder	ONNX	w8a8	Snapdragon® X2 Elite	5.446 ms	208 - 208 MB	NPU
decoder	ONNX	w8a8	Snapdragon® X Elite	12.356 ms	177 - 177 MB	NPU
decoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	9.205 ms	4 - 284 MB	NPU
decoder	ONNX	w8a8	Qualcomm® QCS6490	138.336 ms	47 - 68 MB	CPU
decoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	11.963 ms	1 - 53 MB	NPU
decoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	50.617 ms	50 - 70 MB	CPU
decoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	7.0 ms	2 - 250 MB	NPU
decoder	ONNX	w8a8	Qualcomm® QCS9075	13.715 ms	1 - 49 MB	NPU
decoder	ONNX	w8a8	Qualcomm® QCM6690	58.145 ms	45 - 68 MB	CPU
decoder	ONNX	w8a8	Qualcomm® QCS7790	50.617 ms	50 - 70 MB	CPU
decoder	ONNX	w8a8	Qualcomm® QCS8750	7.0 ms	2 - 250 MB	NPU
decoder	ONNX	w8a8	Qualcomm® QCS7181	12.356 ms	177 - 177 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	2.724 ms	4 - 190 MB	NPU
decoder	QNN_DLC	float	Snapdragon® X2 Elite	3.19 ms	16 - 16 MB	NPU
decoder	QNN_DLC	float	Snapdragon® X Elite	5.818 ms	16 - 16 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	3.733 ms	4 - 215 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS8275	12.586 ms	16 - 199 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	5.565 ms	16 - 18 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8775P	7.265 ms	16 - 231 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8650P	7.265 ms	16 - 231 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8255P	7.265 ms	16 - 231 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA7255P	12.586 ms	16 - 199 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8295P	8.889 ms	0 - 194 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS9075	7.242 ms	16 - 34 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	10.936 ms	4 - 220 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS7181	5.818 ms	16 - 16 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	0.931 ms	1 - 110 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	1.381 ms	4 - 4 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® X Elite	2.438 ms	4 - 4 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	1.413 ms	0 - 110 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS6490	9.535 ms	1 - 7 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	2.124 ms	4 - 6 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	2.694 ms	4 - 215 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® SA8295P	3.167 ms	4 - 88 MB	NPU
decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	1.082 ms	1 - 90 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS9075	2.543 ms	1 - 6 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	2.61 ms	0 - 112 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCM6690	10.537 ms	4 - 263 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS7790	2.694 ms	4 - 215 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS8750	1.082 ms	1 - 90 MB	NPU
decoder	QNN_DLC	w8a8	Qualcomm® QCS7181	2.438 ms	4 - 4 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	86.21 ms	56 - 1635 MB	NPU
encoder	ONNX	float	Snapdragon® X2 Elite	90.819 ms	200 - 200 MB	NPU
encoder	ONNX	float	Snapdragon® X Elite	159.174 ms	169 - 169 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	121.266 ms	36 - 3794 MB	NPU
encoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	161.393 ms	39 - 46 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	93.481 ms	44 - 1589 MB	NPU
encoder	ONNX	float	Qualcomm® QCS9075	215.791 ms	12 - 69 MB	NPU
encoder	ONNX	float	Qualcomm® QCS8750	93.481 ms	44 - 1589 MB	NPU
encoder	ONNX	float	Qualcomm® QCS7181	159.174 ms	169 - 169 MB	NPU
encoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	36.776 ms	14 - 1699 MB	NPU
encoder	ONNX	w8a8	Snapdragon® X2 Elite	38.896 ms	210 - 210 MB	NPU
encoder	ONNX	w8a8	Snapdragon® X Elite	85.405 ms	178 - 178 MB	NPU
encoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	58.874 ms	14 - 1938 MB	NPU
encoder	ONNX	w8a8	Qualcomm® QCS6490	2210.942 ms	519 - 542 MB	CPU
encoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	85.855 ms	12 - 109 MB	NPU
encoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	1278.774 ms	417 - 432 MB	CPU
encoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	44.977 ms	12 - 1679 MB	NPU
encoder	ONNX	w8a8	Qualcomm® QCS9075	108.831 ms	13 - 58 MB	NPU
encoder	ONNX	w8a8	Qualcomm® QCM6690	1354.249 ms	440 - 454 MB	CPU
encoder	ONNX	w8a8	Qualcomm® QCS7790	1278.774 ms	417 - 432 MB	CPU
encoder	ONNX	w8a8	Qualcomm® QCS8750	44.977 ms	12 - 1679 MB	NPU
encoder	ONNX	w8a8	Qualcomm® QCS7181	85.405 ms	178 - 178 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	90.75 ms	12 - 2375 MB	NPU
encoder	QNN_DLC	float	Snapdragon® X2 Elite	97.103 ms	12 - 12 MB	NPU
encoder	QNN_DLC	float	Snapdragon® X Elite	173.289 ms	12 - 12 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	131.252 ms	88 - 1945 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS8275	416.989 ms	1 - 1588 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	173.333 ms	12 - 15 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8775P	187.953 ms	1 - 1584 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8650P	187.953 ms	1 - 1584 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8255P	187.953 ms	1 - 1584 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA7255P	416.989 ms	1 - 1588 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8295P	266.723 ms	0 - 1572 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS9075	228.938 ms	12 - 46 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	354.948 ms	12 - 1870 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS7181	173.289 ms	12 - 12 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	35.063 ms	3 - 1756 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	37.509 ms	3 - 3 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® X Elite	82.057 ms	3 - 3 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	54.876 ms	3 - 2207 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS6490	630.421 ms	3 - 13 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	78.447 ms	3 - 1315 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	98.667 ms	3 - 2385 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® SA8295P	91.426 ms	3 - 1740 MB	NPU
encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	40.554 ms	3 - 1678 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS9075	92.282 ms	4 - 13 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	136.838 ms	4 - 2201 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCM6690	1500.794 ms	18 - 2768 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS7790	98.667 ms	3 - 2385 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS8750	40.554 ms	3 - 1678 MB	NPU
encoder	QNN_DLC	w8a8	Qualcomm® QCS7181	82.057 ms	3 - 3 MB	NPU

License

• The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

• SAM 2 Segment Anything in Images and Videos
• Source Model Implementation

Community

• Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
• For questions or feedback please reach out to us.