USFM: Universal Ultrasound Foundation Model

A transformers port of USFM (Universal Ultrasound Foundation Model). Weights converted from the official openmedlab/USFM checkpoint USFM_latest.pth to BeitModel.

Usage

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModel

processor = AutoImageProcessor.from_pretrained("Larxel/USFM")
model = AutoModel.from_pretrained("Larxel/USFM")

image = Image.open("ultrasound.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    patch_embeddings = outputs.last_hidden_state  # [1, 197, 768]

Citation

If you use this model, please credit the authors:

@article{JIAO2024103202,
  title = {USFM: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis},
  journal = {Medical Image Analysis},
  volume = {96},
  pages = {103202},
  year = {2024},
  issn = {1361-8415},
  doi = {https://doi.org/10.1016/j.media.2024.103202},
  url = {https://www.sciencedirect.com/science/article/pii/S1361841524001270},
  author = {Jing Jiao and Jin Zhou and Xiaokang Li and Menghua Xia and Yi Huang and Lihong Huang and Na Wang and Xiaofan Zhang and Shichong Zhou and Yuanyuan Wang and Yi Guo}
}